A system and method for diagnosing abnormal delay of business messages based on multi-source data
By constructing a business logic timeline and comprehensive indicator analysis independent of the physical clock, the problem of clock asynchrony in multi-channel environments was solved, enabling accurate diagnosis of data packet loss and latency anomalies, and improving the reliability and accuracy of transaction decisions.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-26
- Publication Date
- 2026-04-10
AI Technical Summary
Existing technologies cannot effectively address measurement errors caused by clock asynchrony in multi-channel environments, nor can they deeply diagnose the consistency of data at the business logic level, leading to the neglect of abnormal transaction decisions.
A business message anomaly delay diagnosis system based on multi-source data is constructed. By establishing a business logic timeline independent of the physical clock, a benchmark channel is selected by comprehensively considering packet loss rate and delay variance. Completeness, jump point time difference and continuity difference indicators are analyzed. Gradient boosting decision tree is used for classification training to generate a diagnostic report.
It eliminates analysis errors caused by clock asynchrony, accurately diagnoses data packet loss and latency anomalies, improves diagnostic accuracy and adaptability, can detect potential channel degradation trends, and ensures the reliability of transaction decisions.
Smart Images
Figure CN121418324B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of business message analysis, and particularly relates to a business message abnormal delay diagnosis system and method based on multi-source data. BACKGROUND
[0002] The transmission speed and quality of market data directly determine the yield and risk of business strategy. In order to pursue the ultimate low delay, business analysis institutions generally deploy multiple independent physical channels to receive market data at the same time, in order to ensure the continuity of service through redundant architecture.
[0003] The prior art solution is limited to isolated monitoring of a single data stream. Such a method usually only focuses on the absolute time delay of the data packet or the simple sequence number continuity, and triggers an alarm by setting a fixed static threshold. It cannot effectively deal with the measurement error caused by clock asynchronization in a multi-channel environment, and is prone to misjudgment. Secondly, it can only perceive the surface transmission delay and cannot deeply diagnose the consistency of data at the business logic level, for example, the breakage of the order and transaction chain caused by packet loss of a specific channel. Such abnormality which is more destructive to transaction decision-making is often ignored.
[0004] Therefore, the present application discloses a business message abnormal delay diagnosis system and method based on multi-source data to solve the above problems. SUMMARY
[0005] The present application aims to provide a business message abnormal delay diagnosis system and method based on multi-source data to solve the problems in the prior art.
[0006] To achieve the above object, the present application provides the following technical scheme: a business message abnormal delay diagnosis method based on multi-source data, which comprises the following steps:
[0007] S1: real-time collection of business message streams from at least two independent physical channels, analysis of message content, extraction of business identifiers generated by information source ends, analysis of data quality evaluation indexes of each physical channel, and confirmation of a reference channel based on the data quality evaluation indexes;
[0008] S2: matching and alignment of messages describing the same market event in different physical channels based on a business logic timeline, analysis of completeness indexes, jump point time differences and continuity difference indexes;
[0009] S3: forming a mixed feature pair of the completeness indexes, the jump point time differences and the continuity difference indexes, and performing classification training using the mixed feature pair with a known channel abnormal type label;
[0010] S4: screening candidate abnormal channels based on mixed features generated based on real-time running data; locating main abnormal channels causing the abnormality; and generating a diagnosis report.
[0011] According to the above scheme, in S1, the following is included:
[0012] S101: real-time collection of service message streams from at least two independent physical channels, the service message streams including transaction-by-transaction transaction messages, transaction-by-transaction order messages, and market snapshot messages; parsing the content of each message to extract a service identifier generated by an information source end; wherein for transaction-by-transaction transaction and transaction-by-transaction order messages, a corresponding message record number field is extracted; and for market snapshot messages, a corresponding output packet sequence number field is extracted;
[0013] S102: globally sorting and mapping the service identifiers extracted from all physical channels according to numerical size using a global sorting algorithm to synthesize a continuous service logical time axis; each point in the service logical time axis is defined by a unique service identifier;
[0014] S103: analyzing the average packet loss rate and delay variance of each physical channel; generating a data quality evaluation index by weighted fusion of the normalized average packet loss rate and delay variance; marking the physical channel corresponding to the minimum data quality evaluation index as a reference channel, and marking the physical channels other than the reference channel as non-reference channels.
[0015] The application constructs a service logical time axis independent of a physical clock, eliminates analysis errors caused by clock asynchronization between different physical channels, and dynamically elects a channel with optimal quality as a reference by comprehensively considering the two key indicators of packet loss rate and delay variance, which makes the entire diagnosis system adaptive, can always measure other channels with the optimal channel, ensures the reliability and fairness of the diagnosis reference, and avoids misjudgment caused by possible degradation of the fixed reference channel.
[0016] According to the above scheme, in S2, the following is included:
[0017] S201: performing completeness checking on a message chain with strong causal relationship on the established service logical time axis; the strong causal relationship includes but is not limited to the causal relationship between order messages and transaction confirmation messages;
[0018] On the service logical time axis, when a result message is received in a non-reference channel, the existence of a prerequisite message corresponding to the result message in the historical data of the reference channel is verified, and a completeness index is analyzed;
[0019] S202: Monitor the key state fields in the market snapshot message, including but not limited to the latest transaction price and cumulative transaction volume; when detecting that the key state field jumps beyond the preset threshold, record the logical time point of the jump point first appearing in the reference channel and each non-reference channel data stream; calculate the jump point time difference of each non-reference channel relative to the reference channel; the jump point time difference is equal to the logical time of the jump point in the reference channel minus the logical time of the jump point in the non-reference channel;
[0020] S203: Perform continuity analysis on the service identifiers of each channel, and count the number of sequence discontinuities of each non-reference channel; the sequence discontinuity includes but is not limited to identifier jump, repetition or absence; divide the difference between the sequence discontinuity number of each non-reference channel and the sequence discontinuity number of the reference channel by the total number of messages during the statistics, and record it as the continuity difference index.
[0021] The application can directly expose and quantify the serious problem of data packet loss by checking the completeness of the causal chain (such as commission-deal), accurately measure the synchronization of each channel response for the most sensitive price / transaction volume jump event in the market, and effectively capture the delay anomaly of the channel when transmitting key information. Since the jump point usually corresponds to an important trading opportunity, this feature is directly related to the performance loss of high-frequency trading and other businesses, and has high business significance. A channel with a significantly higher sequence discontinuity number than the reference channel indicates that it has unstable and large jitter problems. This feature is an important indicator for diagnosing the overall health of the channel, and can find potential problems that have not yet caused serious delay or packet loss but have shown a deterioration trend.
[0022] According to the above scheme, in S3, the following contents are included:
[0023] S301: Combine the completeness index, jump point time difference and continuity difference index into a mixed feature pair; normalize the mixed feature pair;
[0024] S302: Use gradient boosting decision tree as the basic classifier, and use the mixed feature pair with known channel anomaly type label for training, the anomaly type label includes but is not limited to: channel delay anomaly, data packet loss anomaly and sequence disorder anomaly; k-fold cross-validation is used to optimize parameters during training.
[0025] According to the above scheme, in S4, the following contents are included:
[0026] S401: bring the real-time running data into the trained base classifier to generate the abnormal occurrence probability and the abnormal type of each physical channel; filter out the physical channels with the abnormal occurrence probability exceeding the preset confidence threshold to list as the candidate abnormal channels; and check the original consistency features directly associated with the candidate abnormal channels, the original consistency features at least including the completeness index, the time difference of the jump point and the continuity difference index;
[0027] comprehensive decision based on the predefined evidence decision rule set to confirm the main abnormal channel;
[0028] S402: integrate the abnormal event identifier, the timestamp, the main abnormal channel identifier, the abnormal type label and the triggered original consistency features into a diagnosis report and send the diagnosis report to the administrator.
[0029] The completeness evidence is prioritized: the business priority is embodied, and data loss is more serious than delay; this rule ensures that the most serious problem can be located preferentially and accurately. The time delay evidence and the continuity evidence: provide clear criteria for diagnosing delay and stability problems without data loss, avoid the black box misjudgment of the model, and make the diagnosis conclusion very reliable.
[0030] Another aspect of the application provides a business message abnormal delay diagnosis system based on multi-source data, which is applied to the business message abnormal delay diagnosis method based on multi-source data.
[0031] The data acquisition channel analysis module is used to collect the business message stream from at least two independent physical channels in real time, analyze the message content, extract the business identifier generated by the information source end, analyze the data quality evaluation index of each physical channel, and confirm the reference channel based on the data quality evaluation index.
[0032] The feature analysis module is used to match and align the messages describing the same market event in different physical channels based on the business logic timeline, analyze the completeness index, the time difference of the jump point and the continuity difference index.
[0033] The classification training module is used to form a mixed feature pair by combining the completeness index, the time difference of the jump point and the continuity difference index; and perform classification training using the mixed feature pair with the known channel abnormal type label.
[0034] The decision analysis module is used to screen the candidate abnormal channels based on the mixed feature pairs generated by the real-time running data, locate the main abnormal channel causing the abnormality, and generate a diagnosis report.
[0035] According to the scheme, the data collection channel analysis module comprises a data collection unit and a channel analysis unit.
[0036] The data collection unit is configured to collect service message streams from at least two independent physical channels in real time, parse the content of each message, extract a service identifier generated by an information source end, perform global sorting and mapping on the service identifiers extracted from all physical channels according to a global sorting algorithm according to the numerical value, and synthesize a continuous service logical time axis.
[0037] The channel analysis unit is configured to analyze the average packet loss rate and delay variance of each physical channel, perform weighted fusion on the normalized average packet loss rate and delay variance to generate a data quality evaluation index, and mark the physical channel corresponding to the minimum data quality evaluation index as a reference channel and mark the physical channels other than the reference channel as non-reference channels.
[0038] According to the scheme, the feature analysis module comprises a completeness index analysis unit, a jump point time difference analysis unit, and a continuity difference index analysis unit.
[0039] The completeness index analysis unit is configured to perform completeness checking on a message chain having a strong causal relationship on the established service logical time axis, verify the existence of a prerequisite message corresponding to a result message received on a non-reference channel in historical data of a reference channel on the service logical time axis, and analyze a completeness index.
[0040] The jump point time difference analysis unit is configured to monitor a key state field in a market snapshot message, record a logical time point at which a jump point first appears in a data stream of a reference channel and each non-reference channel when a jump of the key state field beyond a preset threshold is detected, and calculate a jump point time difference of each non-reference channel relative to the reference channel.
[0041] The continuity difference index analysis unit is configured to perform continuity analysis on service identifiers of each channel, count the number of sequence discontinuities of each non-reference channel, and divide the difference between the number of sequence discontinuities of each non-reference channel and the number of sequence discontinuities of the reference channel by the total number of messages during the counting period to obtain a continuity difference index.
[0042] According to the scheme, the classification training module comprises a feature fusion unit and a training unit.
[0043] The feature fusion unit is configured to form a mixed feature pair by combining the completeness index, the jump point time difference, and the continuity difference index, and perform normalization processing on the mixed feature pair.
[0044] The training unit is configured to use a gradient boosting decision tree as a basic classifier, and train using a mixed feature pair injected with a known channel anomaly type label, and use k-fold cross-validation to optimize parameters during the training process.
[0045] According to the above scheme, the arbitration analysis module comprises a main abnormal channel analysis unit and a report generation unit.
[0046] The main abnormal channel analysis unit is configured to input real-time running data into the trained basic classifier to generate an abnormal occurrence probability and an abnormal type of each physical channel, filter out a physical channel with an abnormal occurrence probability exceeding a preset confidence threshold to list as a candidate abnormal channel, and check original consistency features directly associated with the candidate abnormal channel to comprehensively arbitrate and confirm a main abnormal channel based on a predefined evidence arbitration rule set.
[0047] The report generation unit is configured to integrate an abnormal event identification, a timestamp, a main abnormal channel identification, an abnormal type label and triggered original consistency features into a diagnostic report and send the diagnostic report to an administrator.
[0048] Compared with the prior art, the present application has the beneficial effects that: the present application constructs a business logic time axis independent of a physical clock, eliminates analysis errors caused by clock asynchronization between different physical channels, dynamically elects a channel with the best quality as a reference by comprehensively considering packet loss rate and delay variance, which makes the entire diagnostic system adaptive and capable of measuring other channels with the optimal channel to ensure the reliability and fairness of the diagnostic reference and avoid misjudgment caused by possible degradation of the fixed reference channel. The present application can directly expose and quantify the serious problem of data packet loss by checking the completeness of the cause-effect chain, accurately measure the synchronization of channel responses for the most sensitive price / transaction volume jump events in the market, and the feature can effectively capture delay abnormalities of the channel during key information transmission. Since the jump point usually corresponds to an important trading opportunity, the feature is directly related to the performance loss of high-frequency trading and other businesses and has high business significance. A channel with a sequence discontinuity number significantly higher than the reference indicates that the channel has problems of instability and large jitter. The feature is an important indicator of the overall health of the channel and can find potential problems that have not caused serious delay or packet loss but have shown a degradation trend. The present application gives priority to completeness evidence: reflects business priority, data loss is a more serious problem than delay, and this rule ensures that the most serious problem can be located preferentially and accurately. The time delay evidence and the continuity evidence: provide clear criteria for diagnosing delay and stability problems in the absence of data loss, avoid black box misjudgment of the model, and improve the accuracy of data analysis. BRIEF DESCRIPTION OF DRAWINGS
[0049] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the application. In the drawings:
[0050] Figure 1 A flowchart of a service message abnormal delay diagnosis method based on multi-source data according to the present application;
[0051] Figure 2 A structural diagram of a service message abnormal delay diagnosis system based on multi-source data according to the present application. DETAILED DESCRIPTION
[0052] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.
[0053] Please refer to Figure 1 The present application provides a technical solution: a service message abnormal delay diagnosis method based on multi-source data, which comprises the following steps:
[0054] S1: collecting service message streams from at least two independent physical channels in real time, analyzing message content, extracting service identifiers generated by information source ends, analyzing data quality evaluation indexes of each physical channel, and confirming a reference channel based on the data quality evaluation indexes;
[0055] In S1, the following content is included:
[0056] S101: collecting service message streams from at least two independent physical channels in real time, wherein the service message streams comprise transaction-by-transaction message, order-by-order message and market snapshot message; analyzing the content of each message, and extracting service identifiers generated by information source ends; wherein for the transaction-by-transaction message and the order-by-order message, the corresponding message record number field is extracted; and for the market snapshot message, the corresponding output packet serial number field is extracted;
[0057] S102: performing global sorting and mapping on the service identifiers extracted from all physical channels by using a global sorting algorithm according to numerical value size, to synthesize a continuous service logical time axis; each point in the service logical time axis is defined by a unique service identifier;
[0058] S103: Analyze the average packet loss rate and delay variance of each physical channel; weight and fuse the normalized average packet loss rate and delay variance to generate a data quality evaluation index; mark the physical channel corresponding to the minimum data quality evaluation index as the reference channel, and mark the physical channels other than the reference channel as non-reference channels.
[0059] S2: Based on the business logic timeline, match and align the messages describing the same market event in different physical channels, analyze the completeness index, jump point time difference, and continuity difference index;
[0060] In S2, the following content is included:
[0061] S201: On the established business logic timeline, perform completeness checking on the message chain with strong causal relationship; strong causal relationship includes but is not limited to the causal relationship between the order message and the trade confirmation message;
[0062] On the business logic timeline, when one of the result messages is received on the non-reference channel, verify the existence of the corresponding prerequisite message in the historical data of the reference channel, and analyze the completeness index;
[0063] Embodiment 1: In this embodiment, the result message is a trade confirmation message, and the corresponding prerequisite message is an order message; for order execution events, when a trade confirmation message appears on one of the non-reference channels, query the corresponding order message in the reference channel; record the ratio of the number of queried corresponding order messages to the pre-set number of order messages as the completeness index;
[0064] S202: Monitor the key state fields in the market snapshot message, including but not limited to the latest trade price and cumulative trade volume; when a jump of the key state field beyond the pre-set threshold is detected, record the logical time point at which the jump point first appears in the reference channel and each non-reference channel data stream; calculate the jump point time difference of each non-reference channel relative to the reference channel; the jump point time difference is equal to the logical time of the jump point in the reference channel minus the logical time of the jump point in the non-reference channel;
[0065] S203: Perform continuity analysis on the business identifiers of each channel, and count the number of sequence discontinuities that occur in each non-reference channel; sequence discontinuity includes but is not limited to identifier jump, repetition or absence; divide the difference between the number of sequence discontinuities of each non-reference channel and the number of sequence discontinuities of the reference channel by the total number of messages during the statistics period to obtain the continuity difference index.
[0066] S3: Combine the completeness index, jump point time difference, and continuity difference index into a mixed feature pair; use the mixed feature pair with known channel anomaly type label for classification training;
[0067] In S3, the following is included:
[0068] S301: A complete index, a time difference of a jump point, and a continuity difference index are combined into a mixed feature pair; the mixed feature pair is normalized;
[0069] S302: Gradient boosting decision trees are used as a basic classifier, and the mixed feature pair with a known channel anomaly type label is used for training. The anomaly type label includes but is not limited to: channel delay anomaly, data packet loss anomaly, and sequence disorder anomaly. In the training process, k-fold cross-validation is used to optimize parameters.
[0070] S4: The mixed feature pair generated based on real-time running data is used to screen candidate abnormal channels; the main abnormal channel causing the anomaly is located; and a diagnosis report is generated.
[0071] In S4, the following is included:
[0072] S401: Real-time running data is input into the trained basic classifier to generate an anomaly occurrence probability and an anomaly type of each physical channel. Physical channels with an anomaly occurrence probability exceeding a pre-set confidence threshold are screened out as candidate abnormal channels. The original consistency features directly associated with the candidate abnormal channels are checked, and the original consistency features at least include a completeness index, a time difference of a jump point, and a continuity difference index;
[0073] Comprehensive decision is made based on a pre-defined evidence decision rule set to confirm the main abnormal channel;
[0074] Embodiment 2: In this embodiment, the evidence decision rule set includes completeness evidence, timing delay evidence, and continuity evidence.
[0075] The completeness evidence indicates that when the completeness index is zero, the corresponding candidate abnormal channel is marked as the main abnormal channel.
[0076] The timing delay evidence indicates that if there is no completeness evidence, the time difference of the jump point is detected. For the same jump event, if the time difference of the jump point of one of the candidate abnormal channels exceeds a pre-set delay threshold, the corresponding candidate abnormal channel is marked as the main abnormal channel.
[0077] The continuity evidence indicates that if there is no completeness evidence and timing delay evidence, the continuity difference index exceeding a pre-set continuity difference threshold is detected, and the corresponding candidate abnormal channel is marked as the main abnormal channel.
[0078] S402: An anomaly event identifier, a timestamp, a main abnormal channel identifier, an anomaly type label, and triggered original consistency features are integrated into a diagnosis report and sent to an administrator.
[0079] Please refer toFigure 2 The application provides a technical solution: a business message abnormal delay diagnosis system based on multi-source data, which comprises a data acquisition channel analysis module, a feature analysis module, a classification training module and a decision analysis module.
[0080] The data acquisition channel analysis module is used for acquiring business message streams in real time from at least two independent physical channels, analyzing message content, extracting business identifiers generated by information source ends, analyzing data quality evaluation indexes of the respective physical channels, and confirming a reference channel based on the data quality evaluation indexes.
[0081] The feature analysis module is used for matching and aligning messages describing the same market event in different physical channels based on a business logic timeline, analyzing completeness indexes, jump point time difference indexes and continuity difference indexes.
[0082] The classification training module is used for grouping the completeness indexes, the jump point time difference indexes and the continuity difference indexes into mixed feature pairs, and performing classification training by using the mixed feature pairs with known channel abnormal type labels.
[0083] The decision analysis module is used for screening candidate abnormal channels based on mixed feature pairs generated by real-time running data, locating main abnormal channels causing the abnormality, and generating a diagnosis report.
[0084] The data acquisition channel analysis module comprises a data acquisition unit and a channel analysis unit.
[0085] The data acquisition unit is used for acquiring business message streams in real time from at least two independent physical channels, analyzing the content of each message, extracting business identifiers generated by information source ends, performing global sorting and mapping on the business identifiers extracted from all the physical channels by using a global sorting algorithm according to numerical values, and synthesizing a continuous business logic timeline.
[0086] The channel analysis unit is used for analyzing the average packet loss rate and the delay variance of each physical channel, performing weighted fusion on the normalized average packet loss rate and the delay variance to generate a data quality evaluation index, and marking the physical channel corresponding to the minimum data quality evaluation index as a reference channel and marking the physical channels other than the reference channel as non-reference channels.
[0087] The feature analysis module comprises a completeness index analysis unit, a jump point time difference analysis unit and a continuity difference index analysis unit.
[0088] The completeness index analysis unit is used for performing completeness checking on a message chain with strong causal relationship on the established business logic timeline, verifying the existence of a prerequisite message corresponding to a result message received in a non-reference channel on the business logic timeline, and analyzing a completeness index.
[0089] The jump point time difference analysis unit is configured to monitor a key state field in the market snapshot message, and when detecting that the key state field jumps beyond a preset threshold, record a logical time point at which the jump point first appears in the reference channel and each non-reference channel data stream; and calculate a jump point time difference of each non-reference channel relative to the reference channel.
[0090] The continuity difference index analysis unit is configured to perform continuity analysis on the service identifiers of each channel, and count the number of sequence discontinuities of each non-reference channel; and divide the difference between the number of sequence discontinuities of each non-reference channel and the number of sequence discontinuities of the reference channel by the total number of messages during the statistics period, and record the result as a continuity difference index.
[0091] The classification training module includes a feature fusion unit and a training unit.
[0092] The feature fusion unit is configured to form a mixed feature pair by combining the completeness index, the jump point time difference and the continuity difference index; and perform normalization processing on the mixed feature pair.
[0093] The training unit is configured to use a gradient boosting decision tree as a basic classifier, and use the mixed feature pair with the known channel abnormal type label to perform training, and use k-fold cross-validation to optimize parameters during the training.
[0094] The ruling analysis module includes a main abnormal channel analysis unit and a report generation unit.
[0095] The main abnormal channel analysis unit is configured to bring real-time running data into the trained basic classifier to generate an abnormal occurrence probability and an abnormal type of each physical channel; filter out a physical channel with an abnormal occurrence probability exceeding a preset confidence threshold, and list the physical channel as a candidate abnormal channel; and check original consistency features directly associated with the candidate abnormal channel, and comprehensively rule and confirm a main abnormal channel based on a predefined evidence ruling rule set.
[0096] The report generation unit is configured to integrate an abnormal event identifier, a timestamp, a main abnormal channel identifier, an abnormal type label and triggered original consistency features into a diagnostic report, and send the diagnostic report to an administrator.
[0097] It should be noted that, in this document, relational terms such as first and second and the like can be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus.
[0098] It will be apparent to those skilled in the art that the application is not limited to the details of the above-exemplified embodiments and that the present application can be implemented in other particular forms without departing from the spirit or essential characteristics of the present application. The presently disclosed embodiments are, therefore, to be considered in all respects as illustrative and not restrictive, the scope of the application being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. No reference herein to any prior art is to be taken as an admission that the present application is not entitled to antedate such prior art by virtue of prior application. No reference to an item of art implemented prior to the filing date of this application is to be construed as an admission that the present application is not entitled to an exception under 35 U.S.C. § 103(a) for prior art activity as defined in 35 U.S.C. § 102 (d).
Claims
1. A method for diagnosing abnormal delay of a service message based on multi-source data, characterized in that, The method comprises the following steps: S1: Real-time acquisition of service message streams from at least two independent physical channels, analysis of message content, extraction of service identifiers generated by information source ends; analysis of data quality evaluation indexes of each physical channel; Confirm the reference channel based on the data quality evaluation index; In S1, the following contents are included: S101: Real-time acquisition of service message streams from at least two independent physical channels, the service message streams include transaction-by-transaction transaction messages, transaction-by-transaction order messages and market snapshot messages; analyze the content of each message and extract the service identifier generated by the information source end; for transaction-by-transaction transaction and transaction-by-transaction order messages, the corresponding message record number field is extracted; for market snapshot messages, the corresponding output packet sequence number field is extracted; S102: The service identifiers extracted from all physical channels are globally sorted and mapped according to the numerical value by using a global sorting algorithm to synthesize a continuous service logical time axis; each point in the service logical time axis is defined by a unique service identifier; S103: Analyze the average packet loss rate and delay variance of each physical channel; generate a data quality evaluation index by weighting and fusing the normalized average packet loss rate and delay variance; mark the physical channel corresponding to the minimum data quality evaluation index as the reference channel, and mark the physical channels other than the reference channel as non-reference channels; S2: Based on the service logical time axis, match and align the messages describing the same market event in different physical channels, analyze the completeness index, jump point time difference and continuity difference index; In S2, the following contents are included: S201: On the established service logical time axis, perform completeness check on the message chain with strong causal relationship; the strong causal relationship includes but is not limited to the causal relationship between order messages and transaction confirmation messages; On the service logical time axis, when one of the result messages is received in the non-reference channel, verify the existence of the corresponding prerequisite message in the historical data of the reference channel, and analyze the completeness index; S202: Monitor the key state fields in the market snapshot message, including but not limited to the latest transaction price and cumulative transaction volume; when a jump of the key state field beyond a preset threshold is detected, record the logical time point at which the jump point first appears in the reference channel and each non-reference channel data stream; calculate the jump point time difference of each non-reference channel relative to the reference channel; the jump point time difference is equal to the logical time of the jump point in the reference channel minus the logical time of the jump point in the non-reference channel; S203: Perform continuity analysis on the service identifiers of each channel, and count the number of sequence discontinuities in each non-reference channel; the sequence discontinuity includes but is not limited to identifier jump, repetition or absence; divide the difference between the sequence discontinuity number of each non-reference channel and the sequence discontinuity number of the reference channel by the total number of messages during the statistics, and record it as the continuity difference index; S3: Combine the completeness index, jump point time difference and continuity difference index into a mixed feature pair; use the mixed feature pair with known channel abnormal type label for classification training; S4: screening candidate abnormal channels based on hybrid feature pairs generated from real-time running data; locating the main abnormal channel causing the abnormality; and generating a diagnosis report. 2.The method of claim 1, wherein the method further comprises: In S3, the following is included: S301: forming a hybrid feature pair from the completeness index, the time difference of the jump point, and the continuity difference index; and normalizing the hybrid feature pair; S302: using a gradient boosting decision tree as a basic classifier, training the hybrid feature pair with a known channel abnormal type label injected, the abnormal type label including but not limited to a channel delay abnormality, a data packet loss abnormality, and a sequence disorder abnormality; and using k-fold cross-validation to optimize parameters during the training process. 3.The method of claim 2, wherein: In S4, the following is included: S401: inputting real-time running data into the trained basic classifier to generate an abnormal occurrence probability and an abnormal type of each physical channel; screening a physical channel with an abnormal occurrence probability exceeding a pre-set confidence threshold to list as a candidate abnormal channel; and checking original consistency features directly associated with the candidate abnormal channel, the original consistency features including at least the completeness index, the time difference of the jump point, and the continuity difference index; comprehensively determining a main abnormal channel based on a pre-defined evidence decision rule set; S402: integrating an abnormal event identifier, a timestamp, a main abnormal channel identifier, an abnormal type label, and triggered original consistency features into a diagnosis report and sending the diagnosis report to an administrator.
4. A system for diagnosing abnormal delay of a service message based on multi-source data, the system being applied to implement the method for diagnosing abnormal delay of a service message based on multi-source data according to any one of claims 1-3, characterized in that, The system includes a data collection channel analysis module, a feature analysis module, a classification training module, and a decision analysis module; The data collection channel analysis module is configured to collect business message streams from at least two independent physical channels in real time, parse the content of each message, extract a business identifier generated by an information source end, analyze data quality evaluation indexes of each physical channel, and determine a reference channel based on the data quality evaluation indexes. The feature analysis module is configured to match and align messages describing the same market event in different physical channels based on a business logic timeline, analyze a completeness index, a time difference of a jump point, and a continuity difference index. The classification training module is configured to form a hybrid feature pair from the completeness index, the time difference of the jump point, and the continuity difference index, and perform classification training on the hybrid feature pair with a known channel abnormal type label injected. The decision analysis module is configured to screen candidate abnormal channels based on hybrid feature pairs generated from real-time running data, locate a main abnormal channel causing the abnormality, and generate a diagnosis report.
5. The system for diagnosing abnormal delay of business message based on multi-source data according to claim 4, characterized in that: The data collection channel analysis module includes a data collection unit and a channel analysis unit. The data collection unit is configured to collect business message streams from at least two independent physical channels in real time, parse the content of each message, extract a business identifier generated by an information source end, perform global sorting and mapping on the business identifiers extracted from all physical channels based on a global sorting algorithm, and synthesize a continuous business logic timeline. The channel analysis unit is configured to analyze packet average loss rate and delay variance of each physical channel, to generate a data quality evaluation index by weighted fusion of the normalized packet average loss rate and delay variance, and to mark a physical channel corresponding to a minimum data quality evaluation index as a reference channel and mark physical channels other than the reference channel as non-reference channels.
6. The system for diagnosing abnormal delay of business message based on multi-source data according to claim 4, characterized in that: The feature analysis module comprises a completeness index analysis unit, a jump point time difference analysis unit, and a continuity difference index analysis unit. The completeness index analysis unit is configured to perform completeness checking on a message chain having a strong causal relationship on an established business logic timeline, to verify existence of a prerequisite message in historical data of the reference channel when a result message is received on a non-reference channel, and to analyze a completeness index. The jump point time difference analysis unit is configured to monitor a key state field in a market snapshot message, to record a logical time point at which a jump point first appears in the reference channel and each non-reference channel data stream when a jump of the key state field beyond a preset threshold is detected, and to calculate a jump point time difference of each non-reference channel relative to the reference channel. The continuity difference index analysis unit is configured to perform continuity analysis on a business identifier of each channel, and to count a number of sequence discontinuities of each non-reference channel. The continuity difference index analysis unit is configured to perform continuity analysis on a business identifier of each channel, and to count a number of sequence discontinuities of each non-reference channel. The classification training module comprises a feature fusion unit and a training unit.
7. The system for diagnosing abnormal delay of business message based on multi-source data according to claim 4, characterized in that: The feature fusion unit is configured to form a mixed feature pair by combining the completeness index, the jump point time difference, and the continuity difference index, and to perform normalization processing on the mixed feature pair. The training unit is configured to use a gradient boosting decision tree as a basic classifier, to train the mixed feature pair with a known channel abnormality type label injected, and to optimize parameters in the training process by using k-fold cross-validation. The ruling analysis module comprises a main abnormal channel analysis unit and a report generation unit.
8. The system for diagnosing abnormal delay of business message based on multi-source data according to claim 4, characterized in that: The main abnormal channel analysis unit is configured to input real-time running data into the trained basic classifier to generate an abnormality occurrence probability and an abnormality type of each physical channel, to screen out a physical channel having an abnormality occurrence probability exceeding a preset confidence threshold to list as a candidate abnormal channel, to check original consistency features directly associated with the candidate abnormal channel, and to comprehensively rule and confirm a main abnormal channel based on a predefined evidence ruling rule set. The report generation unit is configured to integrate an abnormal event identifier, a timestamp, a main abnormal channel identifier, an abnormality type label, and triggered original consistency features into a diagnosis report and send the diagnosis report to an administrator.
Citation Information
Patent Citations
Network security method and system for realizing fault prediction based on intelligent analysis
CN115834141A
Time delay determination method and device and intelligent driving equipment
CN118200199A