Anomaly analysis methods, devices, equipment, storage media, and products

Abstract

This application discloses an anomaly analysis method, apparatus, device, storage medium, and product, relating to the field of anomaly analysis technology. The method extracts features from a pre-acquired HTTP network data stream according to a pre-configured communication feature extraction strategy to obtain HTTP communication feature data packets, which contain request association information. Based on the request association information, target anomaly analysis parameters in several of the HTTP communication feature data packets are parsed to perform system anomaly analysis. By extracting feature data from the HTTP network data stream using the above scheme, regardless of whether the fault occurs at the request sending end, the request receiving end, or other important links in the network transmission link, the location of the fault can be accurately located and anomaly analysis can be performed, thereby improving the accuracy and efficiency of fault diagnosis.

Description

Technical Field

[0001] This application relates to the field of anomaly analysis technology, and in particular to anomaly analysis method, apparatus, equipment, storage medium, and product. Background Technology

[0002] In modern software architecture, distributed microservices have become the mainstream approach for building large and complex systems. This architecture improves the scalability and reliability of the system by distributing different service nodes. However, as the dependencies between services increase, tracking and monitoring the execution of requests on each node becomes complex. Especially when a failure occurs, accurately locating the problem becomes a major challenge for system operation and maintenance. To address this challenge, distributed tracing systems are widely used. They can capture and record the performance data of requests on each service node and display this data through a visual interface to help developers and operations personnel understand the system's behavior.

[0003] While distributed link tracing systems offer some capabilities for request tracking and fault monitoring, existing technologies suffer from significant shortcomings in accuracy and ease of use. For example, traditional systems often only monitor the call latency between service nodes, failing to delve into the specific network transmission stages to identify the exact point of failure. Furthermore, when latency issues arise in the service call chain, traditional APM systems cannot accurately pinpoint whether the problem stems from an abnormal write request on the server side, an abnormal request reception on the receiver side, or a problem in a specific link of the network transmission. These limitations force operations personnel to perform tedious manual troubleshooting when facing complex faults, especially when the transmission path is long or the failure frequency is low, making the troubleshooting process exceptionally time-consuming and inefficient.

[0004] Furthermore, existing distributed tracing systems typically require embedding specific monitoring code within application code or using specific frameworks. This not only increases the complexity of system development and maintenance but also makes it difficult to integrate some purchased modules or services, leading to tracing interruptions and decreased location accuracy. These issues indicate that although distributed tracing systems theoretically provide effective monitoring methods, they still face challenges of insufficient accuracy and excessive invasiveness in practical applications.

[0005] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0006] The main objective of this application is to provide an anomaly analysis method, apparatus, device, storage medium, and computer program product, which aims to improve the accuracy and efficiency of fault diagnosis in distributed systems.

[0007] To achieve the above objectives, this application proposes an anomaly analysis method, the method comprising:

[0008] Based on a pre-configured communication feature extraction strategy, features are extracted from the pre-acquired HTTP network data stream to obtain an HTTP communication feature data packet, which contains request association information.

[0009] Based on the request association information, target anomaly analysis parameters in several HTTP communication feature data packets are parsed to perform system anomaly analysis based on the target anomaly analysis parameters.

[0010] In one embodiment, the HTTP network data stream includes an HTTP communication session record and a data packet buffer. The step of extracting features from the pre-acquired HTTP network data stream according to a pre-configured communication feature extraction strategy to obtain HTTP communication feature data packets includes:

[0011] Extract network packets from the HTTP network data stream to obtain TCP packets by unpacking the network packets, wherein the network packets include HTTP communication characteristic data;

[0012] If the TCP message is a data packet, then according to the preset data packet processing strategy, the network data packet is stored in the HTTP communication session record or the data packet buffer.

[0013] If the TCP message is an end packet or a reset packet, then based on the TCP quadruple corresponding to the data packet, the network data packet is stored in the HTTP communication session record;

[0014] When the preset record output conditions are met, the HTTP communication feature data in the HTTP communication session record is output, so as to generate the HTTP communication feature data packet based on the communication feature data.

[0015] In one embodiment, the step of storing the network data packet in the HTTP communication session record or the data packet buffer according to a preset data packet processing strategy if the TCP packet is a data packet includes:

[0016] If the TCP message is a data packet, then obtain the TCP quadruple corresponding to the data packet and sort it to obtain the sorted TCP quadruple;

[0017] Based on the sorted TCP quadruple, query whether the key-value pair of the HTTP network data stream exists in the preset two-column set;

[0018] If a key-value pair of the HTTP network data stream exists, the network data packet is stored in the HTTP communication session record or the data packet buffer according to the data packet type and the attribute information of the HTTP network data stream.

[0019] In one embodiment, the HTTP communication session record includes a first HTTP communication session record and a second HTTP communication session record. The step of obtaining HTTP communication feature data from the HTTP communication session record when a preset record output condition is met, and generating the HTTP communication feature data packet based on the communication feature data, includes:

[0020] Obtain the sequence number and acknowledgment number of the network data packet, as well as the sequence number and acknowledgment number of the existing response data packet in the second HTTP communication session record;

[0021] If the sequence number and acknowledgment number of the network data packet are after the sequence number and acknowledgment number of the existing response data packet, then a data packet splicing payload is generated based on the target data packet in the first HTTP communication session record.

[0022] Identify and acquire HTTP communication feature data in the data packet splicing payload;

[0023] The HTTP communication feature data packet is generated based on the HTTP communication feature data and the pre-acquired metadata information.

[0024] In one embodiment, the method further includes:

[0025] Iterate through the predefined two-column set;

[0026] Determine whether the current silence duration of the current HTTP network data stream in the dual-column set exceeds a preset silence duration threshold;

[0027] If the current silence duration exceeds a preset silence duration threshold, then the target key-value pair of the current HTTP network data stream in the two-column set is obtained;

[0028] The cached data in the current HTTP network data stream is obtained based on the target key-value pair and stored in a preset HTTP communication session record.

[0029] In one embodiment, the request association information includes a session record ID, and the step of parsing target anomaly analysis parameters from several HTTP communication feature data packets based on the request association information, and performing system anomaly analysis based on the target anomaly analysis parameters, includes:

[0030] Among the HTTP communication feature data packets, the feature data packets with the same session record ID are identified as target data packets, and the target data packets contain request data packets and response data packets;

[0031] Based on the data packet parameter information of the request data packet and the response data packet, the target anomaly analysis parameters are calculated. The target anomaly analysis parameters include at least one of the following: processing delay, packet sending speed, message interval time, transmission time, message capacity, and session end type.

[0032] The system anomaly analysis results are obtained by matching the target anomaly analysis parameters with the preset fault analysis rules.

[0033] Furthermore, to achieve the above objectives, this application also proposes an anomaly analysis device, which includes:

[0034] The extraction module is used to extract features from the pre-acquired HTTP network data stream according to a pre-configured communication feature extraction strategy to obtain an HTTP communication feature data packet, wherein the communication feature data packet contains request association information;

[0035] The parsing module is used to parse the target anomaly analysis parameters in several HTTP communication feature data packets according to the request association information, so as to perform system anomaly analysis based on the target anomaly analysis parameters.

[0036] In addition, to achieve the above objectives, this application also proposes an anomaly analysis device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the anomaly analysis method as described above.

[0037] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and which, when executed by a processor, implements the steps of the anomaly analysis method described above.

[0038] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the anomaly analysis method described above.

[0039] One or more technical solutions proposed in this application extract features from pre-acquired HTTP network data streams according to a pre-configured communication feature extraction strategy to obtain HTTP communication feature data packets, which contain request association information. Based on the request association information, target anomaly analysis parameters in several HTTP communication feature data packets are parsed to perform system anomaly analysis. By extracting feature data from the HTTP network data stream through the above solutions, the location of the fault can be accurately located and anomaly analysis can be performed, regardless of whether the fault occurs at the request sending end, the request receiving end, or other important links in the network transmission link, thereby improving the accuracy and efficiency of fault diagnosis. Attached Figure Description

[0040] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0041] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a flowchart illustrating an embodiment of the anomaly analysis method of this application.

[0043] Figure 2 This is a schematic diagram of the architecture of an anomaly analysis system provided in an embodiment of this application;

[0044] Figure 3 This is a flowchart illustrating Embodiment 2 of the anomaly analysis method of this application;

[0045] Figure 4 This application provides a schematic diagram of the overall data packet processing flow.

[0046] Figure 5 This is a schematic diagram of a data packet processing sub-process provided in an embodiment of this application;

[0047] Figure 6 This is a schematic diagram of a data packet processing sub-process in a scenario where the data packet payload is not empty and the data stream does not exist, provided in an embodiment of this application.

[0048] Figure 7 This is a schematic diagram of a processing sub-process in a scenario where the data packet payload is not empty and a data stream exists, provided in an embodiment of this application.

[0049] Figure 8A schematic diagram of a request cache archiving subprocess provided in an embodiment of this application;

[0050] Figure 9 This is a flowchart illustrating Embodiment 3 of the anomaly analysis method of this application;

[0051] Figure 10 A schematic diagram of a network data stream expiration check and processing flow provided in an embodiment of this application;

[0052] Figure 11 This is a schematic diagram of the module structure of the anomaly analysis device according to an embodiment of this application;

[0053] Figure 12 This is a schematic diagram of the device structure of the hardware operating environment involved in the anomaly analysis method in this application embodiment.

[0054] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0055] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0056] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0057] The main solution of this application embodiment is to capture HTTP network data streams at important nodes in the network transmission path of inter-service communication in a distributed system, such as Layer 4 load balancers, Kubernetes Ingress, SDN network entry and exit points, and servers where services reside. Then, a set of HTTP communication feature extraction strategies is used to extract a series of feature data from the network data streams. Finally, these HTTP feature data are analyzed to diagnose faults.

[0058] This solution can accurately pinpoint the location of a fault, whether it occurs at the request sender, a critical link in the network transmission link, or at the request receiver, thus improving the accuracy and efficiency of fault diagnosis. It effectively overcomes the shortcomings of traditional distributed link tracing solutions in fault localization, which can only locate the communication process between two services, but cannot pinpoint the specific link in the message transmission path where the problem occurs.

[0059] Furthermore, unlike traditional distributed tracing, this solution does not require embedding specific monitoring code in application code or using specific frameworks. This avoids increasing the development and maintenance costs of business systems. It also solves the problem of tracing interruptions caused by some services in distributed systems being unable to connect to the existing distributed tracing system due to technical, external procurement, or other reasons.

[0060] Because modern software widely adopts distributed microservice architecture, a single business transaction often needs to span multiple service nodes to complete. As the dependencies between services become increasingly complex, effectively tracking and monitoring the execution of requests across various nodes, and subsequently identifying and troubleshooting faults, has become a major challenge in system operation and maintenance. To address this issue, distributed tracing systems have emerged, which are tools for tracing the business processing links in distributed systems.

[0061] Most distributed tracing systems on the market today are based on the design principles and implementation methods of Google's Dapper paper ("Dapper, a Large-Scale Distributed Systems Tracing Infrastructure"). These systems can capture and record performance data of requests on various service nodes and analyze and display this data through a visual interface.

[0062] Traditional distributed tracing systems still have significant limitations in fault location accuracy. While these systems can monitor and locate anomalies in call response times between service nodes, they cannot deeply analyze the specific causes of these anomalies. For example, when service A calls service B, and service B then calls service C, if a response delay occurs during the call from service B to service C, a traditional APM system can only indicate a communication network failure between service B and service C. It cannot identify anomalies in service B's write requests, service C's request reception, or failures in certain links of the network transmission (such as Layer 4 load balancer forwarding, Kubernetes Ingress forwarding, container network transmission, etc.). The lack of this fine-grained diagnostic capability in traditional distributed tracing systems means that operations personnel still need to manually check for potential fault points layer by layer. When the transmission path is long, the troubleshooting process is time-consuming and laborious. In particular, for low-probability, intermittent fault scenarios, the difficulty and time consumption of troubleshooting increase dramatically.

[0063] Furthermore, traditional distributed tracing systems are typically highly intrusive in terms of code. To integrate these systems, developers need to embed specific monitoring code into their application code or use specific frameworks, significantly increasing the complexity of system development and maintenance. When there are outsourced modules or services in the system, these modules and services are difficult to integrate, leading to tracing interruptions and decreased location accuracy.

[0064] This application provides a solution that extracts features from a pre-acquired HTTP network data stream according to a pre-configured communication feature extraction strategy to obtain HTTP communication feature data packets, which contain request association information. Based on the request association information, target anomaly analysis parameters in several of the HTTP communication feature data packets are parsed to perform system anomaly analysis. By extracting feature data from the HTTP network data stream using the above solution, the location of the fault can be accurately located and anomaly analysis can be performed, regardless of whether the fault occurs at the request sending end, the request receiving end, or other important links in the network transmission link, thereby improving the accuracy and efficiency of fault diagnosis.

[0065] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device or anomaly analysis system capable of performing the above functions. The following description uses an anomaly analysis system as an example to illustrate this embodiment and the subsequent embodiments.

[0066] Based on this, embodiments of this application provide an anomaly analysis method, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the anomaly analysis method of this application.

[0067] In this embodiment, the anomaly analysis method includes steps S1000 to S2000:

[0068] Step S1000: Extract features from the pre-acquired HTTP network data stream according to the pre-configured communication feature extraction strategy to obtain an HTTP communication feature data packet, wherein the communication feature data packet contains request association information;

[0069] It should be noted that the anomaly analysis method proposed in this application embodiment can work in conjunction with a traditional distributed link tracing system, or it can work independently of a traditional distributed link tracing system. When used in conjunction with a traditional distributed link tracing system, it can serve as an enhancement and supplement to it. When performing diagnosis, a traditional distributed link tracing system can only treat the communication link between two services as a whole. It can only know that there is a problem with the communication, but it cannot know which link in the communication process the problem occurs at. In this application embodiment, packet capture analysis is performed at key links in the network communication path between services to obtain information such as the time when HTTP communication characteristic packets arrive at key links. By comprehensively analyzing the characteristic data obtained from each key link point, the location of the fault can be located.

[0070] Specifically, you can refer to Figure 2 , Figure 2 This is a schematic diagram of the architecture of an anomaly analysis system provided in an embodiment of this application. Figure 2 The blue box indicates the network data flow analysis program, online fault diagnosis service, and data query and analysis service, which are the main microservices in this application's embodiments. The yellow box indicates the middleware services used: Kafka, Logstash, and Elastic Search. Kafka is used for message peak smoothing and data distribution, and can be replaced by other message middleware, such as RocketMQ. Logstash and Elastic Search are used for message storage and querying, and can also be replaced by other data storage middleware, such as the ClickHouse technology stack.

[0071] Additionally, it should be noted that the above... Figure 2In the system shown, the network data stream analysis program filters and captures HTTP network data streams, and uses a sophisticated HTTP communication feature extraction strategy to extract a series of HTTP communication feature data from the network data streams and send them to Kafka. The network data stream analysis program is deployed on both sides of the critical links in the inter-service network communication path (such as Layer 4 load balancers, Kubernetes Ingress, container networks, application servers, etc.). The HTTP communication feature data collected by the network data stream analysis program at different locations will contain a request association information, so that the data collected at various locations can be correlated for comprehensive analysis during subsequent data analysis. The selection criteria for this request association information are that it is contained in all the HTTP requests of interest, and its value is different in each request. In this embodiment, the TraceID (session ID) and SpanID (service ID) request headers in the HTTP request header can be selected as the association information. Of course, in this embodiment, other features that meet the selection criteria can also be selected.

[0072] The online fault diagnosis service's function is to subscribe to HTTP communication characteristic data generated by the Kafka network data stream analysis program, and then correlate and comprehensively analyze the characteristic data generated at each key link to locate the fault location, and generate network transmission quality related indicators for each key link of network communication, such as Load Balancer forwarding time indicators, K8S Ingress forwarding time indicators, and container network transmission latency indicators.

[0073] The data query and analysis service functions similarly to the online fault diagnosis service. Both diagnose fault locations by correlating and comprehensively analyzing the feature data extracted from key points in the network data flow analysis program. The difference is that the online fault diagnosis service performs real-time analysis by subscribing to data from Kafka, while the data query and analysis service retrieves historical data from Elastic Search for offline analysis only when the user calls its interface. The data query and analysis service uses the request association information (such as TraceID and SpanID) passed by the user through the interface parameters to query Elastic Search and retrieve the feature data extracted from key points in the network communication path by the network data flow analysis program.

[0074] Specifically, in this embodiment of the application, the network data stream analysis program packages the feature data of an HTTP communication into a message and sends it to Kafka. In addition to the feature data of this HTTP communication, the message also contains some metadata information, such as the network data stream packet capture location encoding.

[0075] More specifically, the network data stream analysis program uses HTTP communication feature extraction strategies to extract the following HTTP communication feature data from the network data stream:

[0076] First and foremost are the request association information TraceID and SpanID. These two features uniquely identify the request. During fault diagnosis, this information is needed to retrieve feature messages and to correlate and aggregate feature messages captured from different packet capture locations. TraceID is used in distributed tracing to uniquely identify a single business transaction, while SpanID represents a specific service. Throughout the request's call chain, the request carries the TraceID downstream. Each service also generates its own SpanID, which is passed downstream along with the TraceID. Of course, other features can be used as substitutes, as long as they uniquely identify the request.

[0077] In addition, HTTP communication characteristic data also includes, but is not limited to: the time when the first packet of the current HTTP communication request data arrives at the packet capture location, the time when the last packet of the current HTTP communication request data arrives at the packet capture location, the time when the first packet of the current HTTP communication response data arrives at the packet capture location, the time when the last packet of the current HTTP communication response data arrives at the packet capture location, the time when the packet with the smallest sequence number of the current HTTP communication request data arrives at the packet capture location, the time when the packet with the largest sequence number of the current HTTP communication request data arrives at the packet capture location, the time when the packet with the smallest sequence number of the current HTTP communication response data arrives at the packet capture location, the time when the packet with the largest sequence number of the current HTTP communication response data arrives at the packet capture location, the request packet source IP, the request packet source port, the request packet destination IP, the request packet destination port, and the sequence number (seq) of the TCP packet carrying the HTTP request line of this HTTP communication, i.e., the smallest sequence number of the request packet.

[0078] In addition, the aforementioned HTTP communication characteristic data also includes the sequence number (seq) of the TCP segment carrying the last byte of the current HTTP communication request data, the maximum sequence number of the request packet, the payload length of the TCP segment with the maximum sequence number of the current HTTP communication request data, the cumulative length of all request packets in the current HTTP communication, the total number of all request packets in the current HTTP communication, the maximum time difference between the arrival times of two adjacent request packets in the current HTTP communication, the cumulative time difference between the arrival times of two adjacent request packets in the current HTTP communication, the sequence number (seq) of the request packet whose time difference with the previous request packet's arrival time is the largest, and the acknowledgment number of the request packet whose time difference with the previous request packet's arrival time is the largest. The sequence numbers are: Number, arrival time of the request message with the largest time difference from the previous request message's arrival location, Sequence Number (seq) of the TCP segment carrying the HTTP response status line of this HTTP communication (i.e., the minimum sequence number of the response packet), Sequence Number (seq) of the TCP segment carrying the last byte of the HTTP response data of this communication (i.e., the maximum sequence number of the response packet), Payload length of the TCP segment with the maximum sequence number of the HTTP response data of this communication, Cumulative length of all response messages in this HTTP communication, Total number of all response messages in this HTTP communication, Maximum time difference between the arrival locations of two adjacent response messages in this HTTP communication, Cumulative time difference between the arrival locations of two adjacent response messages in this HTTP communication, Sequence Number (seq) of the response message with the largest time difference from the previous response message's arrival location, and Acknowledgment number of the response message with the largest time difference from the previous response message's arrival location. The response header includes the following information: the number of responses, the arrival time of the response message with the largest time difference from the previous response message's arrival at the packet capture location, information on all messages containing HTTP status codes in this HTTP communication (including HTTP status code, TCP sequence number, TCP acknowledgment number, message length, and message arrival time; note that multiple response messages containing HTTP status codes may exist simultaneously in one HTTP communication, such as HTTP status code 100 and other status codes), and the value of the response header content-length.

[0079] In addition, it includes the method of ending the TCP connection in the request direction, the sequence number of the TCP segment used to end the connection in the request direction, the acknowledgment number of the TCP segment used to end the connection in the request direction, the number of TCP FIN packets in the request direction, the method of ending the TCP connection in the response direction, the sequence number of the TCP segment used to end the connection in the response direction, the acknowledgment number of the TCP segment used to end the connection in the response direction, and the number of TCP FIN packets in the response direction.

[0080] Step S2000: Based on the request association information, parse the target anomaly analysis parameters in several HTTP communication feature data packets to perform system anomaly analysis based on the target anomaly analysis parameters.

[0081] It should be noted that the purpose of this step is to use request association information to parse target anomaly analysis parameters in HTTP communication characteristic data packets. These parameters are crucial for identifying and diagnosing abnormal behavior in the network. By analyzing these parameters, the system can identify problems such as latency, packet loss, and retransmission in network communication, thus providing a basis for the system's anomaly analysis. In one specific implementation, the system first groups data packets according to session record IDs, and then extracts the request and response data packets from each group. The system then calculates target anomaly analysis parameters such as processing latency and packet sending speed, and matches these parameters with preset fault analysis rules to determine whether any abnormalities exist. For example, if the system detects that the processing latency exceeds a preset threshold, it may mark the communication session corresponding to that session record ID as a potential anomaly for further analysis and processing.

[0082] This embodiment provides an anomaly analysis method. It extracts features from a pre-acquired HTTP network data stream according to a pre-configured communication feature extraction strategy to obtain HTTP communication feature data packets, which contain request association information. Based on the request association information, it parses target anomaly analysis parameters from several of the HTTP communication feature data packets to perform system anomaly analysis. By extracting feature data from the HTTP network data stream using the above scheme, regardless of whether the fault occurs at the request sending end, the request receiving end, or other critical links in the network transmission link, the location of the fault can be accurately located and anomaly analysis can be performed, thereby improving the accuracy and efficiency of fault diagnosis.

[0083] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 3 The HTTP network data stream includes an HTTP communication session record and a data packet buffer. Step S1000 may include steps S1100 to S1300:

[0084] Step S1100: Extract network data packets from the HTTP network data stream to obtain TCP packets by unpacking the network data packets, wherein the network data packets include HTTP communication characteristic data;

[0085] Step S1200: If the TCP message is a data packet, then according to the preset data packet processing strategy, the network data packet is stored in the HTTP communication session record or the data packet buffer.

[0086] Step S1300: If the TCP packet is an end packet or a reset packet, then based on the TCP quadruple corresponding to the data packet, the network data packet is stored in the HTTP communication session record;

[0087] Step S1400: When the preset record output conditions are met, obtain the HTTP communication feature data in the HTTP communication session record, and generate the HTTP communication feature data packet based on the communication feature data.

[0088] It should be noted that in this embodiment, an HTTP communication can be abstracted as a session, and a TCP connection can be abstracted as a stream. Since a TCP connection may be used to transmit multiple HTTP communications, and adjacent HTTP communications may overlap in time (i.e., a new HTTP request is sent before the response data packet of the previous HTTP communication has finished transmitting), an HTTP network data stream needs to store two HTTP communication session records (trace). Furthermore, during algorithm processing, there may be situations where it is temporarily impossible to determine whether the current packet belongs to the previous or subsequent trace. Therefore, this embodiment also defines two packet buffers, one for storing requests whose direction cannot be temporarily determined to belong to the previous trace. For packets belonging to the previous or subsequent trace, and for packets whose storage response direction is temporarily uncertain, the network data flow analysis program extracts feature data from HTTP communication. During this process, all currently processed flows are stored in a Map collection, with the key set to a sorted TCP quadruple (source IP, source port, destination IP, destination port). The sorting ensures that both request and response packets can be stored in the same flow. The sorted TCP quadruples for request and response packets will yield the same result, allowing the generation of identical keys based on these sorted TCP quadruples. This enables the storage of both request and response packets in the same flow.

[0089] Additionally, it should be noted that the generation of HTTP communication feature packets depends on the processing of network packets. Specifically, this includes obtaining and sorting the TCP quadruple tuples corresponding to the packet segments, querying a preset two-column set to check for the existence of key-value pairs for the HTTP network data stream, and storing the network packets in appropriate records based on the packet type and HTTP network data stream attribute information. Furthermore, when the sequence number and acknowledgment number of the network packet follow those of the existing response packet, a packet concatenation payload is generated, and the HTTP communication feature data within it is identified. This helps extract key communication features. Specifically, in one feasible implementation, the system first captures the network data stream and then extracts key information such as the packet sequence number, timestamp, source, and destination IP addresses using a specific algorithm. The system examines each TCP segment; if it is a packet segment, it stores it in the HTTP communication session record or packet buffer according to a preset strategy. For termination or reset packets, the system stores them in the HTTP communication session record based on the TCP quadruple tuple. When the system detects that preset output conditions are met, such as when a session completes or times out, the system outputs HTTP communication characteristic data from the HTTP communication session record and generates HTTP communication characteristic data packets. These data packets contain rich information, such as TraceID, SpanID, and timestamps, which are the basis for fault diagnosis. The system further analyzes these characteristic data packets to identify and locate anomalies in the network, thereby achieving accurate fault diagnosis.

[0090] More specifically, refer to Figure 4 , Figure 4 This is a schematic diagram of the overall data packet processing flow provided in an embodiment of this application. When a network data packet is captured, the IP header version number of the data packet is first checked to confirm whether the IP packet is IPv4 or IPv6. Depending on the version number, the IPv4 or IPv6 packet is further unpacked to obtain a TCP packet. Next, it is determined whether the payload length of the TCP packet is not 0, i.e., whether it is a data packet. If the payload length is not 0, it is a data packet, and the system enters the "data packet processing sub-flow" to process the data packet. If the payload length is 0, it is further determined whether it is a FIN packet or an RST packet, and enters the corresponding processing sub-flow respectively. Packets that do not meet the above conditions will be regarded as useless data and discarded.

[0091] In one feasible implementation, step S1200 may include steps S1210 to S1240:

[0092] Step S1210: If the TCP message is a data packet, then obtain the TCP quadruple corresponding to the data packet and sort it to obtain the sorted TCP quadruple;

[0093] Step S1220: Query the preset two-column set to see if the key-value pair of the HTTP network data stream exists based on the sorted TCP quadtuple;

[0094] Specifically, you can refer to Figure 5 , Figure 5 This is a schematic diagram of a data packet processing sub-flow provided in an embodiment of this application. When processing a data packet where the payload is not empty (the effective payload is not empty), the four-tuple of the data packet is first obtained, the four-tuple is sorted, and then a primary key value Key is generated from the sorted four-tuple. The primary key value Key is used to query whether the HTTP communication data flow corresponding to the four-tuple exists in the two-column set Map storing the flow. If it does not exist, the processing sub-flow for the scenario where the payload is not empty and the communication data flow does not exist is entered. In one feasible implementation, please refer to the following. Figure 6 , Figure 6 This embodiment of the application provides a schematic diagram of a data packet processing sub-flow in a scenario where the data packet payload is not empty and the data stream does not exist. First, it determines whether the data packet is a request-oriented packet. If it is not a request-oriented packet, the packet is discarded and the process ends. If it is confirmed to be a request packet, the current cache size is further checked to see if it exceeds a set threshold. If the cache size exceeds the threshold, the data packet is directly discarded to prevent memory exhaustion. If the cache size is less than the set threshold, a new flow and its corresponding HTTP communication session record, trace1, are created. Trace1 is initialized with the information from the data packet, and the sequence number of the data packet is assigned to the reqMinSe of the communication session record trace1. The code sets q, reqMaxSeq, reqFirstSeq, and reqLastSeq. It assigns the acknowledgment number of the data packet to reqMinAck and reqMaxAck of trace1, the payload of the data packet to reqMinPayload of trace1, the payload length of the data packet to reqMaxLen and reqPktLen of trace1, the received timestamp of the data packet to reqFirstTs and reqLastTs of trace1, and sets reqPktCount of trace1 to 1. It then stores the data in the HTTP communication data stream two-column collection flow Map with the key as the key and the flow as the value.

[0095] Step S1240: If a key-value pair of the HTTP network data stream exists, the network data packet is stored in the HTTP communication session record or the data packet buffer according to the data packet type of the network data packet and the attribute information of the HTTP network data stream.

[0096] In another feasible implementation, reference can be made to Figure 7 , Figure 7 This is a schematic diagram of a processing sub-process in a scenario where the data packet payload is not empty and the data stream exists, provided by an embodiment of this application. If the data packet payload is not empty and the HTTP communication data stream exists, the network data packet is stored in the HTTP communication session record or the data packet buffer according to the type of the network data packet and the attribute information of the HTTP network data stream.

[0097] Specifically, first determine if the data packet is a request-oriented packet; if it is, then proceed... Figure 7 The request packet processing sub-flow shown will proceed if the packet is a response packet. Figure 7 The response packet processing sub-flow shown above, in which the core of the request packet processing sub-flow is to determine whether the request packet (i.e., the network packet, which has been determined to be a request packet) should be stored in trace1, trace2, or the request buffer. In one feasible implementation, if the request packet meets one of the following conditions, then the packet is stored in the HTTP communication session record trace1 through the request packet storage trace sub-flow:

[0098] First, trace1 only contains data packets in the request direction and no corresponding response data packets. It should be understood that this means the communication session in trace1 has not ended. Therefore, new request data packets should be added to trace1. Just like in a conversation, if you only hear the question (request) but not the answer (response), then the new question (request data packet) should continue in the current conversation (trace1).

[0099] Secondly, if trace1 contains a response-oriented data packet, and trace2 is empty, and the current data packet's sequence number (seq) is before the smallest request packet's sequence number in trace2, it should be understood that if trace1 already contains a response-oriented data packet, and trace2 is empty, and the current data packet's sequence number (seq) is before the smallest request packet's sequence number in trace2, this means the new request data packet actually belongs to the previous session (trace1), not the new session (trace2). This is similar to a conversation where, if an answer has already been heard, but the new question (request data packet) predates the start of the next session, it should be attributed to the previous session.

[0100] Thirdly, for packets in the response direction of trace1, if the sequence number of the current packet is less than or equal to the maximum sequence number in trace1 (Note: This condition does not need to be explicitly checked, because if it is determined that the packet does not belong to trace2, it will be stored in the buffer and a final attribution check will be performed), it should be understood that if there are packets in the response direction in trace1, and the sequence number of the current packet is less than or equal to the maximum sequence number in trace1, this means that the new request packet is actually sent in the current session (trace1), not a new session. This is like in a conversation, if the new question (request packet) does not exceed the range of the current session in terms of sequence number, then it should be classified into the current session.

[0101] Fourth, if trace1 contains response-oriented data packets, trace2 is empty, and the first few bytes of the current data packet payload are HTTP request command strings (GET, POST, etc.), but the sequence number of the current packet is not before the acknowledgment number (ack) of the smallest sequence number response packet in trace1, or its acknowledgment number is after the smallest sequence number of the response packet in trace1, it should be understood that if trace1 contains response-oriented data packets, trace2 is empty, and the first few bytes of the current data packet are HTTP request command strings (such as GET or POST), but the sequence number of the current data packet is not before the acknowledgment number (ack) of the smallest sequence number response packet in trace1, or its acknowledgment number is after the smallest sequence number of the response packet in trace1, this means that the new request data packet actually belongs to the current session (trace1), not the new session (trace2). Just like in a conversation, if a new question (request data packet) is consistent with the context of the current session in terms of content and matches the last answer of the current session in terms of sequence number, then it should be classified into the current session.

[0102] In another feasible implementation, if the request packet meets one of the following conditions, the packet is stored in the HTTP communication session record trace2 via the request packet storage trace sub-process:

[0103] Firstly, if trace1 has a response-oriented data packet, trace2's request is not empty, and the current data packet's sequence number (seq) is not before the smallest sequence number of the request packet in trace2, and trace2's response is empty, it should be understood that if trace1 already has a response-oriented data packet, it means that the session in trace1 is nearing its end or has already ended. At the same time, if trace2 already has a request-oriented data packet, it means that a new session (trace2) has begun. If the sequence number (seq) of the current request data packet is not before the smallest sequence number of the request packet in trace2, it indicates that it is not the first data packet in trace2. Furthermore, if trace2 does not yet have a response-oriented data packet, then the current request data packet should be part of trace2, because it arrived after the request in trace2 and before the response.

[0104] Secondly, if trace1 contains response-oriented data packets, trace2 has an empty request, and the first few bytes of the current data packet payload are HTTP request command strings (GET, POST, etc.), and the current packet's sequence number (seq) is not before the ACK of the smallest sequence number response packet in trace1, but its ACK is after the smallest sequence number of the trace1 response packet, it should be understood that if trace1 already contains response-oriented data packets, but trace2 does not yet contain request-oriented data packets (i.e., trace2 has an empty request), this may mean that the trace2 session has not yet officially started. If the payload (i.e., data content) of the current request data packet begins with an HTTP request command string (such as GET or POST), this indicates that it is the start of a new request. Furthermore, if the sequence number (seq) of this request data packet is not before the ACK of the smallest sequence number response packet in trace1, but its ACK is after the smallest sequence number of the trace1 response packet, this indicates that this request arrived after the last response of trace1, therefore it should belong to a new session, i.e., trace2.

[0105] Finally, it should be noted that when a request data packet meets the second condition for storing it in the HTTP communication session record trace2, a new trace2 needs to be created immediately and initialized with this data packet. This data packet is then used to mark the end method of the trace1 request direction. Finally, the request cache archiving sub-process is entered to determine the ownership of the data packets in the cache. If it is temporarily impossible to determine whether the data packet belongs to trace1 or trace2, the request cache storage sub-process is entered to place the data packet into the request cache for subsequent processing.

[0106] Additionally, it should be noted that the above-mentioned request data packet storage in the trace sub-process specifically includes comparing the data packet's seq, ack, timestamp, and other information with the existing data in the trace, and then updating the corresponding fields in the trace. If the current packet's seq is greater than the maximum seq in the trace, then the maximum packet information is updated. If the second small packet does not exist, then the information of the second small packet is updated to the information of the current packet, and the current packet's seq is equal to the maximum packet's seq in the trace. If the length of the current packet's payload is greater than the length of the maximum packet's payload, then the maximum packet information is updated; otherwise, the information of the second small packet is updated.

[0107] Furthermore, if the current packet seq is less than the smallest packet seq in the trace, then it is determined whether the current trace already contains a get / post packet. If it already exists, the current packet is discarded directly; if it does not exist, the current packet replaces the smallest packet, and the smallest packet replaces the second smallest packet. The current packet seq is equal to the smallest packet seq in the trace. If the current packet payload length is greater than the smallest packet payload length, then the smallest packet is replaced.

[0108] Finally, if the current packet's sequence number (seq) is less than the second smallest packet's sequence number (seq) in the trace, then the second smallest packet is replaced. If the current packet's sequence number (seq) is not less than the second smallest packet's sequence number (seq) in the trace, then it is further determined whether the current packet's sequence number (seq) is equal to the second smallest packet's sequence number (seq) in the trace. 若等于且 If the current packet payload length is greater than the second packet payload length, then the second packet will be replaced.

[0109] Additionally, it should be noted that the above request cache archiving sub-process can be referred to... Figure 8 , Figure 8 This diagram illustrates a request cache archiving sub-process according to an embodiment of this application. This sub-process traverses the request cache and determines whether a data packet belongs to trace1 or trace2 based on whether its sequence number (seq) is before the minimum seq requested in trace2. If the data packet sequence number is before the minimum seq requested in trace2, the packet is assigned to trace1; otherwise, it is assigned to trace2.

[0110] Additionally, it should be noted that the above-mentioned request cache storage sub-process includes a process where, when it is temporarily impossible to determine whether a request data packet belongs to trace1 or trace2, the system will enter this sub-process to temporarily cache the network data packet for subsequent processing. This process is designed to deal with scenarios where the captured packets are out of order. Assuming that out-of-order packets will not deviate too much, the cache only needs to be set to a certain size.

[0111] More specifically, in another case, if the network packet is a response packet, then it enters... Figure 7 The response packet processing sub-flow shown in the diagram is crucial in determining whether the response packet can be placed in trace1 or trace2. If it cannot be determined temporarily, it is stored in the response buffer.

[0112] Specifically, if a response packet meets one of the following conditions, then the response packet will be included in trace1 through the trace subprocess:

[0113] First, there are no request packets in trace2. It should be understood that if no request packets are recorded in trace2, it means that the session represented by trace2 has not started. In this case, the current response packets are obviously part of the current session (trace1).

[0114] Secondly, trace2 contains both request and response packets, and the sequence number (seq) of this packet is before the smallest sequence number response packet in trace2. It should be understood that if trace2 already contains both request and response packets, it indicates that the session represented by trace2 has started and is in progress. If the sequence number (seq) of the current response packet is before the smallest sequence number response packet recorded in trace2, it means that the current response packet was sent before the response from trace2, and therefore this network packet should belong to trace1.

[0115] Thirdly, if trace2 contains a request packet but no response packet, and the current packet payload does not begin with "http1." (i.e., it is not a packet carrying an HTTP response status line), and the current packet's sequence number (seq) precedes the acknowledgment (ack) of trace2's minimum sequence number request packet, it should be understood that if trace2 contains a request packet but no response packet, and the current response packet payload does not begin with "http1.", this usually means it is not a complete HTTP response. Furthermore, if the current packet's sequence number (seq) precedes the acknowledgment number (ack) of trace2's minimum sequence number request packet, this indicates it belongs to trace1, as it was sent after trace2's request and before trace2's response. In this case, the packet can be directly sent to the response buffer, awaiting further processing and judgment.

[0116] Fourth, if trace2 contains a request packet but no response packet, and the payload of the current packet begins with "http1.", and the sequence number (seq) of the current packet is before the sequence number (seq) of the minimum sequence number packet requested by trace2, or its acknowledgment number (ack) is not after the minimum sequence number packet requested by trace2, it should be understood that if trace2 contains a request packet but no response packet, and the payload of the current response packet begins with "http1.", it indicates the beginning of an HTTP response. If the sequence number (seq) of the current packet is before the sequence number (seq) of the minimum sequence number packet requested by trace2, or its acknowledgment number (ack) is not after the minimum sequence number packet requested by trace2, this means that the current response packet belongs to trace1, because it was sent after the request of trace2. In addition, when the response packet meets this condition, it is also necessary to parse the sequence number, ack, timestamp, HTTP response code, length, and other information of the packet and fill them into the response list of trace1.

[0117] Specifically, if a response packet meets one of the following conditions, then the response packet will be included in trace2 through the trace subprocess:

[0118] Firstly, if trace2 contains both request and response packets, and the sequence number (seq) of this packet is not before the smallest sequence number response packet in trace2, it should be understood that if trace2 has already recorded both request and response packets, it indicates that the session represented by trace2 has started and data exchange has occurred. If the sequence number (seq) of the current response packet is not before the smallest sequence number response packet recorded in trace2, it means that the current response packet was sent after a certain response in trace2. Therefore, the current response packet should be part of trace2.

[0119] Secondly, if trace2 contains a request packet but no response packet, and the current packet payload begins with "http1.", and the ACK of the current packet is after the minimum sequence number (seq) of the trace2 request packet, but not before the seq of the trace2 request minimum sequence number packet, it should be understood that if trace2 contains a request packet but has not yet contained a response packet, it means that the trace2 session has started but not yet completed. If the payload of the current response packet begins with "http1.", it indicates that it is the beginning of an HTTP response. If the ACK of the current packet is after the ACK of the trace2 request minimum sequence number packet, but not before the seq of the trace2 request minimum sequence number packet, it indicates that the current response packet was sent after the trace2 request and before the trace2 response. Therefore, the current response packet should be part of trace2. In addition, when the response packet meets this condition, it is also necessary to parse the packet's seq, ACK, timestamp, HTTP response code, length, and other information to fill in the trace2 response list. Simultaneously, set `respMinFindCode` of `trace2` to `true`. Then, use the information from this packet to fill in the response end information of `trace1`, mark the response end method of `trace1`, and finally, enter the response buffer archiving sub-process to determine the ownership of the response packets in the buffer.

[0120] If it is temporarily impossible to determine whether the response data packet should be assigned to trace1 or trace2, then the response buffer storage sub-process is entered to temporarily store the data packet in the response buffer for subsequent judgment.

[0121] Specifically, the response buffer storage sub-process includes temporarily caching response packets when it's unclear whether they belong to trace1 or trace2, awaiting subsequent processing. We assume out-of-order packets won't deviate too much; that is, before receiving the minimum sequence number (seq) response packet from trace2, at most only a few trace2 response packets will be received. Therefore, when the buffer is full, packets dequeued can be directly stored in trace1.

[0122] Specifically, when the response buffer archiving subprocess includes response data packets in the archived response buffer, it is necessary to dequeue the data in the buffer sequentially and determine whether its sequence number (seq) is before the smallest sequence number (seq) of the response packet in trace2. If the condition is met, the response data packet is stored in trace1 through the response buffer storage subprocess and the relevant information of the data packet is stored in trace1. If the condition is not met, the response data packet is stored in trace2 through the response buffer storage subprocess and the relevant information of the data packet is stored in trace2.

[0123] In another feasible implementation, step S1300: if the TCP packet is an end packet or a reset packet, then storing the network data packet in the HTTP communication session record based on the TCP quadruple corresponding to the data packet may include steps S1310 to S1320:

[0124] Step S1310: If the TCP packet is an end packet, then based on the TCP quadruple corresponding to the data packet, store the network data packet in the HTTP communication session record;

[0125] It should be noted that, in this embodiment of the application, when processing FIN packets, the TCP quadruple (source IP: source port, destination IP: destination port) of the data packet is first sorted to generate a key. The key is recorded to determine whether the quadruple is the same after sorting and before sorting. This is to facilitate the subsequent comparison with the dirFlag in the flow to determine whether the packet is a request-direction or response-direction packet. The generated key is used to query whether there is a corresponding flow in the flow map. If it does not exist, the data packet is directly discarded.

[0126] If it exists, retrieve the flow, and perform an XOR operation between the variable recording whether the sorted quadruple is the same as the original and the dirFlag in the flow. If the result is true, record the direction of the data packet as the request direction; otherwise, record it as the response direction.

[0127] If the direction of the data packet is not the request direction, increment the reqFINCount of trace1 and trace2 by 1 respectively, and check whether the reqFinishType of trace1 and trace2 is empty. If it is empty, mark the reqFinishType of the trace as the end of the FIN packet, and assign the seq, ack, and arrival timestamp of the FIN packet to the reqFinishSeq, reqFinishAck, and reqFinishTs properties of the trace respectively.

[0128] If the direction of the data packet is not the request direction, increment the respFINCount of trace1 and trace2 by 1 respectively, and check whether the respFinishType of trace1 and trace2 is none. If none, mark the respFinishType of the trace as the end of the FIN packet, and assign the seq, ack, and arrival timestamp of the FIN packet to the respFinishSeq, respFinishAck, and respFinishTs properties of the trace respectively.

[0129] Next, it is determined whether reqFINCount and respFINCount of trace1 are both greater than 0. If not, the sub-process ends. If yes, it is determined whether there is data in the request cache and response cache of the flow. If there is, the above request data packet storage in trace sub-process and the above response data packet storage in trace sub-process are used to store all the data packets in the corresponding cache into trace1.

[0130] Step S1320: If the TCP packet is a reset packet, then based on the TCP four-tuple corresponding to the data packet, store the network data packet in the HTTP communication session record.

[0131] It should be noted that, in this embodiment, when processing Reset (abbreviated as rst) packets, the TCP quadruple (source IP: source port, destination IP: destination port) of the packet is first sorted to generate a key. The difference between the sorted and unsorted quadruples is recorded to facilitate subsequent comparison with the dirFlag in the flow map to determine whether the packet is a request or response packet. The generated key is used to query whether a corresponding flow exists in the flow map; if not, the packet is discarded.

[0132] If it exists, retrieve the flow, and perform an XOR operation between the variable recording whether the sorted quadruple is the same as the original and the dirFlag in the flow. If the result is true, record the direction of the data packet as the request direction; otherwise, record it as the response direction.

[0133] If the direction of the data packet is not the request direction, check whether the reqFinishType of trace1 and trace2 is none. If it is none, record the reqFinishType of the trace as the end of the rst packet, and assign the seq, ack, and arrival timestamp of the rst packet to the reqFinishSeq, reqFinishAck, and reqFinishTs properties of the trace respectively.

[0134] If the direction of the data packet is not the request direction, check whether the respFinishType attribute of trace1 and trace2 is empty. If it is empty, mark the respFinishType attribute of the trace as the end of the rst packet, and assign the seq, ack, and arrival timestamp of the rst packet to the respFinishSeq, respFinishAck, and respFinishTs attributes of the trace respectively.

[0135] Next, it is determined whether there is data in the request cache and response cache of the flow. If there is, the above-mentioned request data packet storage in trace sub-process and the above-mentioned response data packet storage in trace sub-process are used to store all data packets in the corresponding cache into trace1.

[0136] This application provides an anomaly analysis method. It extracts network data packets from the HTTP network data stream and unpacks them to obtain TCP packets. If the TCP packet is a data packet, it stores the network data packet in the HTTP communication session record or the data packet buffer according to a preset data packet processing strategy. If the TCP packet is a termination packet or a reset packet, it stores the network data packet in the HTTP communication session record based on the TCP quadruple corresponding to the data packet. When preset record output conditions are met, HTTP communication feature data is obtained from the HTTP communication session record, and HTTP communication feature data packets are generated based on this data. This method non-intrusively and efficiently locates the specific link of a network fault, significantly improving the accuracy and response speed of fault diagnosis. Furthermore, this method avoids modification of the business system code, reduces the complexity of system development and maintenance, and ensures seamless access to link tracing for services in a distributed system. Even when facing complex network transmission paths and occasional low-probability faults, it can quickly and accurately troubleshoot faults.

[0137] Based on the first embodiment of this application, in the third embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 9 The HTTP communication session record includes a first HTTP communication session record and a second HTTP communication session record. Step S1400 of the anomaly analysis method further includes steps S1410 to S1440:

[0138] Step S1410: Obtain the sequence number and acknowledgment number of the network data packet, as well as the sequence number and acknowledgment number of the existing response data packet in the second HTTP communication session record;

[0139] Step S1420: If the sequence number and acknowledgment number of the network data packet are after the sequence number and acknowledgment number of the existing response data packet, then generate a data packet splicing payload based on the target data packet in the first HTTP communication session record;

[0140] Step S1430: Identify and obtain HTTP communication feature data in the data packet splicing payload;

[0141] Step S1440: Generate the HTTP communication feature data packet based on the HTTP communication feature data and the pre-acquired metadata information.

[0142] It should be noted that, in the embodiments of this application, when the HTTP network communication data stream satisfies the following conditions: the first HTTP communication session record trace1 has a response data packet, the second HTTP communication session record trace2 has a request data packet, and the seq of the current data packet is not before the minimum seq of the request packet in trace2, and the response of trace2 is not empty, and at the same time, the acknowledgment number ack of the current packet is after the minimum seq of the response packet in trace2, then trace1 can be output through the output sub-process, trace2 can be assigned to trace1, then trace2 can be set to empty, and finally, the data packet can be re-entered into the request data packet processing sub-process for processing.

[0143] Specifically, the fact that the first HTTP communication session record trace1 contains data packets in the response direction means that in the current flow, there already exists at least one complete HTTP communication session (trace1), which contains not only data packets in the request direction but also data packets in the corresponding response direction.

[0144] The second HTTP communication session record, Trace2, contains request-oriented data packets, indicating that the next HTTP communication session (trace2) following trace1 has begun and that at least request-oriented data packets have been received.

[0145] The fact that the sequence number (seq) of the current data packet is not before the smallest sequence number of the request packets in trace2 means that the sequence number (seq) of the newly captured request data packet is not the smallest sequence number among all request data packets in trace2. In other words, this new data packet follows the existing request data packets in trace2 in terms of sequence number.

[0146] If the Trace2 response is not empty, it means that Trace2 has received at least one response packet. ACK refers to the acknowledgment number of the packet, which confirms that the packet has been received. This means that the acknowledgment number of the current packet is after the smallest sequence number response packet recorded in Trace2, indicating that the current packet arrived after a certain response from Trace2.

[0147] Based on the above conditions, this passage describes a situation where the current flow has completed a full HTTP communication session (trace1) and has begun processing the next session (trace2). The newly captured request packet follows the existing request packet in trace2 in terms of sequence number. At this point, the first HTTP communication session record trace1 is output, which actually outputs all the feature data included in a complete HTTP communication.

[0148] Specifically, the trace output subprocess first determines whether the HTTP request line and response status line have been parsed. If not, it parses them first and stores the parsed information in the trace object. Next, it concatenates the payloads of the two data packets with the smallest and second smallest sequences in the request. Then, it searches for newline characters sequentially from the first byte of the merged byte array to obtain the HTTP Request Headers. It stops when it identifies information such as traceid, spanid, and content-length in these request headers, or when it encounters two consecutive newline characters. Finally, after obtaining the complete HTTP communication characteristics, it packages the characteristic data of one HTTP communication into a message and sends it to Kafka. In addition to the aforementioned characteristic data, the message also contains some metadata information, such as the network data stream packet capture location encoding.

[0149] In one feasible implementation, the anomaly analysis method further includes steps A1000 to A4000:

[0150] Step A1000: Traverse the preset double-column set;

[0151] Step A2000: Determine whether the current silence duration of the current HTTP network data stream in the dual-column set exceeds the preset silence duration threshold;

[0152] Step A3000: If the current silence duration exceeds a preset silence duration threshold, then obtain the target key-value pair of the current HTTP network data stream in the double-column set;

[0153] Step A4000: Obtain the cached data in the current HTTP network data stream according to the target key-value pair, and store it in the preset HTTP communication session record.

[0154] It should be noted that, in this embodiment, in addition to triggering the data packet processing flow upon capturing a data packet, a timer will periodically trigger the flow expiration check and processing flow to clean up flows in the flow map that have been inactive for a long time (i.e., the current inactivity duration of the HTTP network data flow exceeds a preset inactivity duration threshold, where the inactivity duration threshold can be on the order of minutes or hours). (Refer to...) Figure 10 As shown, Figure 10 This is a schematic diagram of a network data stream expiration check and processing flow provided in an embodiment of this application. The system traverses the flow map, which is a storage structure that stores all currently active network sessions. During the traversal, the system checks the last update time (lastUpdateTime) of each flow and determines whether this time is within 10 minutes before the current time. If it is, it means that the flow is still active and the process will continue to traverse. If not, it means that the current silent duration of the HTTP network data stream exceeds the preset silent duration threshold. At this time, the flow has expired, and the process stores the key of the flow in the queue to be deleted, ready for cleanup.

[0155] Next, the system iterates through the queue to be deleted, checking if the process is complete. If not, the system retrieves the corresponding flow using the current key and removes the item corresponding to that key from the flow map. Then, the system checks if there is data in the flow's request cache. If so, the system uses the request data packet storage trace sub-process to store all data packets in the cache into trace1, and outputs trace1 and trace2 from the flow, completing the processing of expired flows.

[0156] This process ensures that the system can promptly clean up inactive network sessions, avoiding resource waste while maintaining system performance and responsiveness. By placing the code for flow expiration checks and packet processing in the same thread or coroutine, the system avoids using locks or thread-safe maps, reducing performance overhead.

[0157] It should be noted that both the processing flow and the data packet processing flow operate on the flow map. Therefore, in this embodiment, the code for the flow expiration check process and the data packet processing flow is placed in the same thread (or coroutine). When a data packet is captured or a timer expires, the thread (or coroutine) is triggered to run through a signal pipe, thus avoiding the serious performance degradation caused by locking or using thread-safe maps.

[0158] In one feasible implementation, the request association information includes a session record ID, and step S2000 may include steps S2100 to S2300:

[0159] Step S2100: Identify the HTTP communication feature data packets with the same session record ID as the target data packets, wherein the target data packets contain request data packets and response data packets;

[0160] Step S2200: Based on the data packet parameter information of the request data packet and the response data packet, calculate the target anomaly analysis parameters. The target anomaly analysis parameters include at least one of processing delay, packet sending speed, message interval time, transmission time, message capacity, and session end type.

[0161] Step S2300: Match the target anomaly analysis parameters based on the preset fault analysis rules to obtain the system anomaly analysis results.

[0162] It should be noted that in this embodiment, system fault diagnosis is performed through an online fault diagnosis service or a data query and analysis service. The online fault diagnosis service subscribes to HTTP communication feature data generated by a network data stream analysis program from Kafka, and then correlates and comprehensively analyzes the feature data generated at each key stage to locate the fault and generate network transmission quality-related indicators for each key network communication stage. The data query and analysis service functions similarly to the online fault diagnosis service, both correlating and comprehensively analyzing the feature data extracted from each key stage by the network data stream analysis program to diagnose the fault location. The difference is that the online fault diagnosis service subscribes to data from Kafka for real-time analysis, while the data query and analysis service retrieves historical data from Elastic Search for offline analysis only when the user calls its interface. The data query and analysis service uses user-provided request association information (such as TraceID and SpanID) through interface parameters to query Elastic Search and retrieve the feature data extracted from each key stage of the network communication path by the network data stream analysis program.

[0163] Specifically, regardless of the type of fault diagnosis service, its core lies in the in-depth analysis of HTTP communication characteristic data packets to identify and respond to abnormal situations in network communication. First, the HTTP network data stream is analyzed through a preset communication feature extraction strategy to extract data packets containing request-related information. Then, using this related information, the system parses out the abnormal analysis parameters, which include key indicators such as processing latency, packet sending speed, message interval time, transmission time and message capacity, and session termination type.

[0164] Additionally, it should be noted that the system identifies target data packets with the same session record ID as the target data packets. These packets contain both request and response data. Based on information from these packets, the system calculates target anomaly analysis parameters, which are used to identify abnormal situations in network communication. Finally, the system matches the target anomaly analysis parameters according to preset fault analysis rules to obtain system anomaly analysis results, providing data support for system anomaly analysis.

[0165] For example, in one feasible implementation, by measuring the time difference between the arrival of characteristic data packets (first request packet, last request packet, first response packet, and last response packet) of HTTP requests with the same traceID at each packet capture point, the processing delay of each intermediate link can be clearly known, and thus the failure can be determined by the length of the delay.

[0166] Alternatively, the cumulative time difference between adjacent request or response packets, along with the total number of packets, can be used to calculate the average interval between adjacent messages. This metric reflects the average packet sending speed of this packet capture point.

[0167] Alternatively, the maximum time difference between adjacent data packets of a request or response, when compared with the average packet sending speed of this capture point, can reflect the fluctuation of the packet sending speed of this capture point.

[0168] Alternatively, by analyzing the time difference between the last packet and the first packet in a request or response, the total transmission time in the request or response direction can be determined. By comparing the transmission time at each stage, the faulty stage can also be identified.

[0169] There's also the analysis of the HTTP session termination method for requests or responses. If the termination packet type is rst, it may indicate that an unexpected connection interruption or failure occurred during the request or response process.

[0170] Furthermore, by using the maximum seq, minimum seq, and maximum seq packet length of the request or response, the total number of bytes in the request or response can be calculated. By comparing this total with the cumulative length of the request or response packet, it can be determined whether there are any packet retransmissions, thereby assessing the network quality in the intermediate stages.

[0171] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the anomaly analysis method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0172] Please refer to Figure 11 This application also provides an anomaly analysis device, the anomaly analysis device comprising:

[0173] Extraction module 111 is used to extract features from a pre-acquired HTTP network data stream according to a pre-configured communication feature extraction strategy to obtain an HTTP communication feature data packet, wherein the communication feature data packet contains request association information;

[0174] The parsing module 112 is used to parse the target anomaly analysis parameters in several HTTP communication feature data packets according to the request association information, so as to perform system anomaly analysis based on the target anomaly analysis parameters.

[0175] The anomaly analysis device provided in this application, employing the anomaly analysis method in the above embodiments, can solve the technical problems of anomaly analysis. Compared with the prior art, the beneficial effects of the anomaly analysis device provided in this application are the same as those of the anomaly analysis method provided in the above embodiments, and other technical features in the anomaly analysis device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0176] This application provides an anomaly analysis device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the anomaly analysis method in Embodiment 1 above.

[0177] The following is for reference. Figure 12 The diagram illustrates a structural schematic of an anomaly analysis device suitable for implementing embodiments of this application. The anomaly analysis device in these embodiments may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 12 The anomaly analysis device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0178] like Figure 12As shown, the anomaly analysis device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the anomaly analysis device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows the anomaly analysis device to communicate wirelessly or wiredly with other devices to exchange data. While the figure shows anomaly analysis devices with various systems, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0179] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0180] The anomaly analysis device provided in this application adopts the anomaly analysis method in the above embodiments. Compared with the prior art, the beneficial effects of the anomaly analysis device provided in this application are the same as the beneficial effects of the anomaly analysis method provided in the above embodiments. Moreover, other technical features of the anomaly analysis device are the same as the features disclosed in the method of the previous embodiment, and will not be repeated here.

[0181] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0182] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0183] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the anomaly analysis method in the above embodiments.

[0184] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0185] The aforementioned computer-readable storage medium may be included in the anomaly analysis device; or it may exist independently and not assembled into the anomaly analysis device.

[0186] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0187] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0188] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0189] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described anomaly analysis method, thereby solving the technical problem of anomaly analysis. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the anomaly analysis method provided in the above embodiments, and will not be repeated here.

[0190] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described anomaly analysis method.

[0191] The computer program product provided in this application can solve the technical problem of anomaly analysis. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as the beneficial effects of the anomaly analysis method provided in the above embodiments, and will not be repeated here.

[0192] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. An anomaly analysis method, characterized in that, The method includes: Based on a pre-configured communication feature extraction strategy, features are extracted from the pre-acquired HTTP network data stream to obtain an HTTP communication feature data packet, which contains request association information. Based on the request association information, target anomaly analysis parameters in several HTTP communication feature data packets are parsed to perform system anomaly analysis based on the target anomaly analysis parameters; The HTTP network data stream includes network data packets. The step of extracting features from the pre-acquired HTTP network data stream according to a pre-configured communication feature extraction strategy to obtain HTTP communication feature data packets includes: Extract network packets from the HTTP network data stream to obtain TCP packets by unpacking the network packets, wherein the network packets include HTTP communication characteristic data; If the TCP packet is a data packet, then according to the preset data packet processing strategy, the network data packet is stored in the HTTP communication session record used to store complete communication session data or in the data packet buffer area used to temporarily store data packets that are not explicitly assigned to a session. The preset data packet processing strategy is to determine the storage location based on the association between the sequence number and acknowledgment number of the network data packet and the existing session data, and whether the network data packet explicitly belongs to a specific HTTP communication session. If the TCP packet is an end packet or a reset packet, the network data packet is stored in the HTTP communication session record based on the TCP quadruple corresponding to the data packet. The TCP quadruple is used to uniquely identify different network connections so that the end packet or reset packet of the connection corresponding to the TCP quadruple is accurately classified into the HTTP communication session record to which it belongs. When the preset record output conditions are met, the HTTP communication feature data of the network data packets in the HTTP communication session record is output, so as to generate the HTTP communication feature data packets based on the communication feature data.

2. The method as described in claim 1, characterized in that, If the TCP message is a data packet, the step of storing the network data packet in the HTTP communication session record or the data packet buffer according to the preset data packet processing strategy includes: If the TCP message is a data packet, then obtain the TCP quadruple corresponding to the data packet and sort it to obtain the sorted TCP quadruple; The algorithm queries a preset two-column set to determine whether the HTTP network data stream key-value pair exists in the sorted TCP quad-tuple, in order to determine whether the network data stream corresponding to the current data packet has been tracked. The preset two-column set is a set that stores the currently active HTTP network data streams. The key-value pairs stored in the set use the sorted TCP quad-tuple as the primary key and the HTTP network data stream corresponding to the sorted TCP quad-tuple as the value. If a key-value pair of the HTTP network data stream exists, the network data packet is stored in the HTTP communication session record or the data packet buffer according to the data packet type and the attribute information of the HTTP network data stream. Specifically, if the data packet type matches an existing session and the sequence number is within an existing range, the network data packet is stored in the HTTP communication session record; if the ownership of the network data packet is not clear, the network data packet is stored in the data packet buffer.

3. The method as described in claim 1, characterized in that, The HTTP communication session record includes a first HTTP communication session record and a second HTTP communication session record. The step of obtaining HTTP communication feature data from the HTTP communication session record when a preset record output condition is met, and generating the HTTP communication feature data packet based on the communication feature data, includes: Obtain the sequence number and acknowledgment number of the network data packet, as well as the sequence number and acknowledgment number of the existing response data packet in the second HTTP communication session record. The preset record output condition is: at least one complete HTTP communication session already exists in the first HTTP communication session record, and the sequence number and acknowledgment number of the network data packet are after the sequence number and acknowledgment number of the existing response data packet. If the preset record output conditions are met, a data packet splicing payload is generated based on the target data packet in the first HTTP communication session record. The target data packet is a data packet in the first HTTP communication session record that contains an HTTP request line, a response status line, and core communication feature data. Identify and acquire HTTP communication feature data in the data packet splicing payload; The HTTP communication feature data packet is generated based on the HTTP communication feature data and the pre-acquired metadata information.

4. The method as described in claim 1, characterized in that, The method further includes: Traverse the preset two-column set, which is a set that stores currently active HTTP network data streams. The key-value pairs stored in the set are based on sorted TCP quadtuples as the primary key and the HTTP network data streams corresponding to the sorted TCP quadtuples as the values. Determine whether the current silence duration of the current HTTP network data stream in the dual-column set exceeds a preset silence duration threshold, wherein the current silence duration is the difference between the current time and the last update time of the HTTP network data stream; If the current silence duration exceeds a preset silence duration threshold, then the target key-value pair of the current HTTP network data stream in the two-column set is obtained; The cached data in the current HTTP network data stream is obtained based on the target key-value pair and stored in a preset HTTP communication session record.

5. The method as described in claim 1, characterized in that, The request association information includes a session record ID. The step of parsing target anomaly analysis parameters from several HTTP communication feature data packets based on the request association information, and performing system anomaly analysis based on the target anomaly analysis parameters, includes: Among the HTTP communication feature data packets, the feature data packets with the same session record ID are identified as target data packets, and the target data packets contain request data packets and response data packets; Based on the data packet parameter information of the request data packet and the response data packet, the target anomaly analysis parameters are calculated. The target anomaly analysis parameters include at least one of the following: processing delay, packet sending speed, message interval time, transmission time, message capacity, and session end type. The system anomaly analysis results are obtained by matching the target anomaly analysis parameters with the preset fault analysis rules.

6. An anomaly analysis device, characterized in that, The device includes: The extraction module is used to extract features from a pre-acquired HTTP network data stream according to a pre-configured communication feature extraction strategy to obtain an HTTP communication feature data packet. The communication feature data packet contains request association information, and the HTTP network data stream includes network data packets. The parsing module is used to parse the target anomaly analysis parameters in several HTTP communication feature data packets according to the request association information, so as to perform system anomaly analysis based on the target anomaly analysis parameters; The extraction module is further configured to extract network data packets from the HTTP network data stream, and to unpack the network data packets to obtain TCP packets. The network data packets include HTTP communication characteristic data. If the TCP packet is a data packet, the network data packet is stored in an HTTP communication session record for storing complete communication session data or a data packet buffer for temporarily storing packets not explicitly belonging to a session, according to a preset data packet processing strategy. The preset data packet processing strategy is to determine the storage location based on the association between the sequence number and acknowledgment number of the network data packet and existing session data, and whether the network data packet explicitly belongs to a specific HTTP communication session. If the TCP packet is an end packet or a reset packet, the network data packet is stored in the HTTP communication session record based on the TCP quadruple corresponding to the data packet. The TCP quadruple is used to uniquely identify different network connections so that the end packet or reset packet of the connection corresponding to the TCP quadruple is accurately assigned to the HTTP communication session record to which it belongs. When a preset record output condition is met, the HTTP communication characteristic data of the network data packet in the HTTP communication session record is output to generate the HTTP communication characteristic data packet based on the communication characteristic data.

7. An anomaly analysis device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the anomaly analysis method as described in any one of claims 1 to 5.

8. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the anomaly analysis method as described in any one of claims 1 to 5.

9. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the anomaly analysis method as described in any one of claims 1 to 5.

Images