Abnormal flow detection method, device, equipment, medium and product

Abstract

The embodiment of the invention provides an abnormal traffic detection method and device, equipment, a medium and a product, and relates to the field of data processing.The method comprises the steps that PCAP packet data collected by a preset network interface within a specified time period is directly obtained to obtain the core attribute of a data packet; the target byte position in the PCAP packet data is directly positioned on the basis of a plurality of preset field offsets for feature extraction to obtain structured traffic data, the PCAP packet data is directly analyzed, the intermediate step of traditional text conversion is skipped, and it is not needed that traffic features are firstly extracted in a generalization mode and then compared with entries in a huge rule base one by one in a traditional method; according to the method, the feature matching redundancy step caused by continuous expansion of the rule base is avoided, calculation resource consumption needed by mass real-time traffic comparison is reduced, finally, the structured traffic data are input into the anomaly detection model to recognize the abnormal traffic data which does not conform to the normal traffic mode, rapid recognition of the abnormal traffic is achieved, and the detection efficiency is improved. The technical problem of low detection efficiency in the prior art is solved.

Description

Technical Field

[0001] This application relates to the field of data processing, and in particular to an abnormal flow detection method, apparatus, equipment, medium, and product. Background Technology

[0002] Anomaly traffic detection is used to identify data streams that deviate from normal behavior patterns from massive amounts of network communication data, enabling real-time and accurate monitoring of network traffic, timely detection and early warning of abnormal behavior, and has become an important cornerstone for ensuring cyberspace security and maintaining business continuity.

[0003] In existing technologies, pattern matching is usually performed based on a preset rule base or feature base. This method extracts specific features from network traffic and compares them with a rule base of known attack features. If the traffic features match the attack patterns defined in the rule base, it is determined to be abnormal.

[0004] However, existing technologies suffer from low efficiency in detecting abnormal traffic. As the types of network attacks increase, the rule base needs to be continuously expanded to cover more attack patterns. This means that during traffic detection, each piece of traffic data must be compared with the entries in the huge rule base one by one. The matching process of massive real-time traffic and multiple rules consumes a lot of computing resources, prolongs the detection response time, and makes it difficult to quickly identify abnormal traffic. Summary of the Invention

[0005] This application provides an abnormal flow detection method, apparatus, device, medium, and product to solve the problem of low efficiency in abnormal flow detection in the prior art.

[0006] In a first aspect, embodiments of this application provide an abnormal traffic detection method, including:

[0007] Acquire multiple PCAP packet data; wherein, the multiple PCAP packet data are used to represent the attributes of multiple data packets collected by a preset network interface within a preset time period;

[0008] Based on multiple preset field offsets, feature extraction is performed on the multiple PCAP packet data to obtain multiple structured traffic data; wherein, the multiple field offsets are used to represent multiple byte positions in the multiple PCAP packet data for feature extraction, and each of the structured traffic data refers to the feature data extracted at each of the byte positions.

[0009] The multiple structured traffic data are input into a preset anomaly detection model to obtain multiple abnormal traffic data; wherein, the multiple abnormal traffic data refers to data that does not conform to the preset normal traffic pattern identified from the multiple structured traffic data, and the multiple abnormal traffic data is used to provide data input for network security protection decisions.

[0010] In one possible design, feature extraction is performed on the multiple PCAP packet data based on preset multiple field offsets to obtain multiple structured traffic data, including:

[0011] Offset parsing is performed on the offsets of the multiple fields to obtain the positions of the multiple bytes;

[0012] Based on the multiple byte positions, feature extraction is performed on the multiple PCAP packet data to obtain the multiple structured traffic data.

[0013] In one possible design, parsing the offsets of the plurality of fields to obtain the positions of the plurality of bytes includes:

[0014] Obtain the network protocol version of the multiple PCAP packet data;

[0015] Based on the network protocol version of the multiple PCAP packet data, the offsets of the multiple fields are dynamically compensated to obtain multiple compensated field offsets adapted to the network protocol version; wherein, the dynamic compensation is used to correct the byte position deviation of the feature fields under different protocol versions;

[0016] Offset parsing is performed on the offsets of the multiple compensated fields to obtain the positions of the multiple bytes.

[0017] In one possible design, the anomaly detection model includes an empirical model, a judgment model, and a verification model. The input of the multiple structured traffic data into the preset anomaly detection model yields multiple anomaly traffic data, including:

[0018] The multiple structured traffic data are input into the empirical model to obtain multiple first abnormal data and multiple first normal data; wherein, the empirical model is used to detect data in the multiple structured traffic data that conform to preset attack rules;

[0019] The plurality of first normal data are input into the judgment model to obtain a plurality of second abnormal data; wherein, the judgment model is used to mine abnormal data from the plurality of first normal data;

[0020] The plurality of second abnormal data are input into the verification model to obtain a plurality of third abnormal data; wherein, the verification model is used to verify whether the plurality of second abnormal data are abnormal data or non-abnormal data;

[0021] The multiple first abnormal data and the multiple third abnormal data are integrated to obtain the multiple abnormal traffic data.

[0022] In one possible design, after inputting the multiple structured traffic data into a preset anomaly detection model to obtain multiple abnormal traffic data, the method further includes:

[0023] The multiple abnormal traffic data are input into a preset verification model to obtain multiple final abnormal data and the verification result of each of the abnormal traffic data; wherein, the verification result of each of the abnormal traffic data is used to indicate whether each of the abnormal traffic data is misjudged as abnormal traffic data or is not misjudged as abnormal traffic data, and the multiple final abnormal data refers to the data among the multiple abnormal traffic data that is not misjudged as abnormal traffic data.

[0024] The anomaly detection model is updated based on the multiple final anomaly data to obtain the updated anomaly detection model.

[0025] In one possible design, the verification model includes a first verification model and a second verification model. The step of inputting the multiple abnormal traffic data into the preset verification model to obtain multiple final abnormal data and the verification result of each abnormal traffic data includes:

[0026] Obtain the anomaly type of each of the abnormal traffic data; wherein, the anomaly type of each of the abnormal traffic data includes a first type and a second type, the first type being an anomaly type related to protocol layer attacks on protocol ports, and the second type being an anomaly type related to network group behavior;

[0027] Each of the abnormal traffic data of the first type is input into the first verification model, and each of the abnormal traffic data of the second type is input into the second verification model to obtain the multiple final abnormal data and the verification result of each of the abnormal traffic data; wherein, the first verification model is a feature verification model constructed based on a preset protocol port matching logic, and the second verification model is a group behavior verification model constructed based on a preset node group interaction graph structure.

[0028] In one possible design, after inputting each of the abnormal traffic data of the first type of abnormality into the first verification model and inputting each of the abnormal traffic data of the second type of abnormality into the second verification model to obtain the plurality of final abnormal data and the verification result of each of the abnormal traffic data, the method further includes:

[0029] Obtain the first misjudgment correction rate and the second misjudgment correction rate within the time period; wherein, the first misjudgment correction rate refers to the ratio of the amount of misjudged data successfully removed by the first verification model to the amount of input abnormal traffic data, and the second misjudgment correction rate refers to the ratio of the amount of misjudged data successfully removed by the second verification model to the amount of input abnormal traffic data.

[0030] In response to the first misjudgment correction rate being greater than or equal to a preset first correction rate threshold, the first verification model is updated according to the first misjudgment correction rate to obtain the updated first verification model.

[0031] In response to the second misjudgment correction rate being greater than or equal to a preset second correction rate threshold, the second verification model is updated according to the second misjudgment correction rate to obtain the updated second verification model.

[0032] In one possible design, updating the anomaly detection model based on the multiple final anomaly data to obtain an updated anomaly detection model includes:

[0033] Obtain the PCAP packet characteristics and anomaly type corresponding to each of the final anomaly data;

[0034] A training dataset is constructed based on the PCAP packet features and anomaly types corresponding to each of the final anomaly data.

[0035] The parameters of the anomaly detection model are updated based on the training dataset to obtain the updated anomaly detection model.

[0036] Secondly, embodiments of this application provide an abnormal flow detection device, comprising:

[0037] The first acquisition module is used to acquire multiple PCAP packet data; wherein, the multiple PCAP packet data is used to represent the attributes of multiple data packets collected by a preset network interface within a preset time period;

[0038] The first feature extraction module is used to extract features from the multiple PCAP packet data based on multiple preset field offsets to obtain multiple structured traffic data; wherein, the multiple field offsets are used to represent multiple byte positions in the multiple PCAP packet data for feature extraction, and each of the structured traffic data refers to the feature data extracted at each of the byte positions.

[0039] The first input module is used to input the multiple structured traffic data into a preset anomaly detection model to obtain multiple abnormal traffic data; wherein, the multiple abnormal traffic data refers to data that does not conform to the preset normal traffic pattern identified from the multiple structured traffic data, and the multiple abnormal traffic data is used to provide data input for network security protection decisions.

[0040] In one possible design, the first feature extraction module includes:

[0041] A parsing unit is used to parse the offsets of the multiple fields to obtain the positions of the multiple bytes;

[0042] The feature extraction unit is used to extract features from the multiple PCAP packet data based on the multiple byte positions to obtain the multiple structured traffic data.

[0043] In one possible design, the parsing unit includes:

[0044] The first acquisition component is used to acquire the network protocol version of the plurality of PCAP packet data;

[0045] The compensation component is used to dynamically compensate the offsets of the multiple fields according to the network protocol version of the multiple PCAP packet data, so as to obtain multiple compensated field offsets adapted to the network protocol version; wherein, the dynamic compensation is used to correct the byte position deviation of the feature field under different protocol versions;

[0046] A parsing component is used to parse the offsets of the multiple compensated fields to obtain the positions of the multiple bytes.

[0047] In one possible design, the anomaly detection model includes an empirical model, a judgment model, and a verification model, and the first input module includes:

[0048] The first input unit is used to input the plurality of structured traffic data into the empirical model to obtain a plurality of first abnormal data and a plurality of first normal data; wherein, the empirical model is used to detect data in the plurality of structured traffic data that conform to preset attack rules;

[0049] A judgment unit is used to input the plurality of first normal data into the judgment model to obtain a plurality of second abnormal data; wherein, the judgment model is used to mine abnormal data from the plurality of first normal data;

[0050] A verification unit is used to input the plurality of second abnormal data into the verification model to obtain a plurality of third abnormal data; wherein, the verification model is used to verify whether the plurality of second abnormal data is abnormal data or non-abnormal data;

[0051] An integration unit is used to integrate the plurality of first abnormal data and the plurality of third abnormal data to obtain the plurality of abnormal traffic data.

[0052] In one possible design, the abnormal flow detection device further includes:

[0053] The second input module is used to input the multiple abnormal traffic data into a preset verification model to obtain multiple final abnormal data and the verification result of each of the abnormal traffic data; wherein, the verification result of each of the abnormal traffic data is used to indicate whether each of the abnormal traffic data is misjudged as abnormal traffic data or is not misjudged as abnormal traffic data, and the multiple final abnormal data refers to the data among the multiple abnormal traffic data that is not misjudged as abnormal traffic data.

[0054] The first update module is used to update the anomaly detection model based on the multiple final anomaly data to obtain the updated anomaly detection model.

[0055] In one possible design, the verification model includes a first verification model and a second verification model, and the second input module includes:

[0056] The first acquisition unit is used to acquire the anomaly type of each of the abnormal traffic data; wherein, the anomaly type of each of the abnormal traffic data includes a first type and a second type, the first type is an anomaly type related to protocol layer attacks on protocol ports, and the second type is an anomaly type related to network group behavior;

[0057] The second input unit is used to input each of the abnormal traffic data of the first type of abnormality into the first verification model, and to input each of the abnormal traffic data of the second type of abnormality into the second verification model, so as to obtain the plurality of final abnormal data and the verification result of each of the abnormal traffic data; wherein, the first verification model is a feature verification model constructed based on a preset protocol port matching logic, and the second verification model is a group behavior verification model constructed based on a preset node group interaction graph structure.

[0058] In one possible design, the second input module further includes:

[0059] The second acquisition unit is used to acquire the first misjudgment correction rate and the second misjudgment correction rate within the time period; wherein, the first misjudgment correction rate refers to the ratio of the amount of misjudged data successfully removed by the first verification model to the amount of input abnormal traffic data, and the second misjudgment correction rate refers to the ratio of the amount of misjudged data successfully removed by the second verification model to the amount of input abnormal traffic data.

[0060] The first update unit is configured to update the first verification model according to the first misjudgment correction rate in response to the first misjudgment correction rate being greater than or equal to a preset first correction rate threshold, thereby obtaining the updated first verification model.

[0061] The second update unit is used to update the second verification model according to the second misjudgment correction rate in response to the second misjudgment correction rate being greater than or equal to a preset second correction rate threshold, so as to obtain the updated second verification model.

[0062] In one possible design, the first update module includes:

[0063] The third acquisition unit is used to acquire the PCAP packet characteristics and exception type corresponding to each of the final exception data;

[0064] The first construction unit is used to construct a training dataset based on the PCAP packet features and anomaly types corresponding to each of the final anomaly data.

[0065] The third update unit is used to update the parameters of the anomaly detection model based on the training dataset to obtain the updated anomaly detection model.

[0066] Thirdly, embodiments of this application provide an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0067] The memory stores computer-executed instructions;

[0068] When the processor executes the computer execution instructions stored in the memory, it is used to implement the abnormal traffic detection method as described in any of the first aspects.

[0069] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the abnormal traffic detection method as described in any of the first aspects.

[0070] Fifthly, embodiments of this application provide a computer program product, including a computer program, which, when executed by a processor, is used to implement the abnormal traffic detection method as described in any of the first aspects.

[0071] This application provides an abnormal traffic detection method, apparatus, device, medium, and product. It first directly acquires PCAP packet data collected from a preset network interface within a specified time period to obtain the core attributes of the data packets. Then, based on preset offsets of multiple fields, it directly locates the target byte positions in the PCAP packet data for feature extraction, obtaining structured traffic data. This allows for direct parsing of the PCAP packet data, eliminating the need for traditional methods that first generalize and extract traffic features and then compare them one by one with entries in a large rule base. This avoids redundant feature matching steps caused by continuous rule base expansion and reduces the computational resource consumption required for massive real-time traffic comparison. Finally, the structured traffic data is input into an anomaly detection model to identify abnormal traffic data that does not conform to normal traffic patterns. This replaces the inefficient traditional method of matching rules one by one, shortens the detection response time, and achieves rapid identification of abnormal traffic, solving the technical problem of low detection efficiency in existing technologies. Attached Figure Description

[0072] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0073] Figure 1 This is a schematic diagram illustrating an application scenario of the abnormal traffic detection method provided in the embodiments of this application;

[0074] Figure 2 This is one of the flowcharts illustrating the abnormal traffic detection method provided in the embodiments of this application;

[0075] Figure 3 This is one of the flowcharts illustrating the abnormal traffic detection method provided in the embodiments of this application;

[0076] Figure 4 This is one of the flowcharts illustrating the abnormal traffic detection method provided in the embodiments of this application;

[0077] Figure 5 A schematic diagram of the PCAP packet file structure is provided for embodiments of this application;

[0078] Figure 6 Traffic content diagram provided for embodiments of this application;

[0079] Figure 7 A module configuration diagram provided for an embodiment of this application;

[0080] Figure 8 This is a schematic diagram of a series combination mode provided in an embodiment of this application;

[0081] Figure 9 A diagram illustrating the voting combination pattern provided in this application embodiment;

[0082] Figure 10This is a schematic diagram of the abnormal flow detection device provided in the embodiments of this application;

[0083] Figure 11 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application.

[0084] The accompanying drawings have illustrated specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to specific embodiments. Detailed Implementation

[0085] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0086] In the embodiments of this application, the terms "first" and "second" are used to distinguish identical or similar items with substantially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that "first" and "second" do not necessarily imply difference. It should be noted that in the embodiments of this application, words such as "exemplary" or "for example" are used to indicate that something is being used as an example, illustration, or description. Any embodiment or design scheme described as "exemplary" or "for example" in this application should not be construed as being better or more advantageous than other embodiments or design schemes. Specifically, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner. In the embodiments of this application, "at least one" refers to one or more, and "more than one" refers to two or more.

[0087] It should be noted that the phrase "at...time" in the embodiments of this application can refer to the instant at which a certain situation occurs, or to a period of time after the occurrence of a certain situation. The embodiments of this application do not specifically limit this. In addition, the abnormal traffic detection method, device, equipment, medium and product provided in the embodiments of this application are only examples. An abnormal traffic detection method, device, equipment, medium and product may also include more or less content.

[0088] To facilitate a clear description of the technical solutions in the embodiments of this application, some terms and technologies involved in the embodiments of this application will be briefly introduced below:

[0089] Packet Capture (PCAP) packets are network packet files stored according to the PCAP format specification. Essentially, they are binary records of various data packets during network transmission. These files completely preserve the core information of the data packets, including low-level transmission header information such as the physical layer and data link layer, as well as payload data from the network layer, transport layer, and even application layer. They also include metadata such as the packet capture timestamp and length.

[0090] Unix timestamps are a standardized way of representing time. At their core, they convert time into the number of seconds that have elapsed since January 1, 1970, 00:00:00. In some scenarios, they are extended to millisecond or microsecond precision to meet higher time granularity requirements. This time representation has the advantages of no time zone format ambiguity, strong cross-platform compatibility, and ease of computer storage and time calculation. It is often used in scenarios such as log recording, database time field storage, and network communication event marking.

[0091] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0092] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of the present invention will now be described with reference to the accompanying drawings.

[0093] To clearly understand the technical solution of this application, the solutions of existing technologies will first be described in detail. Abnormal traffic detection is used to identify data flows that deviate from normal behavior patterns from massive amounts of network communication data, enabling real-time and accurate monitoring of network traffic, timely detection and early warning of abnormal behavior, and has become an important cornerstone for ensuring cyberspace security and maintaining business continuity.

[0094] In existing technologies, pattern matching is typically based on a pre-defined rule base or feature base. This method extracts specific features from network traffic and compares them with a rule base containing known attack characteristics. If the traffic features match the attack patterns defined in the rule base, it is considered abnormal. However, as the types of network attacks increase, the rule base needs continuous expansion to cover more attack patterns. This results in each traffic data point being compared with entries in a massive rule base during traffic detection. The matching process of massive real-time traffic and multiple rules consumes significant computing resources, prolongs detection response time, and makes it difficult to quickly identify abnormal traffic. Therefore, existing technologies suffer from low efficiency in abnormal traffic detection.

[0095] Therefore, addressing the low efficiency of abnormal traffic detection in existing technologies, this research found that a structured traffic feature extraction mechanism based on precise field localization, combined with an intelligent detection model, can be constructed to replace the traditional method of comparing all rules one by one. ① Based on the standardized format of network data packets, precise localization rules for feature fields can be preset, directly extracting core and necessary traffic features from the original data packets without parsing and generalizing the entire data. This significantly reduces computational resource consumption in the feature extraction stage and avoids redundant features interfering with subsequent detection processes. ② The feature extraction and anomaly identification stages of the original data packets can be deeply coupled. The extracted structured traffic data can be directly input into a dedicated anomaly detection model for anomaly identification, eliminating the intermediate step of matching each rule in the rule base. Simultaneously, the model's intelligent recognition capability replaces the mechanical comparison of manual rules, achieving an efficient closed loop from feature acquisition to anomaly determination. ③ A unified storage format, field dimensions, and data type for traffic features can be defined to ensure seamless adaptation of the extracted feature data to various anomaly detection models, avoiding repetitive conversion and adaptation work caused by inconsistent feature data formats.

[0096] Specifically, by relying on the standardized transmission structure of network data packets, pre-defined rules for precise location of feature fields can be established. Core traffic features can be extracted directly from the original collected data packets to form a structured and lightweight traffic feature dataset. This replaces the traditional generalized extraction of traffic features and comparison with a large rule base one by one, reducing redundant calculations and ineffective resource consumption in the feature extraction stage. The generated structured traffic data is then directly connected to a dedicated anomaly detection model. The model's intelligent recognition of traffic patterns enables rapid determination of abnormal traffic, achieving an efficient closed loop from the collection of original data packets to the identification of abnormal traffic.

[0097] This application discloses an abnormal traffic detection method, apparatus, device, medium, and product. It first directly acquires PCAP packet data collected from a preset network interface within a specified time period to obtain the core attributes of the data packets. Then, based on preset offsets of multiple fields, it directly locates the target byte positions in the PCAP packet data for feature extraction, obtaining structured traffic data. This allows for direct parsing of the PCAP packet data, eliminating the need for generalized traffic feature extraction and comparison with entries in a large rule base, as is done in traditional methods. This avoids redundant feature matching steps caused by continuous rule base expansion and reduces the computational resource consumption required for massive real-time traffic comparison. Finally, the structured traffic data is input into an anomaly detection model to identify abnormal traffic data that does not conform to normal traffic patterns. This replaces the inefficient traditional rule-by-rule matching method, shortens the detection response time, and achieves rapid identification of abnormal traffic, solving the technical problem of low detection efficiency in existing technologies.

[0098] Based on the above-mentioned inventive discovery, the technical solution of this application is proposed.

[0099] The following describes the application scenarios of the abnormal traffic detection method provided in the embodiments of the present invention. Figure 1 This is a schematic diagram illustrating an application scenario of the abnormal traffic detection method provided in the embodiments of this application. For example... Figure 1 As shown, the application scenario includes a mobile terminal 101 and a server 102. The mobile terminal 101 sends multiple PCAP packet data to the server 102. The server 102 performs feature extraction on the multiple PCAP packet data based on multiple preset field offsets to obtain multiple structured traffic data. The server 102 inputs the multiple structured traffic data into a preset anomaly detection model to obtain multiple abnormal traffic data.

[0100] The embodiments of the present invention will now be described with reference to the accompanying drawings.

[0101] Figure 2 This is one of the flowcharts illustrating the abnormal traffic detection method provided in this application. Figure 2 As shown, in this embodiment, the execution subject of this invention is a server. Therefore, the abnormal traffic detection method provided in this embodiment includes the following steps:

[0102] S201. Obtain multiple PCAP packet data; wherein, the multiple PCAP packet data is used to represent the attributes of multiple data packets collected by a preset network interface within a preset time period.

[0103] Specifically, by deploying data acquisition tools on the network interface and configuring acquisition rules that match a preset time period, multiple data packets flowing through the interface within that time period can be captured in real time to generate PCAP packet data. This step is used to obtain data packet information that accurately reflects the data transmission status of the preset network interface within a specific time period. This provides a data source that meets the detection requirements for subsequent feature extraction based on preset field offsets and the generation of structured traffic data, ensuring the authenticity and relevance of the input data for subsequent anomaly detection models, and providing fundamental support for effectively identifying abnormal traffic.

[0104] S202. Based on multiple preset field offsets, feature extraction is performed on multiple PCAP packet data to obtain multiple structured traffic data; wherein, multiple field offsets are used to represent multiple byte positions in multiple PCAP packet data for feature extraction, and each structured traffic data refers to the feature data extracted at each byte position.

[0105] Specifically, the process begins by parsing the file structure of multiple PCAP packet data sets to identify the header information and payload start position of each PCAP packet's data frame. Then, based on preset offsets for multiple fields, such as the byte positions corresponding to key features like source Internet Protocol (IP) address, destination IP address, transport protocol type, and port number, corresponding feature values ​​are directly extracted from specified byte positions within each PCAP packet. For example, the IP address can be extracted from a specific offset byte segment in the IP header. Subsequently, the extracted feature values ​​from each PCAP packet are integrated according to a preset structured format to form a single structured traffic data entry corresponding to that PCAP packet. Finally, feature extraction is completed for all PCAP packet data, resulting in multiple structured traffic data sets. This step transforms the complex, redundant raw PCAP packet data into well-formatted, structured data that retains only key traffic features, removing useless information that interferes with subsequent processing. It also makes the traffic features more compatible with the input requirements of the anomaly detection model, laying a data foundation for the model to efficiently and accurately identify abnormal traffic data.

[0106] S203. Input multiple structured traffic data into a preset anomaly detection model to obtain multiple abnormal traffic data; wherein, multiple abnormal traffic data refers to data that does not conform to the preset normal traffic pattern identified from multiple structured traffic data, and the multiple abnormal traffic data is used to provide data input for network security protection decisions.

[0107] Specifically, multiple structured traffic data sets can be preprocessed first. The feature dimensions of each data set are adjusted to match the input requirements of a pre-defined anomaly detection model. Numerical features are normalized, and categorical features are encoded and converted to eliminate the impact of data format differences on model computation. Then, the preprocessed structured traffic data is input into the model according to its supported input methods. The model analyzes each structured traffic data set's features against a pre-trained normal traffic pattern, outputting anomaly determination results for each data set. Finally, based on pre-defined anomaly determination criteria, data deemed anomalous is selected from all structured traffic data, resulting in multiple anomalous traffic data sets. This step transforms regular structured traffic data into accurate anomalous traffic identification results, directly locating data that does not conform to normal traffic patterns, providing direct and crucial data support for subsequent network security protection decisions.

[0108] This embodiment provides an abnormal traffic detection method that obtains core attributes of data packets by acquiring PCAP packet data collected from network interfaces within a specified time period. Based on multiple preset field offsets, it directly locates the target byte positions in the PCAP packet data for feature extraction, resulting in structured traffic data. The PCAP packet data is then directly parsed, eliminating the need for generalized traffic feature extraction and comparison with entries in a large rule base, as is done in traditional methods. This avoids redundant feature matching steps caused by continuous rule base expansion and reduces the computational resource consumption required for massive real-time traffic comparison. Finally, the structured traffic data is input into an anomaly detection model to identify abnormal traffic data that does not conform to normal traffic patterns. This replaces the inefficient traditional rule-by-rule matching method, shortens the detection response time, and achieves rapid identification of abnormal traffic, solving the technical problem of low detection efficiency in existing technologies.

[0109] In one possible design, S202, based on multiple preset field offsets, performs feature extraction on multiple PCAP packet data to obtain multiple structured traffic data, including:

[0110] S2021. Parse the offsets of multiple fields to obtain the positions of multiple bytes.

[0111] Specifically, we can first define the reference baseline corresponding to each preset field offset, distinguishing whether it is an absolute offset relative to the start of the PCAP packet data frame or a relative offset relative to the start of the header of the same layer protocol such as the IP header or Transmission Control Protocol (TCP) header. Then, we unify the numerical format of each field offset to decimal byte position values ​​or specific byte ranges, and verify whether the converted byte positions are within the valid byte range of the corresponding PCAP packet data, eliminating invalid offsets that exceed the data length. Finally, we obtain the specific multiple byte positions corresponding to each field offset. This step is used to transform the abstract field offsets into specific byte positions that can be directly used for PCAP packet data feature extraction, ensuring that key traffic features such as source IP address and destination port number can be accurately located and extracted, avoiding feature extraction errors caused by ambiguous offset pointers.

[0112] S2022. Based on multiple byte positions, feature extraction is performed on multiple PCAP packet data to obtain multiple structured traffic data.

[0113] Specifically, the process begins by analyzing the overall data structure of multiple PCAP packets, clarifying the boundaries between the header and payload regions of each data frame. Then, based on the obtained byte positions, target features are extracted from the corresponding byte range of each PCAP packet. This includes extracting key information such as source address, destination address, transmission protocol type, port number, and data packet length from specified byte segments. The extracted features from each PCAP packet are then integrated according to fixed feature dimensions and order to form a single, well-organized data entry corresponding to that PCAP packet. Finally, feature extraction is completed for all PCAP packets, resulting in multiple structured traffic data sets. This step transforms the complex original PCAP packet data, containing a large amount of irrelevant information, into structured data with a uniform format that retains only core traffic features. This eliminates redundant information that interferes with subsequent processes and ensures that the extracted features accurately match the input standards of the anomaly detection model, laying a solid data foundation for efficient identification of abnormal traffic data.

[0114] The technical effect of this scheme in this embodiment is that by parsing the abstract field offset, it is converted into a specific byte position, so that the feature extraction process changes from indirect inference based on protocol semantics to direct positioning based on physical storage location. This eliminates the additional computational overhead caused by protocol stack parsing and string matching in traditional methods, and realizes fast and accurate feature extraction of massive PCAP packet data.

[0115] In one possible design, S2021, offset parsing is performed on multiple field offsets to obtain multiple byte positions, including:

[0116] S20211, Obtain the network protocol version of multiple PCAP packet data.

[0117] Specifically, the protocol header regions of multiple PCAP packets can be parsed one by one to locate the characteristic bytes that identify the network protocol version. Then, the corresponding numerical or encoded information of these characteristic bytes is read. Combined with general network protocol version identification rules, the specific network protocol version corresponding to each PCAP packet can be determined. Simultaneously, the protocol version information of all PCAP packets is summarized to obtain the network protocol versions of multiple PCAP packets. This step is used to clarify the network protocol version used by each PCAP packet, providing crucial version information for subsequent dynamic compensation of field offsets based on different protocol versions. This avoids deviations in the determination of the byte position of characteristic fields due to protocol version differences, ensuring the accuracy of subsequent offset parsing.

[0118] S20212. Based on the network protocol versions of multiple PCAP packet data, dynamically compensate for the offsets of multiple fields to obtain multiple compensated field offsets adapted to the network protocol versions; among them, dynamic compensation is used to correct the byte position deviation of feature fields under different protocol versions.

[0119] Specifically, the process begins by establishing offset compensation mappings for different network protocol versions, clarifying the inherent byte position deviations of feature fields under various protocol versions. Then, for each specific network protocol version of the PCAP packet data, corresponding compensation rules are matched, and targeted adjustments are made to the initial offsets of multiple fields. For example, for byte offsets caused by differences in header structure in a specific protocol version, the corresponding field offset is increased or decreased by the appropriate deviation value. Simultaneously, it is verified whether the compensated offsets are within the valid byte range of the PCAP packet data, eliminating invalid compensation results that exceed the range. Finally, multiple compensated field offsets adapted to the network protocol versions of each PCAP packet data are obtained. This step corrects the byte position deviations of feature fields under different network protocol versions, ensuring that the field offsets accurately match the PCAP packet data structure of the corresponding protocol version, avoiding errors in the byte positions obtained from subsequent offset parsing due to protocol version differences.

[0120] When encountering incompatible protocol versions, such as when PCAP packet data uses an unknown or unpreset protocol version, the dynamic compensation process will activate a preset default processing strategy: automatically classify the packet as an unknown protocol category, skip the dynamic compensation and subsequent feature extraction steps for the packet, and directly mark it as abnormal traffic data to be verified and send it to the subsequent verification model for processing. At the same time, the identification information of the unknown protocol will be recorded and fed back to the protocol version mapping library, triggering the expansion and update of the protocol analysis process, thereby ensuring that the same protocol can be correctly identified and compensated when encountered in the future.

[0121] S20213. Perform offset parsing on multiple compensated field offsets to obtain multiple byte positions.

[0122] Specifically, the reference base corresponding to each compensated field offset can be clearly defined first, distinguishing between an absolute offset base relative to the start of the PCAP packet data frame and a relative offset base relative to the start of the corresponding protocol header. Then, the numerical format of each compensated field offset is uniformly converted to decimal byte measurement form. Simultaneously, it is verified whether the byte range corresponding to the converted offset is within the valid byte range of the corresponding PCAP packet data, eliminating invalid offsets exceeding the overall data length. Finally, each valid compensated field offset is converted into a specific byte position that can be directly used for feature extraction, and the valid results are summarized to obtain multiple byte positions. This step is used to convert the protocol version-adapted compensated field offsets into precise and directly usable byte positions, eliminating offset deviations caused by protocol version differences. This ensures that key traffic features can be extracted from the correct positions in the PCAP packet data, providing reliable location guidance for generating accurate structured traffic data.

[0123] The technical effect of this solution in this embodiment is that it introduces a dynamic offset compensation mechanism based on network protocol version, which enables the feature extraction process to adapt to the field position differences caused by different protocol versions. This avoids feature extraction errors or parsing failures caused by inconsistent protocol versions, and ensures that efficient and accurate structured feature extraction capabilities can still be maintained in a mixed protocol network environment, thereby improving the adaptability of the abnormal traffic detection method in complex network scenarios.

[0124] In one possible design, the anomaly detection model includes an empirical model, a judgment model, and a verification model. S203: Input multiple structured traffic data into the preset anomaly detection model to obtain multiple anomaly traffic data, including:

[0125] S2031. Input multiple structured traffic data into an empirical model to obtain multiple first abnormal data and multiple first normal data; wherein, the empirical model is used to detect data in the multiple structured traffic data that conform to preset attack rules;

[0126] Specifically, multiple structured traffic data sets can first be format-calibrated to unify the feature dimensions and arrangement order of each data set to match the input specifications of the empirical model. Then, the calibrated structured traffic data is batch-input into the empirical model. Based on built-in preset attack rules, the model compares the feature information of each data set one by one, filtering out multiple first-order abnormal data sets that meet the attack rules and multiple first-order normal data sets that do not meet the attack rules. This step is used to quickly separate traffic data with obvious attack characteristics from traffic data that has not yet been determined to be abnormal, providing a clearly categorized data source for the subsequent model to specifically mine potential anomalies, ensuring that the anomaly detection process proceeds in an orderly manner according to hierarchy.

[0127] Among them, the empirical model can be an anomaly identification model built based on pre-defined rules summarizing known attack characteristics and historical data. Through explicit threshold settings, regular expression matching, or feature comparison logic, it can quickly filter out traffic data with obvious anomaly indicators, completing the initial judgment without complex calculations. Its core is to directly apply mature attack detection rules, focusing on verified and effective anomaly features to achieve rapid identification of high-frequency, typical anomaly traffic, while outputting traffic data that does not match the rules for subsequent in-depth detection.

[0128] For example, an empirical model is designed to address distributed denial-of-service attacks. The default rule is that a single source IP sends more than 1000 TCP connection requests to the same destination IP within 10 seconds, and it also includes high-frequency access thresholds for common attack ports. When structured traffic data shows that a source IP makes 1200 TCP connection requests to a destination IP within 10 seconds, and the target port is 80, the empirical model directly classifies this traffic as the first abnormal data. If the traffic data does not trigger any default rules, it is classified as the first normal data and passed to the judgment model.

[0129] S2032. Input multiple first normal data into the judgment model to obtain multiple second abnormal data; wherein, the judgment model is used to mine abnormal data from multiple first normal data;

[0130] Specifically, multiple sets of first-order normal data can be first processed by feature normalization, unifying feature dimensions and data formats to suit the input requirements of the judgment model. Then, the normalized first-order normal data is batch-input into the judgment model. Based on pre-trained normal traffic behavior patterns and feature association logic, the model identifies and filters out potential abnormal data that deviates from the pattern, resulting in multiple second-order abnormal data. This step is used to mine hidden abnormal information from traffic data that the empirical model did not classify as abnormal, supplementing the detection range of the empirical model and providing targeted verification data for subsequent model validation, ensuring the comprehensiveness of abnormal traffic detection.

[0131] The judgment model is an intelligent model trained and built based on algorithms such as machine learning and deep learning. By learning the behavioral patterns and feature correlation logic of massive amounts of normal traffic, it can uncover hidden and atypical potential anomalies from the first normal data that the empirical model did not identify as abnormal. Its core is to break through the limitations of preset rules, identify unknown attacks or highly disguised abnormal traffic, supplement the detection blind spots of the empirical model, and output second abnormal data that needs further verification.

[0132] For example, a random forest algorithm can be used to build a judgment model. First, 100,000 normal traffic data points and 20,000 hidden abnormal data points are collected. Fifteen features, such as source IP access frequency, packet size distribution, and connection duration, are extracted for model training to determine the behavioral benchmark for normal traffic. When the first normal data point output by the empirical model is input, the model analyzes the access characteristics of a certain source IP: sending one probe packet to the destination IP every 30 seconds for one hour, with a fixed packet size of 64 bytes. This significantly deviates from the trained normal pattern, and the traffic is then judged as the second abnormal data point.

[0133] S2033. Input multiple second abnormal data into the verification model to obtain multiple third abnormal data; wherein, the verification model is used to verify whether the multiple second abnormal data are abnormal data or non-abnormal data;

[0134] Specifically, the process begins by completing the features and standardizing the format of multiple second-order abnormal data sets to ensure that the data dimensions match the judgment logic of the verification model. Then, the processed second-order abnormal data is batch-input into the verification model. Based on preset normal traffic characteristic benchmarks and abnormality authenticity judgment rules, the model identifies the abnormal attributes of each data set, filtering out data confirmed as genuine abnormalities, thus obtaining multiple third-order abnormal data sets. This step is used to eliminate false positives from the second-order abnormal data, refining the truly valid abnormal information and providing reliable data support for subsequent integration with the first-order abnormal data to form a complete abnormal traffic result, ensuring the accuracy of the abnormality detection results.

[0135] The verification model is a secondary verification model built on the second abnormal data output by the judgment model. Based on preset normal traffic characteristic benchmarks, protocol logic, or group behavior rules, it identifies the authenticity of abnormal data attributes, eliminates falsely judged data, and purifies valid abnormal data. Its core is to ensure the accuracy of anomaly judgment through double verification, avoid black-box misjudgments by the judgment model, and provide highly reliable data support for the final abnormal traffic integration.

[0136] For example, a combined verification model of protocol feature verification and group behavior verification can be constructed. For the second abnormal data output by the judgment model, protocol feature verification is performed: checking whether the port number (port 22), protocol type (SSH), and packet interaction logic in the traffic match the SSH protocol specification, eliminating false positives where the port is not 22 but is still judged as an SSH attack; then, group behavior verification is performed: constructing an IP interaction graph, analyzing the connection relationship between the source IP and other nodes. If the source IP only has low-frequency connections with the target IP and no other normal interaction nodes (consistent with the isolated behavior characteristics of brute-force attacks), it is confirmed as a genuine anomaly, and the third abnormal data is output. If the source IP has regular interactions with multiple normal nodes, it is judged as a false positive and is not identified as the third abnormal data.

[0137] S2034. Integrate multiple first-abnormal data and multiple third-abnormal data to obtain multiple abnormal traffic data.

[0138] Specifically, the process begins by aligning the formats of multiple primary and secondary anomaly data points to standardize their characteristic fields and representations. Duplicate records are then removed using pre-defined deduplication rules. Following this, the data is categorized and organized according to anomaly type or data characteristics. Finally, all deduplicated and categorized valid data is aggregated and integrated to obtain multiple anomaly traffic data sets. This step combines explicitly anomaly data selected by the experience model with genuine anomaly data refined by the verification model, resulting in comprehensive and non-redundant anomaly traffic results. This provides a complete and accurate data source for network security protection decisions.

[0139] The technical effect of this solution in this embodiment is as follows: By constructing a hierarchical detection architecture of experience model + judgment model + verification model, the experience model is first used to quickly filter out the first abnormal data that meets the preset attack rules, avoiding the omission of obvious abnormal traffic. Then, the judgment model is used to deeply mine the potential anomalies in the first normal data that the experience model did not identify, generating the second abnormal data, which makes up for the shortcomings of traditional single models in detecting hidden anomalies. Finally, the verification model verifies the authenticity of the second abnormal data, eliminates the falsely judged data to obtain the third abnormal data, effectively reducing the false alarm rate. Finally, the two types of valid abnormal data are integrated to form a complete abnormal traffic result. This not only achieves comprehensive coverage of different types of abnormal traffic, but also balances detection efficiency and judgment accuracy through hierarchical verification, solving the technical problems of high false alarm rate and missed detection in traditional anomaly detection methods.

[0140] Figure 3 This is one of the flowcharts illustrating the abnormal traffic detection method provided in this application. In this embodiment, in Figure 2 Based on the provided embodiments, the abnormal traffic detection method is further explained. The abnormal traffic detection method includes:

[0141] S301. Obtain multiple PCAP packet data; wherein, the multiple PCAP packet data is used to represent the attributes of multiple data packets collected by a preset network interface within a preset time period.

[0142] S302. Based on multiple preset field offsets, feature extraction is performed on multiple PCAP packet data to obtain multiple structured traffic data; wherein, multiple field offsets are used to represent multiple byte positions in multiple PCAP packet data for feature extraction, and each structured traffic data refers to the feature data extracted at each byte position.

[0143] S303. Input multiple structured traffic data into a preset anomaly detection model to obtain multiple abnormal traffic data; wherein, multiple abnormal traffic data refers to data that does not conform to the preset normal traffic pattern identified from multiple structured traffic data, and the multiple abnormal traffic data is used to provide data input for network security protection decisions.

[0144] S301-S303 are similar to S201-S203, and will not be described again in this embodiment.

[0145] S304. Input multiple abnormal traffic data into a preset verification model to obtain multiple final abnormal data and the verification result of each abnormal traffic data; wherein, the verification result of each abnormal traffic data is used to indicate whether each abnormal traffic data is misjudged as abnormal traffic data or is not misjudged as abnormal traffic data, and multiple final abnormal data refers to the data among multiple abnormal traffic data that is not misjudged as abnormal traffic data.

[0146] Specifically, the process begins by completing the features and standardizing the format of multiple abnormal traffic data sets. This ensures that the feature dimensions and data format of each data set conform to the input standards of the preset verification model. The processed abnormal traffic data is then batch-input into the verification model. The model uses built-in normal traffic feature benchmarks and abnormal judgment verification rules to verify the feature matching degree of each abnormal traffic data set, distinguishing between traffic data that is misjudged as abnormal and truly abnormal traffic data. It generates verification results for each abnormal traffic data set and simultaneously filters out truly abnormal data that was not misjudged from all abnormal traffic data, integrating them to form multiple final abnormal data sets. This step is used to verify the abnormal traffic data output by the anomaly detection model, eliminating misjudged data caused by model identification bias, refining the truly valid abnormal traffic data, and generating corresponding verification results. This provides an accurate and reliable data foundation for subsequent updates to the anomaly detection model based on real abnormal data.

[0147] The verification model is an intermediate verification component within the anomaly detection model. It verifies whether the second abnormal data mined from the first normal data by the judgment model is genuine abnormal data, and filters out valid abnormal data as the third abnormal data, serving the hierarchical detection process within the anomaly detection model. The validation model is a subsequent validation component after the anomaly detection model outputs the overall abnormal traffic data. It receives the abnormal traffic data after integrating the first and third abnormal data, identifies the misjudged data, and outputs the final abnormal data. It also provides data support for the parameter updates of the anomaly detection model. The difference between the two is that the verification model is located inside the anomaly detection model and processes the second abnormal data in the intermediate stage, with the function of filtering the authenticity of the intermediate abnormal data. The validation model is located outside the anomaly detection model in the subsequent process and processes the integrated complete abnormal traffic data, with the function of eliminating overall misjudged data and supporting the iterative updates of the model.

[0148] S305. Update the anomaly detection model based on multiple final anomaly data to obtain the updated anomaly detection model.

[0149] Specifically, the process begins by labeling and classifying multiple final anomaly data sets to clarify the anomaly type and core feature dimensions for each data point. The labeled final anomaly data is then integrated into the training dataset of the anomaly detection model. Incremental training is conducted using existing normal traffic data and historical anomaly data to adjust the model's internal feature weights and judgment thresholds. Simultaneously, the adjusted model's performance is validated to ensure its ability to identify new anomaly types meets preset requirements, ultimately resulting in an updated anomaly detection model. This step integrates real and effective new anomaly traffic features into the model, optimizes the model's anomaly judgment logic, improves the model's accuracy in identifying new types of network anomaly traffic, and allows the model to continuously adapt to changes in network attack patterns, ensuring reliability and effectiveness in long-term detection.

[0150] After updating the anomaly detection model based on multiple final anomaly data, and obtaining the updated anomaly detection model, a multi-dimensional correlation analysis can be performed on the anomaly level, impact range, and associated network nodes of multiple anomaly traffic data. A dynamic risk map is constructed by combining the propagation path of the anomaly data and potential attack intent. Then, a hierarchical protection response mechanism is activated based on the risk map hierarchy. For high-risk nodes, temporary port isolation and traffic cleaning operations are triggered first. For medium-risk areas, protocol layer feature filtering and temporary access control policies are activated. For low-risk data, behavior monitoring and source tracing processes are initiated. At the same time, the entire network protection components are linked to achieve cross-regional synchronous adaptation of policies. The protection rule base can also be updated based on the type characteristics of anomaly data, so that the protection decision has both real-time emergency response capabilities and forms a long-term protection closed loop. In addition, the risk transmission prediction logic of anomaly traffic is introduced. By analyzing the node correlation of anomaly data, the potentially affected areas can be predicted in advance. Protection deployment can be completed before the attack behavior spreads completely, realizing an upgrade from passive response to proactive prediction of the protection mode.

[0151] The technical effect of this solution in this embodiment is that it effectively eliminates false alarm data generated by the anomaly detection model through secondary verification, generates anomaly judgment results with high confidence, and uses the results to iteratively update the anomaly detection model in reverse. Thus, it continuously corrects model deviations and adapts to new attack modes during the continuous detection process, thereby improving detection accuracy and enabling system adaptive evolution. It solves the problem of long-term detection performance decay caused by the inability of traditional static rule bases or models to self-correct.

[0152] In one possible design, the verification model includes a first verification model and a second verification model. S304: Input multiple abnormal traffic data into the preset verification model to obtain multiple final abnormal data and the verification results of each abnormal traffic data, including:

[0153] S3041. Obtain the anomaly type of each abnormal traffic data; wherein, the anomaly type of each abnormal traffic data includes a first type and a second type. The first type is an anomaly type related to protocol layer attacks on protocol ports, and the second type is an anomaly type related to network group behavior.

[0154] Specifically, the core characteristics of each abnormal traffic data point can be analyzed first, extracting features associated with protocol ports and those related to network node interactions. Then, compared to pre-defined anomaly type classification criteria, abnormal traffic data containing protocol-layer attack-related features (including those related to protocol ports) is classified as Type 1, and abnormal traffic data containing features related to network group behavior is classified as Type 2. This clarifies the specific anomaly type corresponding to each abnormal traffic data point. This step is used to accurately classify abnormal traffic data with different characteristic attributes, providing a classification basis for subsequently inputting different types of abnormal traffic data into the appropriate verification model. This avoids verification deviations caused by mismatches between the verification model and the abnormal traffic type, improving the accuracy and adaptability of the overall verification process.

[0155] The first type refers to anomalies where the data packets themselves violate the established specifications or semantic logic of a certain layer in the TCP / IP network protocol stack. Examples include invalid protocol header field values, abnormal state machine sequences, or abuse of standard protocol commands for specific ports. The second type refers to situations where a single data packet may conform to protocol specifications, but multiple data packets exhibit coordinated or aggregated behavior patterns that deviate from the normal baseline at a macroscopic level. Examples include scanning behavior from distributed source addresses towards the same target within a time window, or sudden cluster deviations in the request rate of a specific service. This classification standard will be integrated into the classification step in the form of a feature logic rule base, serving as a definitive basis for determining the type of abnormal traffic data.

[0156] S3042. Input each abnormal traffic data of the first type of abnormality into the first verification model, and input each abnormal traffic data of the second type of abnormality into the second verification model to obtain multiple final abnormal data and the verification results of each abnormal traffic data; wherein, the first verification model is a feature verification model constructed based on a preset protocol port matching logic, and the second verification model is a group behavior verification model constructed based on a preset node group interaction graph structure.

[0157] Specifically, the process begins by aligning the protocol port-related feature dimensions of the first type of abnormal traffic data, and supplementing the network node group interaction-related feature information of the second type of abnormal traffic data. This ensures that the formats of both types of data conform to the input requirements of their respective verification models. The processed first-type abnormal traffic data is then fed into the first verification model, which uses its built-in protocol port matching logic to verify whether the protocol layer features of each data point match the characteristic benchmarks of a real attack. Simultaneously, the processed second-type abnormal traffic data is fed into the second verification model, which uses a pre-defined node group interaction graph structure to analyze whether the group behavior characteristics of the data deviate from normal interaction patterns. The verification results from both models are then integrated, and the abnormal data that was not misjudged is selected and aggregated into multiple final abnormal data sets. The verification results for each abnormal traffic data set are also organized. This step uses appropriate verification logic for different types of abnormal traffic data, avoiding the judgment bias caused by a single verification standard, and improving the accuracy of abnormal data verification. It not only refines the truly valid final abnormal data but also generates traceable verification results for each data set, providing high-quality and clearly categorized data support for the accurate updating of subsequent anomaly detection models.

[0158] The first verification model is a feature verification model built on a pre-defined protocol port matching logic. Its core mechanism lies in verifying the compliance of specific fields, sequences, and states of data packets according to strictly defined protocol specifications and port usage rule bases. The second verification model is a group behavior verification model built on a pre-defined node group interaction graph structure. Its core mechanism lies in abstracting traffic data into a graph of network nodes and interaction relationships, and using graph algorithms to analyze macroscopic features such as connection patterns and behavioral clustering of node groups. The core difference between the two lies in the dimensions and objects of verification: the first verification model focuses on the microscopic compliance of single or a few data packets at the protocol semantic level, relying on deterministic rule logic; the second verification model focuses on macroscopic anomalies in the interaction behavior between network entities reflected by a large number of data packets, relying on graph structure and group behavior pattern recognition.

[0159] The technical effect of this solution in this embodiment is as follows: by refining the verification model into dedicated verification modules for different anomaly types and performing diversion processing on the abnormal data, the verification process is made more refined and professional: protocol port verification quickly filters false alarms of logical contradictions, and group behavior verification deeply identifies false judgments of outliers. This collaborative verification system reduces the overall false alarm rate, optimizes the allocation of verification resources, improves the verification efficiency and accuracy in complex attack scenarios, and enhances the interpretability and credibility of the detection results.

[0160] In one possible design, S305 updates the anomaly detection model based on multiple final anomaly data to obtain an updated anomaly detection model, including:

[0161] S3051. Obtain the PCAP packet characteristics and exception type corresponding to each final exception data.

[0162] Specifically, a correspondence can be established between each final anomaly data point and the original PCAP packet data. This involves tracing the original PCAP packet data corresponding to each final anomaly data point, extracting the core features previously obtained based on field offsets from this original data, and combining this with existing anomaly type classification criteria to clarify the specific anomaly type corresponding to each final anomaly data point. During this process, the consistency of the correlation information must be verified to ensure that the extracted PCAP packet features and the determined anomaly types completely match the corresponding final anomaly data, avoiding mismatches in features or types. This step provides accurate and complete core data for the subsequent construction of the training dataset, ensuring that the training dataset contains both specific traffic feature information and clear anomaly type labels, laying the data foundation for subsequent parameter updates to the anomaly detection model.

[0163] S3052. Construct a training dataset based on the PCAP packet features and anomaly types corresponding to each final anomaly data.

[0164] Specifically, the process begins by integrating the PCAP packet features and anomaly types corresponding to each final anomaly data point. This involves matching each final anomaly data point with a corresponding feature set and type label. Next, feature and type information from existing normal traffic data and historical anomaly data are added to enrich the data coverage. Then, all integrated data is formatted to standardize feature dimensions and representation, ensuring the input format meets the basic requirements for model training. Finally, the integrated data undergoes integrity verification, eliminating invalid data with missing features or type mismatches. This results in a structurally complete and clearly labeled training dataset. This step provides high-quality and comprehensive data support for updating the anomaly detection model's parameters, ensuring the training dataset contains both the features and types of new anomaly traffic while retaining valid information from existing data. This guarantees that subsequent model parameter updates can be optimized based on diverse data.

[0165] S3053. Update the parameters of the anomaly detection model based on the training dataset to obtain the updated anomaly detection model.

[0166] Specifically, the process begins by aligning the feature dimensions of the training dataset with the input dimensions of the anomaly detection model, thus establishing a mapping between features and the model's recognition logic. Next, the model's incremental training logic is invoked to integrate the training dataset into the existing model's learning framework. This involves adjusting the feature association weights and anomaly judgment benchmarks within the model, while introducing dedicated recognition branches for new anomaly types. Finally, the adjusted model undergoes functional verification to ensure its ability to recognize both new and old anomaly types meets predefined requirements, ultimately resulting in an updated anomaly detection model. This step integrates new anomaly traffic features and type information into the model, optimizes the model's parameter configuration and judgment logic, compensates for the original model's shortcomings in recognizing new anomalies, and improves the model's accuracy and adaptability in recognizing various types of anomaly traffic.

[0167] The technical effect of this solution in this embodiment is that by explicitly utilizing the original PCAP packet features and confirmed anomaly types contained in the final anomaly data as core training samples, a training dataset with high confidence labels is constructed. This provides an accurate and reliable incremental learning basis for the parameter update of the anomaly detection model, avoiding the noise and bias introduced by relying on unverified anomaly data or manual annotation in the traditional model update process. This ensures that the model can continuously and stably improve its ability to identify real threats and its generalization performance during iterative optimization.

[0168] Figure 4 This is one of the flowcharts illustrating the abnormal traffic detection method provided in this application. In this embodiment, in Figure 3 Based on the provided embodiments, the abnormal traffic detection method is further explained. The abnormal traffic detection method includes:

[0169] S401. Obtain the anomaly type of each abnormal traffic data; wherein, the anomaly type of each abnormal traffic data includes a first type and a second type. The first type is an anomaly type related to protocol layer attacks on protocol ports, and the second type is an anomaly type related to network group behavior.

[0170] S402. Input each abnormal traffic data of the first type of abnormality into the first verification model, and input each abnormal traffic data of the second type of abnormality into the second verification model to obtain multiple final abnormal data and the verification results of each abnormal traffic data; wherein, the first verification model is a feature verification model built based on a preset protocol port matching logic, and the second verification model is a group behavior verification model built based on a preset node group interaction graph structure.

[0171] S401-S402 are similar to S3041-S3042, and will not be described again in this embodiment.

[0172] S403. Obtain the first misjudgment correction rate and the second misjudgment correction rate within the time period; wherein, the first misjudgment correction rate refers to the ratio of the amount of misjudged data successfully removed by the first verification model to the amount of input abnormal traffic data, and the second misjudgment correction rate refers to the ratio of the amount of misjudged data successfully removed by the second verification model to the amount of input abnormal traffic data.

[0173] Specifically, the process involves first calculating the total amount of abnormal traffic data input to the first verification model within a preset time period, then calculating the amount of misjudged data successfully removed by the model during this period. The first misjudgment correction rate is obtained by calculating the ratio of the latter to the former. Simultaneously, the total amount of abnormal traffic data input to the second verification model within the same time period and the amount of misjudged data successfully removed by that model are calculated. The second misjudgment correction rate is obtained by calculating the ratio of these two. During this process, it is necessary to verify that the calculation range of the data strictly corresponds to the preset time period. This step quantifies the misjudgment correction effectiveness of each verification model within the preset time period, providing a quantifiable basis for subsequent judgments on whether to update the corresponding verification models. This ensures that model update decisions are supported by objective data rather than subjective judgment, guaranteeing that both types of verification models maintain a consistently high misjudgment correction capability.

[0174] S404. In response to the first misjudgment correction rate being greater than or equal to a preset first correction rate threshold, the first verification model is updated according to the first misjudgment correction rate to obtain the updated first verification model.

[0175] Specifically, the process begins by collecting misjudged and valid verification data processed by the first verification model within the specified time period, based on the initial false positive correction rate. This analysis reveals weaknesses in the protocol port matching logic, identifies biases in the protocol layer attack-related verification rules, and integrates the identified rule optimization directions into the model's core verification logic. This involves adjusting the feature weights and judgment thresholds corresponding to protocol port matching, supplementing with newly emerging protocol layer attack-related verification samples within the specified time period, and conducting functional verification on the adjusted model to ensure its verification logic adapts to new protocol layer attack characteristics. This results in an updated first verification model. This step is used to specifically optimize the verification logic when the first verification model's false positive correction effectiveness reaches a preset standard, compensating for the deficiencies of the original protocol port matching rules and improving the model's accuracy in verifying the first type of abnormal traffic data.

[0176] S405. In response to the second misjudgment correction rate being greater than or equal to the preset second correction rate threshold, the second verification model is updated according to the second misjudgment correction rate to obtain the updated second verification model.

[0177] Specifically, the process begins by collecting misjudged and valid verification data processed by the second verification model within the specified time period, based on the specific value of the second misjudgment correction rate. This involves analyzing the weaknesses in the verification logic related to the node group interaction graph structure within the model, identifying the judgment biases in the network group behavior-related verification rules, and then integrating the identified rule optimization directions into the model's core verification logic. This involves adjusting the association weights and judgment benchmarks corresponding to the node group interaction graph, supplementing with newly emerging network group behavior-related verification samples within the specified time period, and conducting functional verification on the adjusted model to ensure its verification logic can adapt to new network group behavior-related anomalies, resulting in an updated second verification model. This step is used to specifically optimize the verification logic when the misjudgment correction effectiveness of the second verification model reaches a preset standard. It compensates for the deficiencies in the original node group interaction graph structure-related rules, improves the model's verification accuracy for second-type abnormal traffic data, and allows the model to continuously adapt to changes in network group behavior-related attack characteristics, maintaining stable misjudgment correction capabilities.

[0178] The technical effect of this solution in this embodiment is as follows: by introducing an adaptive update mechanism for the verification model based on the false positive correction rate, continuous performance monitoring and dynamic optimization of the first and second verification models are achieved. When the false positive correction capability of a specific verification model reaches a preset threshold within a time period, its parameters are adjusted accordingly, thereby ensuring that the entire verification system can evolve autonomously with changes in network attack patterns and traffic characteristic distribution, and maintain and continuously improve its ability to identify and filter new false positive patterns.

[0179] It should be noted that PCAP packets, also known as network data packets, are transmitted in the form of data packets during network communication. In traditional network security testing, engineers cannot directly process data packets; they must first convert them into structured data before testing. Common detection methods rely on existing regular expression conditions or threshold settings. Some have introduced artificial intelligence models to identify abnormal traffic, but these are not comprehensive enough. In practical network applications, the former suffers from many missed detections, while the latter has a large number of false positives. To address this issue, this application proposes a more complete detection mode for identifying abnormal traffic. This application can be divided into three modules:

[0180] 1. Data Preprocessing Module: This module primarily receives PCAP packets and performs data preprocessing, extracting information from the traffic to form structured data. Generally, when extracting features, engineers convert the traffic packets into readable content before extracting the desired features. However, in reality, PCAP packets have specific meanings at each fixed location. Therefore, it's sufficient to directly truncate and extract the underlying data from specific locations within the PCAP packet. Figure 5 As shown, Figure 5This application provides a schematic diagram of the PCAP packet file structure, specifying the meaning of traffic represented within a certain number of bytes. Figure 6 The traffic content graph provided in this application embodiment is essentially hexadecimal data. We only need to extract features directly from the hexadecimal file. This can improve computing performance and save storage space. Streaming processing can be used to parse the PCP packets and then convert the features into structured data.

[0181] 2. Anomaly Detection Module:

[0182] The anomaly detection module consists of three parts: an empirical model, a judgment model, and a verification model.

[0183] (1) Empirical models, which mainly consist of traditional detection models for identifying abnormal network behaviors. For example, certain known malicious attack traffic has obvious characteristic features that can be directly identified by regular expression matching, or certain attack behaviors have obvious threshold features that can be identified by using the feature thresholds obtained from historical attack behaviors, such as DDoS. The models here are often highly accurate, and the abnormal traffic obtained can be directly output as abnormal traffic results. Traffic that has not been judged will be output to the judgment model for further judgment.

[0184] (2) Judge the model and learn to identify more complex abnormal attack behaviors. These may not be able to be identified by simply using an experience model. Often, a large amount of sample data is needed to let the model learn and discover the complex correlation between features, thereby identifying abnormal traffic. The model here does not refer to a single model. It can be a machine learning model, a deep learning model, or even multiple different trained models.

[0185] (3) Verification Model: The judgment model is sometimes called a black box because it is impossible to clearly define how it identifies abnormal traffic. Therefore, some traffic may have strong expressive power in certain characteristics, which has a high impact on the model's judgment and thus leads to misjudgment. The main function of the verification model is to correct biases and perform a secondary judgment on the output of the judgment model. This mainly involves two verification methods:

[0186] a) Typical feature verification: For example, when building a decision tree model, in order to ensure that there is a lot of training data so that the model can learn abnormal traffic from various features, the brute-force feature data of various protocols can be put into a dataset for multi-class training. However, in the actual network, there may be brute-force attacks of some protocols that are assigned to brute-force attacks of other protocols. In this case, we can use a verification model to verify and correct it by using port or protocol information.

[0187] b) Group Behavior Verification: The results of packet detection are used to identify group behavior among IPs using graph structures. If the detection system classifies it as "abnormal traffic," and the graph model also considers it a structural outlier, then this judgment is consistent and reliable, with a high probability of being correct. Conversely, if the detection system classifies it as "abnormal," but the graph model considers it to be within a normal, dense connection community (e.g., normal traffic from a frequently used update server is misjudged), then this judgment may be a false alarm with low reliability.

[0188] 3. Result Output Module: This module mainly outputs the finally detected abnormal traffic. The format can be a list or a file, etc. Figure 7 This is a module configuration diagram provided for an embodiment of this application.

[0189] The implementation steps are as follows:

[0190] 1. Network data packets: The original PCAP packets or specific PCAP packets to be detected can be obtained from the network side.

[0191] 2. Packet Preprocessing: PCAP packets are identified in hexadecimal format, and features are extracted from different specified byte positions. For example, the first 4 bytes in the packet header represent a timestamp, accurate to seconds. This is a Unix timestamp, and the time of packet capture is typically extracted using this feature. Feature extraction forms structured data for later use. Here, the five-tuples of traffic data can be aggregated to obtain the corresponding traffic logs.

[0192] 3. Empirical Model: Based on existing rule models, threshold settings, and regular expression matching of typical features, abnormal traffic is identified. The empirical model here does not refer to a single model or a type of model, but rather to a combination of the models mentioned above. The results of the data identified as abnormal traffic are output, and the traffic data that is not identified is passed to the next stage (judgment model).

[0193] 4. Judgment Model: Data not identified as abnormal traffic in the previous step is input into the judgment model. The judgment model includes, but is not limited to, random forest models and K-means models from machine learning, as well as RNN models and LSTM models from deep learning. Furthermore, it is not limited to any single model; it can be a combination of multiple models, such as... Figure 8 This is a diagram illustrating a series combination mode provided in an embodiment of this application. Figure 9 A diagram illustrating the voting combination pattern provided in this application embodiment.

[0194] 5. Verification Model: Data marked as abnormal traffic by the model output can be first verified using typical features to filter out problematic results. Then, a graph model is constructed for the remaining data, with the dataset being [(Attack IP1, Destination IP1, Attack Type 1), (Attack IP2, Destination IP2, Attack Type 2)].

[0195]

Step 1

[0196] (Attack IP) - [Attack Type] -> (Destination IP).

[0197] Nodes: Only IP nodes.

[0198] Side: From the attacking IP to the destination IP.

[0199] Edge attribute: Attack type is a label or attribute of an edge, and an edge represents a specific attack event.

[0200]

Step 2

[0201] Out-degree: The number of connections initiated by an IP address as an attacking IP. IPs with extremely high out-degree are likely the controllers of scanners or botnets.

[0202] In-degree: The number of times an IP address is used as a destination IP for connections. Servers with extremely high in-degrees may be key attack targets, or the IP address may be a decoy (honeypot).

[0203] Degree centrality: The total number of connections (out-degree + in-degree) to an IP address. Nodes with high centrality play a key role in the network.

[0204] Edge weight: The number of times that specific attack traffic occurs. A single, isolated attack event may be more suspicious (it could be a slow attack or a precision strike), while high-frequency repetition may be a brute-force attack, and its novelty and anomaly may be lower.

[0205] Public neighbors: The number of other IPs that are commonly connected between the attacking IP and the target IP.

[0206]

Step 3

[0207] Create a feature vector for each judgment:

[0208] [Out-degree (attacking IP), in-degree (destination IP), degree centrality, edge weight, number of common neighbors]

[0209] Unsupervised anomaly detection:

[0210] The Isolation Forest algorithm is used to analyze the entire feature vector dataset. Isolation Forest excels at quickly identifying points that behave differently from the majority of the data.

[0211] determination:

[0212] If an edge corresponding to (attacker IP, destination IP, attack type) is marked as an outlier by these algorithms, it indicates that the graph structure is abnormal. This determination is also considered a high-probability anomalous traffic.

[0213] 6. Abnormal Traffic: The final output is the traffic that is marked as abnormal after passing through the overall detection module.

[0214] Figure 10 This is a schematic diagram of the abnormal flow detection device provided in an embodiment of this application. Figure 10 As shown, the abnormal flow detection device includes:

[0215] The first acquisition module 1001 is used to acquire multiple PCAP packet data; wherein, the multiple PCAP packet data is used to represent the attributes of multiple data packets collected by a preset network interface within a preset time period;

[0216] The first feature extraction module 1002 is used to extract features from multiple PCAP packet data based on multiple preset field offsets to obtain multiple structured traffic data; wherein, the multiple field offsets are used to represent multiple byte positions in the multiple PCAP packet data, and each structured traffic data refers to the feature data extracted at each byte position.

[0217] The first input module 1003 is used to input multiple structured traffic data into a preset anomaly detection model to obtain multiple abnormal traffic data; wherein, the multiple abnormal traffic data refers to the data that does not conform to the preset normal traffic pattern identified from multiple structured traffic data, and the multiple abnormal traffic data is used to provide data input for network security protection decisions.

[0218] In one possible design, the first feature extraction module 1002 includes:

[0219] The parsing unit is used to parse the offsets of multiple fields to obtain the positions of multiple bytes.

[0220] The feature extraction unit is used to extract features from multiple PCAP packet data based on multiple byte positions to obtain multiple structured traffic data.

[0221] In one possible design, the parsing unit includes:

[0222] The first acquisition component is used to acquire the network protocol version of multiple PCAP packet data;

[0223] The compensation component is used to dynamically compensate for the offsets of multiple fields based on the network protocol versions of multiple PCAP packet data, resulting in multiple compensated field offsets adapted to the network protocol versions; among them, dynamic compensation is used to correct the byte position deviation of feature fields under different protocol versions.

[0224] The parsing component is used to parse the offsets of multiple compensated fields to obtain multiple byte positions.

[0225] In one possible design, the anomaly detection model includes an empirical model, a judgment model, and a verification model. The first input module 1003 includes:

[0226] The first input unit is used to input multiple structured traffic data into the empirical model to obtain multiple first abnormal data and multiple first normal data; wherein, the empirical model is used to detect data in the multiple structured traffic data that conform to preset attack rules.

[0227] The judgment unit is used to input multiple first normal data into the judgment model to obtain multiple second abnormal data; wherein, the judgment model is used to mine abnormal data from the multiple first normal data.

[0228] The verification unit is used to input multiple second abnormal data into the verification model to obtain multiple third abnormal data; wherein, the verification model is used to verify whether the multiple second abnormal data are abnormal data or non-abnormal data.

[0229] The integration unit is used to integrate multiple first-abnormal data and multiple third-abnormal data to obtain multiple abnormal traffic data.

[0230] In one possible design, the abnormal flow detection device also includes:

[0231] The second input module is used to input multiple abnormal traffic data into a preset verification model to obtain multiple final abnormal data and the verification result of each abnormal traffic data. The verification result of each abnormal traffic data is used to indicate whether each abnormal traffic data is misjudged as abnormal traffic data or not misjudged as abnormal traffic data. Multiple final abnormal data refer to the data among multiple abnormal traffic data that is not misjudged as abnormal traffic data.

[0232] The first update module is used to update the anomaly detection model based on multiple final anomaly data to obtain the updated anomaly detection model.

[0233] In one possible design, the verification model includes a first verification model and a second verification model, and the second input module includes:

[0234] The first acquisition unit is used to acquire the anomaly type of each abnormal traffic data; wherein, the anomaly type of each abnormal traffic data includes a first type and a second type, the first type is an anomaly type related to protocol layer attacks on protocol ports, and the second type is an anomaly type related to network group behavior;

[0235] The second input unit is used to input each abnormal traffic data of the first type of abnormality into the first verification model and input each abnormal traffic data of the second type of abnormality into the second verification model to obtain multiple final abnormal data and the verification results of each abnormal traffic data; wherein, the first verification model is a feature verification model constructed based on a preset protocol port matching logic, and the second verification model is a group behavior verification model constructed based on a preset node group interaction graph structure.

[0236] In one possible design, the second input module also includes:

[0237] The second acquisition unit is used to acquire the first misjudgment correction rate and the second misjudgment correction rate within a time period; wherein, the first misjudgment correction rate refers to the ratio of the amount of misjudged data successfully removed by the first verification model to the amount of input abnormal traffic data, and the second misjudgment correction rate refers to the ratio of the amount of misjudged data successfully removed by the second verification model to the amount of input abnormal traffic data.

[0238] The first update unit is used to update the first verification model according to the first misjudgment correction rate in response to the first misjudgment correction rate being greater than or equal to the preset first correction rate threshold, so as to obtain the updated first verification model.

[0239] The second update unit is used to update the second verification model according to the second misjudgment correction rate in response to the second misjudgment correction rate being greater than or equal to the preset second correction rate threshold, so as to obtain the updated second verification model.

[0240] In one possible design, the first update module includes:

[0241] The third acquisition unit is used to acquire the PCAP packet characteristics and exception types corresponding to each final exception data;

[0242] The first building unit is used to construct the training dataset based on the PCAP packet features and anomaly types corresponding to each final anomaly data.

[0243] The third update unit is used to update the parameters of the anomaly detection model based on the training dataset, so as to obtain the updated anomaly detection model.

[0244] The abnormal flow detection device provided in this embodiment can perform... Figure 2 , Figure 3 and Figure 4The technical solution of the abnormal traffic detection method embodiment shown herein, its implementation principle and technical effect are similar to Figure 2 , Figure 3 and Figure 4 The embodiment of the abnormal traffic detection method shown is similar and will not be described in detail here.

[0245] Figure 11 This is a schematic diagram of the hardware structure of the electronic device provided in an embodiment of this application. Figure 11 As shown, the electronic device 110 includes at least one processor 1101 and a memory 1102. The electronic device 110 also includes a communication component 1103. The processor 1101, the memory 1102, and the communication component 1103 are connected via a bus 1104.

[0246] In the specific implementation process, at least one processor 1101 executes computer execution instructions stored in memory 1102, so that at least one processor 1101 is used to implement an abnormal traffic detection method of the above embodiment.

[0247] The specific implementation process of processor 1101 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0248] In the above embodiments, it should be understood that the processor 1101 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0249] The memory 1102 may include high-speed RAM memory, and may also include non-volatile memory NVM, such as at least one disk storage.

[0250] Bus 1104 can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Bus 1104 can be divided into address bus, data bus, control bus, etc. For ease of illustration, bus 1104 in the accompanying drawings of this application is not limited to only one bus or one type of bus.

[0251] The above description of the functions implemented by electronic devices and main control devices has introduced the solutions provided by the embodiments of the present invention. It is understood that, in order to implement the above functions, the electronic device or main control device includes hardware structures and / or software modules corresponding to the execution of each function. By combining the units and algorithm steps of the various examples described in the embodiments of the present invention, the embodiments of the present invention can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the technical solutions of the embodiments of the present invention.

[0252] This application also provides a computer-readable storage medium storing computer-executable instructions. When executed by a processor, these instructions are used to implement an abnormal traffic detection method as described in the above embodiments. In a specific implementation of the aforementioned abnormal traffic detection method, each module can be implemented as a processor.

[0253] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0254] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in application-specific integrated circuits (ASICs). Alternatively, the processor and the readable storage medium can exist as discrete components in an electronic device or a host device.

[0255] This application also provides a computer program product, including a computer program, which, when executed by a processor, is used to implement an abnormal traffic detection method according to the above embodiments.

[0256] The computer program is stored in a readable storage medium, and at least one processor can read the computer program from the readable storage medium and execute the computer program to perform the scheme provided in any of the above embodiments.

[0257] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disk, or optical disk.

[0258] The technical solutions of this application have been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it is readily understood by those skilled in the art that the scope of protection of this application is obviously not limited to these specific embodiments. The above embodiments are only used to illustrate the technical solutions of this application and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. These modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. An abnormal flow detection method, characterized in that, include: Acquire multiple PCAP packet data; wherein, the multiple PCAP packet data are used to represent the attributes of multiple data packets collected by a preset network interface within a preset time period; Based on multiple preset field offsets, feature extraction is performed on the multiple PCAP packet data to obtain multiple structured traffic data; wherein, the multiple field offsets are used to represent multiple byte positions in the multiple PCAP packet data for feature extraction, and each of the structured traffic data refers to the feature data extracted at each of the byte positions. The multiple structured traffic data are input into a preset anomaly detection model to obtain multiple abnormal traffic data; wherein, the multiple abnormal traffic data refers to data that does not conform to the preset normal traffic pattern identified from the multiple structured traffic data, and the multiple abnormal traffic data is used to provide data input for network security protection decisions.

2. The abnormal flow detection method according to claim 1, characterized in that, Based on preset offsets for multiple fields, feature extraction is performed on the multiple PCAP packet data to obtain multiple structured traffic data, including: Offset parsing is performed on the offsets of the multiple fields to obtain the positions of the multiple bytes; Based on the multiple byte positions, feature extraction is performed on the multiple PCAP packet data to obtain the multiple structured traffic data.

3. The abnormal flow detection method according to claim 2, characterized in that, The step of parsing the offsets of the multiple fields to obtain the positions of the multiple bytes includes: Obtain the network protocol version of the multiple PCAP packet data; Based on the network protocol version of the multiple PCAP packet data, the offsets of the multiple fields are dynamically compensated to obtain multiple compensated field offsets adapted to the network protocol version; wherein, the dynamic compensation is used to correct the byte position deviation of the feature fields under different protocol versions; Offset parsing is performed on the offsets of the multiple compensated fields to obtain the positions of the multiple bytes.

4. The abnormal flow detection method according to claim 1, characterized in that, The anomaly detection model includes an empirical model, a judgment model, and a verification model. The process of inputting the multiple structured traffic data into the preset anomaly detection model yields multiple anomaly traffic data, including: The multiple structured traffic data are input into the empirical model to obtain multiple first abnormal data and multiple first normal data; wherein, the empirical model is used to detect data in the multiple structured traffic data that conform to preset attack rules; The plurality of first normal data are input into the judgment model to obtain a plurality of second abnormal data; wherein, the judgment model is used to mine abnormal data from the plurality of first normal data; The plurality of second abnormal data are input into the verification model to obtain a plurality of third abnormal data; wherein, the verification model is used to verify whether the plurality of second abnormal data are abnormal data or non-abnormal data; The multiple first abnormal data and the multiple third abnormal data are integrated to obtain the multiple abnormal traffic data.

5. The abnormal flow detection method according to claim 1, characterized in that, After inputting the multiple structured traffic data into a preset anomaly detection model to obtain multiple abnormal traffic data, the process further includes: The multiple abnormal traffic data are input into a preset verification model to obtain multiple final abnormal data and the verification result of each of the abnormal traffic data; wherein, the verification result of each of the abnormal traffic data is used to indicate whether each of the abnormal traffic data is misjudged as abnormal traffic data or is not misjudged as abnormal traffic data, and the multiple final abnormal data refers to the data among the multiple abnormal traffic data that is not misjudged as abnormal traffic data. The anomaly detection model is updated based on the multiple final anomaly data to obtain the updated anomaly detection model.

6. The abnormal flow detection method according to claim 5, characterized in that, The verification model includes a first verification model and a second verification model. The step of inputting the multiple abnormal traffic data into the preset verification model to obtain multiple final abnormal data and the verification result of each abnormal traffic data includes: Obtain the anomaly type of each of the abnormal traffic data; wherein, the anomaly type of each of the abnormal traffic data includes a first type and a second type, the first type being an anomaly type related to protocol layer attacks on protocol ports, and the second type being an anomaly type related to network group behavior; Each of the abnormal traffic data of the first type is input into the first verification model, and each of the abnormal traffic data of the second type is input into the second verification model to obtain the multiple final abnormal data and the verification result of each of the abnormal traffic data; wherein, the first verification model is a feature verification model constructed based on a preset protocol port matching logic, and the second verification model is a group behavior verification model constructed based on a preset node group interaction graph structure.

7. The abnormal flow detection method according to claim 6, characterized in that, After inputting the abnormal traffic data of the first type into the first verification model and the abnormal traffic data of the second type into the second verification model to obtain the plurality of final abnormal data and the verification result of each abnormal traffic data, the method further includes: Obtain the first misjudgment correction rate and the second misjudgment correction rate within the time period; wherein, the first misjudgment correction rate refers to the ratio of the amount of misjudged data successfully removed by the first verification model to the amount of input abnormal traffic data, and the second misjudgment correction rate refers to the ratio of the amount of misjudged data successfully removed by the second verification model to the amount of input abnormal traffic data. In response to the first misjudgment correction rate being greater than or equal to a preset first correction rate threshold, the first verification model is updated according to the first misjudgment correction rate to obtain the updated first verification model. In response to the second misjudgment correction rate being greater than or equal to a preset second correction rate threshold, the second verification model is updated according to the second misjudgment correction rate to obtain the updated second verification model.

8. The abnormal flow detection method according to claim 5, characterized in that, The step of updating the anomaly detection model based on the multiple final anomaly data to obtain the updated anomaly detection model includes: Obtain the PCAP packet characteristics and anomaly type corresponding to each of the final anomaly data; A training dataset is constructed based on the PCAP packet features and anomaly types corresponding to each of the final anomaly data. The parameters of the anomaly detection model are updated based on the training dataset to obtain the updated anomaly detection model.

9. An abnormal flow detection device, characterized in that, include: The first acquisition module is used to acquire multiple PCAP packet data; wherein, the multiple PCAP packet data is used to represent the attributes of multiple data packets collected by a preset network interface within a preset time period; The first feature extraction module is used to extract features from the multiple PCAP packet data based on multiple preset field offsets to obtain multiple structured traffic data; wherein, the multiple field offsets are used to represent multiple byte positions in the multiple PCAP packet data for feature extraction, and each of the structured traffic data refers to the feature data extracted at each of the byte positions. The first input module is used to input the multiple structured traffic data into a preset anomaly detection model to obtain multiple abnormal traffic data; wherein, the multiple abnormal traffic data refers to data that does not conform to the preset normal traffic pattern identified from the multiple structured traffic data, and the multiple abnormal traffic data is used to provide data input for network security protection decisions.

10. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; When the processor executes the computer execution instructions stored in the memory, it is used to implement the abnormal traffic detection method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the abnormal traffic detection method as described in any one of claims 1 to 8.

12. A computer program product, characterized in that, The system includes a computer program, which, when executed by a processor, is used to implement the abnormal traffic detection method as described in any one of claims 1 to 8.

Images