Distributed abnormal traffic detection method and system for network security

Abstract

The invention discloses a distributed abnormal traffic detection method and system for network security, and relates to the field of network security, and the method comprises the steps: obtaining the field position feature information of a current traffic data packet and a historical traffic data packet; calculating the field position fluctuation amplitude of the current packet; calculating a matching degree between the current packet and historical data; selecting a quick matching or analysis recognition mode according to the threshold comparison of the fluctuation amplitude and the matching degree; executing a corresponding mode to judge abnormity; outputting a result and dynamically adjusting a threshold value; according to the method, the problems of low detection efficiency and high false alarm rate of a traditional method can be solved by dynamically selecting the rapid matching mode or the analysis recognition mode, and the method has the advantages of dynamically optimizing the detection mode, improving the detection efficiency and accuracy and reducing the false alarm rate and the missing report rate.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of network security, in particular to a network security-oriented distributed abnormal traffic detection method and system. BACKGROUND

[0002] With the rapid development of Internet technology, network attack means is increasingly complex and variable, and network security threats show the characteristics of scale and distribution. Distributed denial of service attacks, network intrusion, malicious traffic and other security incidents occur frequently, posing a serious threat to the stable operation of network infrastructure. Against this background, the accuracy and real-time performance of abnormal traffic detection technology, as the first line of defense for network security protection, are particularly important.

[0003] The current mainstream abnormal traffic detection technology mainly has the following technical bottlenecks: first, the traditional method mostly uses a static threshold judgment mechanism, which cannot adapt to the dynamic change characteristics of network traffic, resulting in high false positive and false negative rates. Second, the existing detection scheme usually uses a single detection mode, which is difficult to balance detection efficiency and accuracy. When faced with massive network traffic, the selection of a fixed mode often causes waste of computing resources or insufficient detection accuracy. Third, most technical solutions only focus on the surface features of network data packets, lacking the mining and analysis of deep-level information such as field position characteristics, making the detection model have limited recognition ability for new attacks.

[0004] More importantly, the existing technology lacks an effective adaptive adjustment mechanism. When the network traffic characteristics drift, the traditional method cannot dynamically optimize the detection parameters, resulting in gradual degradation of system performance. In addition, traffic detection in a distributed environment often faces problems such as uneven distribution of computing resources and increased processing delay, and an intelligent solution is needed to balance detection accuracy and system load.

[0005] In view of the above problems, the existing technology needs to be improved. SUMMARY

[0006] In view of the deficiencies of the existing technology, the present application provides a network security-oriented distributed abnormal traffic detection method and system.

[0007] In order to achieve the above purpose, the technical solution of the present application is as follows:

[0008] In a first aspect, the present application discloses a network security-oriented distributed abnormal traffic detection method, comprising the following steps:

[0009] Obtain the current traffic data packet and the historical traffic data packet set, the current traffic data packet including the field position characteristic information of at least one network field; the historical traffic data packet set including the field position characteristic information of a plurality of historical traffic data packets;

[0010] According to the field position characteristic information of the current traffic data packet, a field position fluctuation amplitude of the current traffic data packet is calculated;

[0011] According to the field position characteristic information of the current traffic data packet and the field position characteristic information of the historical traffic data packet set, a current historical matching degree is calculated;

[0012] According to whether the field position fluctuation amplitude is less than a first preset threshold value and whether the current historical matching degree is greater than a second preset threshold value, a fast matching mode or an analysis and identification mode is selected to be executed;

[0013] When the fast matching mode is selected to be executed, whether the current traffic data packet is an abnormal traffic packet is judged by comparing the field position characteristic information of the current traffic data packet with the field position characteristic information of the historical traffic data packet set;

[0014] When the analysis and identification mode is selected to be executed, whether the current traffic data packet is an abnormal traffic packet is judged by analyzing a fluctuation coefficient of the current traffic data packet;

[0015] A judgment result of whether the current traffic data packet is an abnormal traffic packet is outputted;

[0016] According to the judgment result, the first preset threshold value and the second preset threshold value are adjusted.

[0017] In a second aspect, the application discloses a distributed abnormal traffic detection system for network security, comprising:

[0018] A data acquisition module is configured to acquire a current traffic data packet and a historical traffic data packet set, wherein the current traffic data packet comprises field position characteristic information of at least one network field, and the historical traffic data packet set comprises field position characteristic information of a plurality of historical traffic data packets;

[0019] A fluctuation calculation module is configured to calculate a field position fluctuation amplitude of the current traffic data packet according to the field position characteristic information of the current traffic data packet;

[0020] A matching degree calculation module is configured to calculate a current historical matching degree according to the field position characteristic information of the current traffic data packet and the field position characteristic information of the historical traffic data packet set;

[0021] A mode selection module is configured to select a fast matching mode or an analysis and identification mode to be executed according to whether the field position fluctuation amplitude is less than a first preset threshold value and whether the current historical matching degree is greater than a second preset threshold value;

[0022] The quick matching judgment module is configured to, when the quick matching mode is selected to be executed, judge whether the current traffic data packet is an abnormal traffic packet by comparing the field position characteristic information of the current traffic data packet with the field position characteristic information of the historical traffic data packet set.

[0023] The analysis recognition judgment module is configured to, when the analysis recognition mode is selected to be executed, judge whether the current traffic data packet is an abnormal traffic packet by analyzing the fluctuation coefficient of the current traffic data packet.

[0024] The output adjustment module is configured to output the judgment result of whether the current traffic data packet is an abnormal traffic packet, and adjust the first preset threshold and the second preset threshold according to the judgment result.

[0025] The present application has the advantages that: by dynamically selecting the quick matching mode or the analysis recognition mode, combining the field position fluctuation amplitude and the historical matching degree for double judgment, and self-adaptively adjusting the detection threshold based on the feedback data, the problems of low detection efficiency and high false alarm rate of the traditional method are solved, and the detection mode is dynamically optimized, the detection efficiency and accuracy are improved, and the false alarm rate and the missing alarm rate are reduced. BRIEF DESCRIPTION OF DRAWINGS

[0026] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor on the basis of these drawings.

[0027] Figure 1 It is the overall block diagram of the method of the embodiment of the present application.

[0028] Figure 2 It is the method flow chart of the embodiment of the present application.

[0029] Figure 3 It is the overall block diagram of the system of the second embodiment of the present application. DETAILED DESCRIPTION

[0030] The technical solutions of the present application will be described clearly and completely in combination with the embodiments. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0031] SUMMARY OF THE APPLICATION: In the prior art, the field of network security has long been challenged by the lack of precision and efficiency in anomaly traffic detection. Traditional methods rely on fixed thresholds and a single detection mode, making it difficult to adapt to the dynamic changes in network traffic. For example, in a cloud computing environment, inter-virtual machine communication traffic exhibits burstiness and periodic fluctuations. Static thresholds can easily lead to normal traffic being misjudged as abnormal, while complex attack traffic may be missed due to pattern mismatch. When the position of network protocol fields is slightly shifted due to version updates, existing technologies cannot effectively identify such hidden anomalies, leading to the failure of security protection mechanisms.

[0032] To solve the above problems, it is found in the development process that the core contradiction lies in the lack of adaptability of the detection mechanism. First, fixed thresholds cannot reflect the true change pattern of traffic characteristics, and a dynamic adjustment mechanism needs to be established. Second, a single detection mode cannot balance efficiency and precision, and the optimal strategy needs to be selected according to traffic characteristics. Further analysis shows that the position characteristics of network fields contain important anomaly information, and their fluctuation amplitude and historical matching degree can be used as the basis for mode switching. By combining field position fluctuation with historical data similarity, a dual-mode detection framework is constructed to realize dynamic optimization of detection strategies.

[0033] Embodiment One:

[0034] As shown in Figures 1-2 A distributed anomaly traffic detection method for network security includes the following steps:

[0035] Obtain the current traffic data packet and the historical traffic data packet set, the current traffic data packet including at least one field position characteristic information of network fields; the historical traffic data packet set including field position characteristic information of multiple historical traffic data packets;

[0036] According to the field position characteristic information of the current traffic data packet, the field position fluctuation amplitude of the current traffic data packet is calculated;

[0037] According to the field position characteristic information of the current traffic data packet and the field position characteristic information of the historical traffic data packet set, the current historical matching degree is calculated;

[0038] According to whether the field position fluctuation amplitude is less than a first preset threshold and whether the current historical matching degree is greater than a second preset threshold, the fast matching mode or the analysis and recognition mode is selected to be executed;

[0039] When the fast matching mode is selected to be executed, by comparing the field position characteristic information of the current traffic data packet with the field position characteristic information of the historical traffic data packet set, it is determined whether the current traffic data packet is an abnormal traffic packet;

[0040] When the analysis identification mode is selected to be executed, whether the current traffic data packet is an abnormal traffic packet is determined by analyzing a fluctuation coefficient of the current traffic data packet;

[0041] An output is made of a determination result of whether the current traffic data packet is an abnormal traffic packet;

[0042] The first preset threshold and the second preset threshold are adjusted according to the determination result.

[0043] The application further proposes that the field position feature information comprises an offset of a network field relative to a start position of the traffic data packet and a length of the network field.

[0044] The offset refers to a byte distance between a start position of the network field and the start position of the traffic data packet, and can be specifically implemented by analyzing field positioning information of a data packet header. For example, in an IP data packet, an offset of a source IP address field is usually a fixed value. The extraction of the offset enables the change of the internal structure of the data packet to be quantified, thereby providing a basis for subsequent fluctuation amplitude calculation. The length refers to a number of bytes occupied by the network field, and can be specifically implemented by reading a preset length identifier of the field or dynamically analyzing field content. For example, a length field in a TCP protocol header can be obtained by analyzing a value at a specific position. The introduction of the length enables the size difference of the data packet field to be identified, thereby enhancing the detection capability of abnormal field expansion or truncation.

[0045] Specifically, in a traffic data packet analysis process, the offset and the length of each network field are synchronously extracted and recorded. For example, for a data packet of an HTTP protocol, an offset of a Host field can be located at a fixed position, and a length of a Content-Length field dynamically changes according to actual content. By taking the two parameters as field position feature information, the structural features of the data packet can be more comprehensively reflected. When detecting abnormal traffic, fluctuation of the offset can indicate that the field position is maliciously tampered with, and abnormal change of the length can reflect that the field content is injected or truncated, and the combination of the two can improve the identification accuracy of complex attack behaviors.

[0046] Existing methods usually only focus on the value features of fixed fields such as source IP addresses and destination ports, and ignore the position and size information of the fields in the data packet. For example, a traditional scheme can only count the frequency of occurrence of IP addresses, but cannot detect the case where the field position is maliciously offset. The present scheme can capture subtle abnormalities of the data packet structure by introducing the offset and the length as core features, such as the behavior of an attacker changing the original field position by inserting redundant fields, or the behavior of implementing a buffer overflow attack by expanding the field length.

[0047] By the technical solution, the application can solve the problem of missing detection caused by incomplete field position feature extraction in the prior art. For example, when an attacker modifies the field position but keeps the field content, the traditional method may not be able to identify the anomaly, but the present solution can detect such covert attacks through offset fluctuation. At the same time, dynamic analysis of the length can effectively identify field size anomalies, avoiding false positives due to field content meeting expectations but length exceeding a reasonable range.

[0048] The application further proposes a specific process for calculating the field position fluctuation amplitude of the current traffic data packet, including: for a specific network field in the current traffic data packet, calculating the difference between the maximum value and the minimum value of the offset of the specific network field in a plurality of traffic data packets collected in a preset time window to obtain the fluctuation amplitude of the specific network field; and the field position fluctuation amplitude of the current traffic data packet is the fluctuation amplitude of the specific network field.

[0049] The preset time window refers to a time range for collecting historical traffic data packets, which can be implemented by using a sliding window mechanism, for example, setting the time window to a set of traffic data packets in the last 10 minutes. By limiting the time range, the short-term characteristics of network traffic changes can be dynamically reflected, avoiding outdated data interference in fluctuation amplitude calculation. The fluctuation amplitude refers to the change range of the offset of a specific network field, which can be calculated by calculating the difference between the maximum value and the minimum value of the same field in the time window. The difference reflects the fluctuation intensity of the field position in a short time, providing a quantitative basis for subsequent mode selection.

[0050] Specifically, traffic data packets are continuously collected in the preset time window, and the offset data of a specific network field in the traffic data packets is extracted. For example, for the source IP address field, the starting position offset of the field in each data packet is recorded. Then all data packets in the time window are traversed to find the maximum value and the minimum value of the offset of the field, and the difference between the two values is calculated as the fluctuation amplitude. If the difference is zero, it means that the field position is stable; the larger the difference, the more intense the field position changes. The fluctuation amplitude is directly used as the field position fluctuation amplitude of the current traffic data packet to determine whether to trigger the fast matching mode.

[0051] The traditional method usually only compares the field position of a single data packet statically, or calculates the fluctuation by using the average value of a fixed time period. However, the present solution can more sensitively capture the sudden offset of the field position by calculating the range in the dynamic time window, for example, the field position may change rapidly when under attack, at which time the range will significantly increase, thereby triggering the analysis and identification mode in time.

[0052] By the technical solution, the short-term fluctuation characteristics of the network field position can be accurately quantified, and reliable basis is provided for selection of a subsequent detection mode. By combining the dynamic time window and the range calculation, the long-term historical data is avoided from covering the sudden anomaly, and the calculation sensitivity of the fluctuation amplitude is enhanced, so that the response speed and accuracy of the abnormal traffic detection are improved.

[0053] The application further provides a specific process for calculating the current historical matching degree, including: calculating the similarity between the field position characteristic information of the current traffic data packet and the field position characteristic information of each historical traffic data packet in the historical traffic data packet set; the current historical matching degree is the average of the calculated similarity; the process of calculating the similarity includes: extracting the offset of all network fields in the current traffic data packet, arranging the offset in a fixed field order to form a current offset vector; extracting the offset of network fields in the historical traffic data packet in the same order to form a historical offset vector; and calculating the cosine value of the included angle between the two vectors to obtain the similarity.

[0054] The current historical matching degree refers to the overall matching degree of the current traffic data packet and the historical traffic data packet set, which can be specifically implemented by using the average of the similarity. By calculating the similarity of multiple historical data packets and taking the average, the comprehensive matching relationship between the current traffic and the historical data can be reflected. The similarity refers to the closeness of two traffic data packets in the field position characteristics, which can be specifically implemented by using the cosine value of the included angle between vectors. By converting the offset into a vector and calculating the cosine similarity, the structural consistency of the field position distribution can be effectively quantified. The current offset vector refers to a numerical sequence formed by arranging the offset of each network field in the current traffic data packet in a preset order, which can be specifically implemented by using a splicing method of a fixed field order, for example, generating a vector in the order of source IP, destination IP, source port and destination port to ensure the consistency of the vector structure of different data packets. The historical offset vector refers to a numerical sequence formed by arranging the offset of the same field order in the historical traffic data packet, which can be specifically implemented by using the same generation rule as the current vector to facilitate mathematical operations between vectors. The cosine value refers to the cosine value of the included angle between two vectors in space, which can be specifically calculated by using the mathematical formula of vector dot product divided by the product of vector lengths, and the value range is-1 to 1. The closer the value is to 1, the more consistent the direction of the vector is.

[0055] Specifically, in the calculation of the current historical matching degree, first, all the network field offset amounts of the current traffic data packet are arranged in a preset order to generate a current offset vector. For each data packet in the historical traffic data packet set, the offset amounts are extracted in the same field order to generate a corresponding historical offset vector. Then, the cosine similarity between the current vector and each historical vector is calculated to obtain a set of similarity values. The average value of the set of similarity values is taken as the overall matching degree of the current traffic data packet and the historical data. For example, if the current traffic data packet contains three fields of source IP, destination IP and source port, and the offset amounts thereof are 50, 120 and 200 respectively, the current vector is [50, 120, 200]; if the corresponding offset amounts of a certain historical data packet are 48, 118 and 202 respectively, the historical vector is [48, 118, 202], and the cosine similarity calculation result of the two vectors is 0.999, indicating a high degree of matching. By statistically analyzing the similarity of all historical data packets and taking the average, it can be determined whether the current traffic conforms to the characteristics of the historical data.

[0056] The traditional method usually only compares the numerical values of a single field or simply judges the matching degree by a threshold value, for example, directly compares whether the source IP offset amounts are the same. However, the present application can consider the position distribution characteristics of multiple fields and capture the overall structural relationship between the fields through vector expression and cosine similarity calculation. For example, when the offset amount of a certain field fluctuates reasonably due to protocol expansion, the traditional method may misjudge it as abnormal due to the difference of a single field, while the present application can identify the consistency of the overall structure through cosine similarity calculation, thereby reducing the misjudgment.

[0057] Through the above technical solutions, the present application can improve the comprehensive analysis capability of the field position characteristics in abnormal traffic detection, accurately quantify the matching degree of the current traffic and the historical data through vector expression and cosine similarity calculation, and avoid misjudgment caused by the fluctuation of a single field. At the same time, the matching degree calculation method based on the average similarity can effectively suppress the interference of individual abnormal historical data on the overall judgment and enhance the stability of the detection result.

[0058] The present application further proposes that if the field position fluctuation amplitude is less than a first preset threshold value and the current historical matching degree is greater than a second preset threshold value, the fast matching mode is selected to be executed; otherwise, the analysis and recognition mode is selected to be executed.

[0059] The field position fluctuation amplitude refers to the difference between the maximum value and the minimum value of the offset of a specific network field in the traffic data packet within a preset time window, and can be specifically calculated by calculating the range of the offset of the continuously collected traffic data packet, and is used to measure the stability of the field position of the current traffic data packet. The current historical matching degree refers to the average similarity of the current traffic data packet and the historical traffic data set, and can be specifically calculated by vector cosine similarity, and is used to reflect the degree of conformity of the current traffic data packet to the historical normal traffic mode. The first preset threshold and the second preset threshold refer to the dynamically adjusted judgment criteria, which can be specifically obtained by historical data statistics or machine learning model training, and are used to optimize the mode switching condition according to the network environment change. The fast matching mode refers to an abnormal traffic detection mode based on hash value comparison, which can be specifically implemented by a feature string conversion and hash matching mechanism, and is suitable for processing traffic data packets conforming to historical features. The analysis and identification mode refers to an abnormal traffic detection mode based on fluctuation coefficient calculation, which can be specifically implemented by statistical standard deviation and offset deviation analysis, and is suitable for processing traffic data packets with large fluctuations or deviating from historical modes.

[0060] Specifically, when the field position fluctuation amplitude of the traffic data packet is small and the matching degree with the historical data is high, it indicates that the current traffic feature is stable and conforms to the known normal mode, and at this time, the fast matching mode is adopted for hash value comparison, which can quickly complete the abnormal judgment. For example, the first preset threshold can be the statistical average value of the offset fluctuation amplitude within the preset time window, and the second preset threshold can be the 80% quantile value of the historical similarity data. When the above two conditions are met at the same time, the system automatically enables the fast matching mode, and completes the detection within milliseconds through the feature string hash matching mechanism. If either condition is not met, the analysis and identification mode is switched to, and more detailed abnormal judgment is performed by calculating the fluctuation coefficient of each field offset, combining protocol type screening and standard deviation analysis. This mode switching mechanism can automatically select the optimal detection path according to the dynamic characteristics of the traffic.

[0061] The traditional method usually adopts a fixed threshold or a single detection mode, and cannot dynamically adjust according to the traffic fluctuation characteristics and the historical matching degree. For example, the prior art can only determine whether to perform pattern matching by a static similarity threshold, resulting in false judgment when the traffic bursts. The present scheme determines by double threshold conditions, and combines the cooperative analysis of fluctuation amplitude and matching degree, so that the mode selection is more suitable for the actual network state, avoiding both the missed detection of high matching degree but burst abnormal traffic and the false detection of low fluctuation but deviating from the historical mode traffic.

[0062] By the technical solution, the application can dynamically switch the detection mode according to the real-time characteristics of the flow data, significantly reduce the processing delay while ensuring the detection accuracy. For stable flow conforming to historical characteristics, the fast matching mode can shorten the detection time by more than 90%; for abnormal or complex flow, the analysis and identification mode improves the detection accuracy through multi-dimensional fluctuation analysis, thereby achieving overall optimization of efficiency and accuracy.

[0063] The application further proposes that the fast matching mode specifically includes: converting field position characteristic information of the current flow data packet into a characteristic string, the characteristic string being composed of offset values and length values of each network field spliced in a preset order; performing hash operation on the characteristic string to generate a current hash value; matching the current hash value with a pre-stored hash value set in the historical flow data packet set; if there is the same hash value, determining that the current flow data packet is a normal flow packet; if there is no same hash value, determining that the current flow data packet is an abnormal flow packet.

[0064] The characteristic string refers to a string formed by encoding the position characteristics of multiple network fields in the flow data packet according to a fixed rule, which can be specifically implemented by splicing the offset values and length values of each field in the field order. By converting multi-dimensional characteristics into a single string, the subsequent comparison process can be simplified, and the processing efficiency can be improved. The hash operation refers to performing hash calculation on the characteristic string to generate a unique identifier, which can be specifically implemented by using MD5, SHA-1 or CRC32 algorithm. The generation of the hash value can convert the complex string into a fixed-length value, reducing the resource consumption of storage and matching. The hash value matching refers to comparing whether there is a duplicate item between the current hash value and the historical hash value set, which can be specifically implemented by constructing a hash table or a Bloom filter. This way avoids comparing each data one by one, greatly shortening the matching time.

[0065] Specifically, in the fast matching mode, first, the offset and length of the key network fields in the flow data packet are spliced in a preset order, for example, a characteristic string is generated in the field order of source IP, destination IP, source port, destination port, etc. Then, the hash operation is performed on the string, for example, a 32-bit hash value is generated by using the CRC32 algorithm. Then, the current hash value is quickly searched with the pre-stored hash value in the historical flow data packet set, if there is the same hash value, it means that the flow packet is consistent with the historical normal flow characteristics, and it is determined as normal; if no same hash value is matched, it is determined as abnormal. This process can complete the flow characteristic comparison in milliseconds through the uniqueness and fast query characteristics of the hash value, which is especially suitable for real-time detection requirements in high-concurrency network environment.

[0066] The traditional method usually needs to compare field by field or calculate a similarity matrix, and has problems of large amount of calculation and high response delay. The scheme compresses multi-dimensional features into a single numerical value for fast retrieval through a hash value matching mechanism, not only reduces the data storage space, but also reduces the comparison complexity from linear level to constant level. For example, the calculation based on cosine similarity in the prior art needs vector operation for each historical data packet, while the hash table query only needs one hash calculation and one table lookup operation, which significantly improves the processing efficiency.

[0067] Through the above technical solutions, the application can realize millisecond-level anomaly detection in a high-flow load scenario, effectively reducing system resource occupation. At the same time, the uniqueness of the hash value avoids misjudgment caused by differences in field order or numerical format, improving detection accuracy. The scheme is particularly suitable for distributed network security systems that require fast response, and can improve the overall detection throughput without increasing hardware costs.

[0068] The application further proposes a specific implementation of analysis and identification mode, including screening a subset with the same protocol type as the current traffic data packet from the historical traffic data packet set; calculating the historical mean and historical standard deviation of each network field offset based on the subset; calculating the fluctuation coefficient of each field offset of the current traffic data packet: fluctuation coefficient = |current offset - historical mean| / historical standard deviation; determining the condition of abnormality as any of the following situations: there is at least one network field with a fluctuation coefficient greater than a first abnormal threshold; more than half of the network fields have a fluctuation coefficient greater than a second abnormal threshold; wherein the first abnormal threshold is greater than the second abnormal threshold.

[0069] The subset with the same protocol type refers to a set of data packets with the same protocol type as the current traffic data packet filtered from the historical traffic data packets, and can be specifically realized by analyzing the protocol type field and performing a matching operation. This filtering process can ensure that the subsequent statistical calculation is based on the data distribution of the same protocol, avoiding misjudgment caused by differences in different protocol field positions. The historical mean and the historical standard deviation refer to the offset of each network field, and the average value and the standard deviation of the offset are calculated in the subset with the same protocol type. The sliding window statistics or batch calculation method can be used to realize it. The historical mean and the standard deviation can dynamically reflect the normal fluctuation range of the field position, and provide a benchmark for anomaly detection. The fluctuation coefficient refers to the absolute difference between the current offset and the historical mean divided by the historical standard deviation, which is calculated by a mathematical formula. This coefficient is used to quantify the deviation of the current offset from the historical distribution, and the larger the value, the higher the possibility of abnormality. The first abnormal threshold and the second abnormal threshold are pre-set judgment condition parameters, wherein the first abnormal threshold is used to detect the significant abnormality of a single field, and the second abnormal threshold is used to detect the coordinated abnormality of multiple fields. The specific values can be adjusted according to the actual scene. Through the double judgment conditions, different types of anomalies can be covered, and the comprehensiveness of detection can be improved.

[0070] Specifically, when entering the analysis and identification mode, first, according to the protocol type of the current traffic data packet, such as TCP, UDP or ICMP, a subset with the same protocol type is extracted from the set of historical traffic data packets. For example, if the current data packet is a TCP protocol, only the data packets of all TCP protocols in the historical data are selected as the analysis benchmark. Then, for the offset of each network field, such as the source port field or the destination IP field, the historical mean and the historical standard deviation of the offset are calculated in the subset. For example, the historical mean of the source port field may be 1024, and the historical standard deviation is 50. Then, the fluctuation coefficient of each field of the current traffic data packet is calculated. For example, when the current source port offset is 2000, the fluctuation coefficient is (2000-1024) / 50=19.52. Finally, according to the pre-set judgment condition, the anomaly is judged: if the fluctuation coefficient of any field exceeds the first abnormal threshold, for example, set to 5, it is directly judged as abnormal; if the fluctuation coefficient of more than half of the fields exceeds the second abnormal threshold, for example, set to 3, it is also judged as abnormal. In this way, both the dramatic offset of a single field and the collective anomaly of multiple fields can be identified, enhancing the ability to identify complex attack patterns.

[0071] The prior art generally uses a fixed threshold to uniformly analyze traffic data of all protocol types, resulting in differences in normal deviation ranges of different protocols being ignored. For example, the port field of the HTTP protocol usually has small fluctuations, while the port of the P2P protocol has large fluctuations, and a unified threshold may cause misjudgment. In addition, the prior art relies on a single abnormality determination condition, such as detecting only the abnormality of a single field, which cannot effectively identify the coordinated abnormality of multiple fields in a distributed attack. The present scheme can dynamically adjust the detection benchmark for different protocols and cover a wider range of abnormal scenarios through protocol type screening and double determination conditions, significantly improving the detection accuracy and adaptability.

[0072] Through the above technical scheme, the present application can effectively solve the misjudgment problem caused by protocol differences in the prior art and improve the detection capability of multi-field coordinated abnormality. For example, when detecting DDoS attacks, the attack traffic may exhibit small-scale abnormalities in multiple fields at the same time, which can be accurately identified by the second abnormality threshold. When detecting port scanning attacks, the dramatic deviation of a single port can be quickly determined by the first abnormality threshold. The scheme realizes accurate identification of different types of abnormal traffic through dynamic statistics and double condition determination, while reducing false positives and false negatives caused by differences in protocol characteristics.

[0073] The present application further proposes that the network field includes at least one of the following: a source IP address field, a destination IP address field, a source port field, a destination port field, and a protocol type field.

[0074] The source IP address field is a unique address used to identify the network device of the data packet sender, which can be implemented in IPv4 or IPv6 format, and its function is to identify the sender address characteristics to determine whether there is abnormal traffic source. The destination IP address field is the network address of the data packet receiver, which has a similar implementation as the source IP address field, and is used to analyze whether the traffic target deviates from the normal access range. The source port field and the destination port field represent the communication port numbers of the sender and the receiver, respectively, which can be implemented as 16-bit unsigned integers, and the port number change can identify unauthorized services or abnormal connection behavior. The protocol type field is used to identify the network protocol type of the data packet, such as TCP, UDP, or ICMP, and its function is to select the corresponding detection rule according to the protocol characteristics.

[0075] Specifically, the selection of the above network fields is based on their criticality in traffic anomaly detection. The combination of source IP address and destination IP address can reflect the legitimacy of the communication path, and if there is a high frequency or irregular address combination, it may indicate a distributed attack or scanning behavior. The variation range of source port and destination port can assist in identifying port scanning or service abuse, such as sending packets from the same source port to multiple destination ports in a short period of time. The matching degree analysis of the protocol type field can quickly filter out packets using non-standard protocols, such as malicious traffic disguised as legitimate protocols. By integrating the positional feature information of these fields, the system can cover multiple dimensions of traffic packet attributes, avoiding detection blind spots caused by insufficient extraction of single field features.

[0076] Traditional methods usually only focus on the static matching of IP addresses or port numbers, and do not include protocol types and field position fluctuations in the analysis range. For example, existing solutions may only filter based on a blacklist of source IP addresses, but cannot identify hidden attacks initiated through abnormal ports or protocols under legitimate IP addresses. However, the present solution can more comprehensively capture traffic anomaly patterns by simultaneously extracting the field position features of IP addresses, port numbers, and protocol types, such as identifying malicious protocol packets that communicate using non-standard ports, or detecting abnormal behavior where the combination of source IP and destination port deviates from the historical normal range.

[0077] Through the above technical solutions, the present application can improve the coverage and accuracy of abnormal traffic detection. By integrating the positional features of key network fields, the system can identify complex abnormal traffic that is difficult to detect by traditional methods, such as data penetration using legitimate protocols but abnormal ports. At the same time, the introduction of the protocol type field allows the detection process to dynamically adapt to the packet structure of different protocols, avoiding false positives caused by protocol differences. In addition, multi-field joint analysis can reduce false positives caused by single feature fluctuations, such as when the source port randomly changes but other fields conform to the normal pattern, the system will not determine the anomaly based solely on port fluctuations.

[0078] The application further proposes a process of adjusting the first preset threshold and the second preset threshold according to the judgment result, which includes: recording the current historical matching degree, the field position fluctuation amplitude, the processing time of executing the selected mode, and the false alarm indication of the judgment result; establishing a feedback data set containing the current historical matching degree, the field position fluctuation amplitude, the processing time, and the false alarm indication of the continuous N traffic data packets; dynamically updating the threshold according to the feedback data set: if the average value of the current historical matching degree in the feedback data set is lower than the historical matching reference value, the second preset threshold is reduced; if the average value of the field position fluctuation amplitude in the feedback data set is higher than the historical fluctuation reference value, the first preset threshold is increased; if the average value of the processing time in the feedback data set exceeds the time reference value, the second preset threshold is increased; if the proportion of the false alarm indication in the feedback data set exceeds the false alarm reference value, the first preset threshold is increased and the second preset threshold is reduced simultaneously; the historical matching reference value is the moving average value of the current historical matching degree recorded in the previous M detections; the historical fluctuation reference value is the moving average value of the field position fluctuation amplitude recorded in the previous M detections; the time reference value is the moving average value of the processing time recorded in the previous M detections; and the false alarm reference value is a preset fixed proportion value.

[0079] Among them, the feedback data set refers to a collection of multi-dimensional index data generated during the detection process of continuous traffic data packets, which can be implemented by a database table or a time series data structure, and its function is to provide a dynamic update data basis for threshold adjustment. Dynamic updating of the threshold refers to real-time correction of the preset threshold according to the statistical indicators in the feedback data set, which can be implemented by a sliding window algorithm or a weighted average algorithm, and its function is to make the threshold adapt to the dynamic changes of network traffic. The moving average value refers to the average value calculated based on historical data, which can be implemented by an exponential weighted moving average or a simple moving average method, and its function is to provide a comparable reference value for the current data. The false alarm reference value refers to the upper limit of the false alarm rate, which can be determined by empirical values or experimental data, and its function is to provide a basis for judging the false alarm proportion.

[0080] Specifically, during the flow detection process, the current historical matching degree, field position fluctuation amplitude, processing time and false positive results are recorded after each detection is completed, and these data are stored in the feedback dataset. When the data quantity reaches the preset N flow packets, the system calculates the average values of each index and compares them with the historical benchmark values. For example, if the average value of the current historical matching degree is lower than the historical matching benchmark value, it indicates that the similarity between the flow characteristics and the historical pattern has decreased, and at this time, reducing the second preset threshold value can expand the triggering range of the fast matching pattern, thereby improving the detection efficiency. If the average value of the field position fluctuation amplitude is higher than the historical fluctuation benchmark value, it indicates that the position offset change of the network field is intensified, and at this time, increasing the first preset threshold value can reduce the misjudgment caused by accidental fluctuations. When the processing time exceeds the time benchmark value, the fast matching pattern can be preferentially used by increasing the second preset threshold value to shorten the processing time consumption. If the false positive rate exceeds the false positive benchmark value, the two threshold values are adjusted synchronously to balance the detection accuracy and efficiency.

[0081] In some embodiments, the data storage period of the feedback dataset can be set in the form of a sliding window, for example, triggering threshold value update calculation once every 100 flow data packets processed. The calculation of the historical matching benchmark value can use the exponential weighted moving average method, giving higher weight to recent data to reflect short-term changes in flow characteristics. The false positive benchmark value can be set as a fixed proportion, for example, triggering threshold value adjustment operation when the false positive rate exceeds 5%.

[0082] The existing method usually uses a fixed threshold value or a single dimension adjustment strategy, which cannot dynamically optimize according to the multi-dimensional changes of flow characteristics. For example, the traditional technology only relies on the false positive rate to adjust the threshold value, but does not consider the relevance of processing time and historical matching degree. The present scheme can more comprehensively reflect the actual running state of flow detection by constructing a feedback mechanism by comprehensively considering multiple indexes, thereby improving the accuracy and adaptability of threshold value adjustment.

[0083] Through the above technical solutions, the present application can dynamically optimize the threshold value parameters according to the real-time detection data, effectively solving the false positive and missed alarm problems caused by the static threshold value. By combining multi-dimensional indexes such as historical matching degree, fluctuation amplitude and processing time, the system can adaptively adjust the triggering conditions of the detection mode, optimize the resource utilization rate while ensuring the detection accuracy. In addition, the introduction of the false positive benchmark value further enhances the reliability of the abnormality judgment, and reduces the risk of false judgment caused by the deviation of the threshold value parameters.

[0084] Embodiment two:

[0085] As shown in Figure 3 , the distributed abnormal flow detection system for network security comprises:

[0086] The data collection module is configured to acquire a current traffic data packet and a historical traffic data packet set, the current traffic data packet comprising field position characteristic information of at least one network field, and the historical traffic data packet set comprising field position characteristic information of a plurality of historical traffic data packets.

[0087] The fluctuation calculation module is configured to calculate a field position fluctuation amplitude of the current traffic data packet according to the field position characteristic information of the current traffic data packet.

[0088] The matching degree calculation module is configured to calculate a current historical matching degree according to the field position characteristic information of the current traffic data packet and the field position characteristic information of the historical traffic data packet set.

[0089] The mode selection module is configured to select a fast matching mode or an analysis and recognition mode according to whether the field position fluctuation amplitude is less than a first preset threshold value and whether the current historical matching degree is greater than a second preset threshold value.

[0090] The fast matching judgment module is configured to, when the fast matching mode is selected, judge whether the current traffic data packet is an abnormal traffic packet by comparing the field position characteristic information of the current traffic data packet with the field position characteristic information of the historical traffic data packet set.

[0091] The analysis and recognition judgment module is configured to, when the analysis and recognition mode is selected, judge whether the current traffic data packet is an abnormal traffic packet by analyzing a fluctuation coefficient of the current traffic data packet.

[0092] The output adjustment module is configured to output a judgment result of whether the current traffic data packet is an abnormal traffic packet, and adjust the first preset threshold value and the second preset threshold value according to the judgment result.

[0093] The above content is merely an example and description of the structure of the present application, and those skilled in the art can make various modifications or supplements or use similar ways to replace the described specific embodiments, as long as the modifications or supplements or replacements do not deviate from the structure of the present application or exceed the scope defined by the present application.

[0094] In the description of the present application, the description of the terms "one embodiment", "example", "specific example" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present application, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0095] The preferred embodiments of the application disclosed above are only to facilitate the elucidation of the application. The preferred embodiments do not describe all the details of the application and limit the application to the specific embodiments. Obviously, many modifications and variations can be made in light of the teachings above. The description is chosen and described in order to provide the best illustration of the application and its practical application to those skilled in the art and to enable those skilled in the art to best utilize the application. The application is limited only by the claims and their full scope and equivalents.

Claims

1. A distributed abnormal traffic detection method for network security, characterized in that, Includes the following steps: Obtain current traffic data packets and a set of historical traffic data packets. The current traffic data packet includes field location feature information of at least one network field. The set of historical traffic data packets includes field location feature information of multiple historical traffic data packets. The fluctuation range of the field position of the current traffic data packet is calculated based on the field position feature information of the current traffic data packet. The current historical matching degree is calculated based on the field location feature information of the current traffic data packet and the field location feature information of the historical traffic data packet set; Based on whether the fluctuation range of the field position is less than the first preset threshold and whether the current historical matching degree is greater than the second preset threshold, select to execute the fast matching mode or the analysis and identification mode. When the fast matching mode is selected, the field position feature information of the current traffic data packet is compared with the field position feature information of the historical traffic data packet set to determine whether the current traffic data packet is an abnormal traffic packet; When the analysis and identification mode is selected, the fluctuation coefficient of the current traffic data packet is analyzed to determine whether the current traffic data packet is an abnormal traffic packet; Output the result of whether the current traffic data packet is an abnormal traffic packet; Adjust the first preset threshold and the second preset threshold according to the judgment result.

2. The distributed abnormal traffic detection method for network security according to claim 1, characterized in that: The field location feature information includes: The offset of the network field relative to the starting position of the traffic data packet, and the length of the network field.

3. The distributed abnormal traffic detection method for network security according to claim 2, characterized in that: The specific process for calculating the fluctuation range of the field position of the current traffic data packet includes: For a specific network field in the current traffic data packet, calculate the difference between the maximum and minimum values ​​of its offset among multiple traffic data packets collected within a preset time window to obtain the fluctuation range of the specific network field. The fluctuation range of the field position in the current traffic data packet is the fluctuation range of the specific network field.

4. The distributed abnormal traffic detection method for network security according to claim 3, characterized in that: The specific process for calculating the current historical matching degree includes: Calculate the similarity between the field location feature information of the current traffic data packet and the field location feature information of each historical traffic data packet in the historical traffic data packet set; The current historical matching degree is the average value of the calculated similarity; The similarity calculation process includes: extracting the offsets of all network fields in the current traffic data packet and arranging them in a fixed field order to form a current offset vector; extracting the offsets of network fields in the same order in historical traffic data packets to form a historical offset vector; and calculating the cosine of the angle between the two vectors to obtain the similarity.

5. The distributed abnormal traffic detection method for network security according to claim 4, characterized in that: The process of selecting to execute the fast matching mode or the analysis and identification mode includes: If the fluctuation range of the field position is less than the first preset threshold and the current historical matching degree is greater than the second preset threshold, then the fast matching mode is selected; otherwise, the analysis and identification mode is selected.

6. The distributed abnormal traffic detection method for network security according to claim 5, characterized in that: The fast matching mode specifically includes: The field location feature information of the current traffic data packet is converted into a feature string, which is composed of the offset value and length value of each network field in a preset order; Perform a hash operation on the feature string to generate the current hash value; The current hash value is matched with the pre-stored set of hash values ​​in the historical traffic data packet set. If the same hash value exists, the current traffic data packet is determined to be a normal traffic packet; If no identical hash value is found, the current traffic data packet is determined to be an abnormal traffic packet.

7. The distributed abnormal traffic detection method for network security according to claim 6, characterized in that: The analysis and identification patterns specifically include: From the historical traffic data packet set, a subset of data packets with the same protocol type as the current traffic data packets is selected; based on the subset, the historical mean and historical standard deviation of the offset of each network field are calculated; the fluctuation coefficient of the offset of each field in the current traffic data packet is calculated. Volatility coefficient = |Current offset - Historical mean| / Historical standard deviation; An anomaly is determined if any of the following conditions are met: There exists at least one network field whose fluctuation coefficient is greater than the first anomaly threshold; More than half of the network field fluctuation coefficients are greater than the second anomaly threshold; The first abnormal threshold is greater than the second abnormal threshold.

8. The distributed abnormal traffic detection method for network security according to claim 7, characterized in that: The network fields include at least one of the following: source IP address field, destination IP address field, source port field, destination port field, and protocol type field.

9. The distributed abnormal traffic detection method for network security according to claim 8, characterized in that: The process of adjusting the first preset threshold and the second preset threshold according to the judgment result includes: recording the current historical matching degree, the fluctuation range of the field position, the processing time of executing the selected mode, and the false alarm indication of the judgment result; Establish a feedback dataset containing the current historical matching degree, field position fluctuation range, processing time, and false alarm indication of N consecutive traffic data packets; The threshold is dynamically updated based on the feedback dataset: If the average value of the current historical matching degree in the feedback dataset is lower than the historical matching benchmark value, then the second preset threshold is reduced. If the average value of the fluctuation range of the field position in the feedback dataset is higher than the historical fluctuation benchmark value, then the first preset threshold is increased; If the average processing time of the feedback dataset exceeds the time base value, then the second preset threshold is increased; If the proportion of false alarms in the feedback dataset exceeds the false alarm baseline, then simultaneously execute: increase the first preset threshold and decrease the second preset threshold; The historical matching baseline value is the moving average of the current historical matching degree recorded in the previous M detections; The historical fluctuation benchmark value is the moving average of the field position fluctuation amplitude recorded in the previous M detections; The time reference value is the moving average of the processing time recorded in the previous M detections; The false alarm baseline value is a preset fixed ratio value.

10. A distributed abnormal traffic detection system for network security, characterized in that: Using a distributed abnormal traffic detection method for network security as described in any one of claims 1 to 9, including: The data acquisition module is used to acquire current traffic data packets and a set of historical traffic data packets. The current traffic data packet includes field location feature information of at least one network field, and the set of historical traffic data packets includes field location feature information of multiple historical traffic data packets. The fluctuation calculation module is used to calculate the fluctuation amplitude of the field position of the current traffic data packet based on the field position feature information of the current traffic data packet; The matching degree calculation module is used to calculate the current historical matching degree based on the field position feature information of the current traffic data packet and the field position feature information of the historical traffic data packet set. The mode selection module is used to select between executing a fast matching mode or an analysis and identification mode based on whether the fluctuation range of the field position is less than a first preset threshold and whether the current historical matching degree is greater than a second preset threshold. The fast matching judgment module is used to determine whether the current traffic data packet is an abnormal traffic packet by comparing the field position feature information of the current traffic data packet with the field position feature information of the historical traffic data packet set when the fast matching mode is selected to be executed. The analysis and identification module is used to determine whether the current traffic data packet is an abnormal traffic packet by analyzing the fluctuation coefficient of the current traffic data packet when the analysis and identification mode is selected to be executed. The output adjustment module is used to output the judgment result of whether the current traffic data packet is an abnormal traffic packet; and adjust the first preset threshold and the second preset threshold according to the judgment result.

Images