Malicious traffic detection method based on feature rule

Abstract

The invention relates to the technical field of traffic detection, and discloses a malicious traffic detection method based on feature rules, which comprises the following steps: extracting malicious traffic detection feature rules one by one for format verification, and extracting related rule option information; constructing a rule dividing and conquering chain according to the related rule option information, and constructing rule groups for rules of the same dividing and conquering chain path; grouping according to the rules, selecting part of rule options of each rule, and constructing a multimode intensive graph of a corresponding group; constructing a precise detection chain of each rule according to the multimode intensive graph; the detection performance of the front malicious traffic detection engine based on the feature rule is improved, the problem that the network security product detection effect is poor due to the fact that the front malicious traffic detection engine based on the feature rule is insufficient in performance is solved, and the effects of finding suspicious traffic in time and removing normal traffic more quickly are achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of flow detection, and in particular to a malicious flow detection method based on characteristic rules. Background Art

[0002] With the continuous development and application of network technology, while promoting social development and progress, various network security attacks have also brought a lot of economic losses. In order to quickly detect attack events in the network, timely warn and block attacks, and avoid economic losses caused by network attacks, various network security detection products currently usually use pre-feature rule malicious traffic detection engines to identify a small amount of suspicious attack traffic, and then cooperate with post-detection engines such as machine learning engines to accurately detect attack behaviors in network traffic.

[0003] Existing feature rule malicious traffic detection engines are usually implemented in the following two ways:

[0004] The first method is based on the fingerprint features of malicious traffic, generates corresponding regular or feature string rules, and uses hyperscan to match all rules with traffic loads for detection.

[0005] The second method uses pre-filtering methods such as fixed addition of ports + single sub-features to filter out some detection rules to reduce the number of actual rule detections and improve the performance of detection engines, such as open source engines such as Suricata.

[0006] These two methods usually have the following disadvantages:

[0007] The pre-filtering mode is too simple. Only one option feature can be selected from the option special rule as the filtering condition, which makes other option features with high detection performance unable to participate in pre-filtering, greatly reducing the effect of pre-filtering. And the filtering mode is fixed. Once the rules are compiled and detected, the rule filtering module will no longer be adjusted in real time regardless of its performance, and the advantages of the pre-filtering module cannot be maximized. Summary of the invention

[0008] In view of the deficiencies in the prior art, the present invention aims to provide a malicious traffic detection method based on feature rules.

[0009] In order to achieve the above object, the present invention provides the following technical solutions:

[0010] A malicious traffic detection method based on feature rules specifically includes the following steps:

[0011] Extract malicious traffic detection feature rules one by one, perform format verification, and extract relevant rule option information;

[0012] Construct a rule divide-and-conquer chain based on relevant rule option information, and group the rules with the same divide-and-conquer chain path into rule groups;

[0013] According to the rule grouping, select some rule options of each rule and construct the multi-mode aggregation graph of the corresponding grouping;

[0014] According to the multi-mode aggregation graph, a precise detection chain of each rule is constructed;

[0015] The data to be detected extracted after deep decoding of network traffic is input into the rule divide-and-conquer chain, the corresponding multi-mode concentration graph is searched, and then according to the detection result of the multi-mode concentration graph, it is input into the corresponding precise detection chain to obtain the detection result;

[0016] Select the traffic that meets the characteristic rules in the detection results and input it into the post-detection engine for further identification and detection.

[0017] In the present invention, preferably, the option feature is a fingerprint feature contained in malicious traffic, and the same rule contains multiple option features.

[0018] In the present invention, preferably, the rule divide-and-conquer chain is constructed in the order of destination port divide-and-conquer group, protocol divide-and-conquer group, and detection direction divide-and-conquer group.

[0019] In the present invention, preferably, the destination port partition group expands all port values ​​into an array to establish a plurality of port arrays.

[0020] In the present invention, preferably, the protocol partition group expands all the values ​​of the protocol into an array to establish several protocol arrays.

[0021] In the present invention, preferably, the construction rule grouping specifically includes:

[0022] According to the requirements of each division group, the three data sources of the destination port, protocol, and detection direction of each rule are extracted in turn as division information. According to the requirements of each division group, the position of each division group is found according to the extracted division information to form a division chain of the corresponding rule.

[0023] Combine the rules on the same divide-and-conquer chain path to form corresponding rule groups.

[0024] In the present invention, preferably, constructing a multi-mode aggregation graph specifically includes:

[0025] In the same rule group, according to the confidence of different types of rule options, several rule options are selected as the preliminary rule option sources for constructing the multi-mode aggregation graph;

[0026] According to the rule options selected by each rule in the same rule group, the frequency of use of the rule options is counted, and the two most frequently used rule options are selected as the data source for constructing the multi-mode intensive graph;

[0027] According to the two selected rule options, each rule selects the corresponding options and puts them into the multi-mode aggregation graph. The multi-mode aggregation graph extracts the option information according to each rule and generates the corresponding rule bitmap set.

[0028] In the present invention, preferably, the confidence of the rule options is based on the fingerprint frequency used by actual malicious traffic, and different confidence values ​​are set for various rule options to form a rule option confidence table.

[0029] In the present invention, preferably, the specific steps of constructing the precise detection chain of each rule are to sort the rule options of each rule according to the confidence level, and put them into the precise detection chain list in sequence to form a precise detection chain.

[0030] In the present invention, preferably, the specific steps of obtaining the test results include:

[0031] Deeply decode network traffic and extract data to be detected;

[0032] The data to be tested goes through the divide-and-conquer chain, multi-mode intensive graph, and precise detection chain in turn to obtain the final rule detection result.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] The method of the present invention is applied in systems such as network security monitoring that require attack detection, early warning and blocking of network data, improves the detection performance of a front-end malicious traffic detection engine based on feature rules, solves the problem of poor detection effect of network security products caused by insufficient performance of a front-end detection engine based on feature rules, and achieves the effect of timely discovering suspicious traffic and more quickly eliminating normal traffic. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 A schematic diagram of the flow of a malicious traffic detection method based on feature rules described in the present invention.

[0036] Figure 2 The figure is a schematic diagram of the divide-and-conquer chain construction process described in the present invention.

[0037] Figure 3 The figure is a schematic diagram of the process of constructing the multi-mode intensive graph according to the present invention.

[0038] Figure 4 Schematic diagram of the process of constructing the precise detection chain described in the present invention. DETAILED DESCRIPTION

[0039] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used herein in the specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The term "and / or" used herein includes any and all combinations of one or more related listed items.

[0041] Related terms explanation:

[0042] Essential features: seven pieces of information: the detection rule's protocol, source IP, destination IP, source port, destination port, detection traffic direction, and rule ID.

[0043] Option feature: Option feature is the fingerprint feature contained in malicious traffic. The same rule can contain multiple option features.

[0044] Rule options: Rule options are the smallest feature unit of a rule, including required features + optional features.

[0045] Option confidence: Option confidence is the confidence set based on the frequency of each option feature participating in the detection in the traffic based on big data statistics. The higher the confidence, the greater the possibility that the option will participate in the detection.

[0046] Data processing module: In network monitoring and network security detection systems, an independent module is usually required to deeply decode the data in the traffic, extract the data to be detected, and send it to the detection engine for detection.

[0047] Please also see Figures 1 to 4 A preferred embodiment of the present invention provides a malicious traffic detection method based on feature rules to improve the detection performance of a front-end malicious traffic detection engine based on feature rules, and solve the problem of poor detection effect of network security products caused by insufficient performance of the front-end detection engine based on feature rules. It can be used to extract the data to be detected based on real-time network traffic, and promptly and quickly discover a small part of suspicious malicious attack traffic for use by post-detection engines such as machine learning detection, and accurately identify, warn, and block malicious attack traffic. The method specifically includes the following steps:

[0048] S1. Extract malicious traffic detection feature rules one by one, perform format verification, and extract relevant rule option information;

[0049] S2. Build a rule divide-and-conquer chain based on relevant rule option information, and group the rules with the same divide-and-conquer chain path into rule groups;

[0050] S3, according to the rule grouping, select some rule options of each rule, and construct a multi-mode intensive graph of the corresponding grouping;

[0051] S4. Constructing a precise detection chain of each rule according to the multi-mode intensive graph;

[0052] S5, input the data to be detected extracted after deep decoding of network traffic into the rule divide-and-conquer chain, search for the corresponding multi-mode concentration graph, and then input it into the corresponding precise detection chain according to the detection result of the multi-mode concentration graph to obtain the detection result;

[0053] S6. Select the traffic that meets the characteristic rules in the detection results and input it into the post-detection engine for further identification and detection.

[0054] In this implementation, the option feature is a fingerprint feature included in malicious traffic, and the same rule includes multiple option features.

[0055] Specifically, in step S1, the original rule decoding mainly involves loading, verifying and decoding the required features and optional features in the feature rules one by one according to the rule format, and extracting the rule information required to build the detection engine.

[0056] Among them, the rule option information of a single rule is extracted. A single rule consists of two parts: required features and option features. The required features and option features are extracted separately. The required features include seven parts of information: the protocol of the traffic to be detected, source IP, destination IP, source port, destination port, the direction of the detected traffic, and the ID of the rule; the option feature is the fingerprint feature contained in the malicious traffic, and the same rule can contain multiple option features.

[0057] Specifically, in step S2, the rule divide-and-conquer chain is constructed in the order of a destination port divide-and-conquer group, a protocol divide-and-conquer group, and a detection direction divide-and-conquer group. The destination port divide-and-conquer group expands all port values ​​into an array to establish a plurality of port arrays. The protocol divide-and-conquer group expands all protocol values ​​into an array to establish a plurality of protocol arrays.

[0058] According to the requirements of each divide-and-conquer group, the three data sources of the destination port, protocol, and detection direction of each rule are extracted in turn as divide-and-conquer information. According to the requirements of each divide-and-conquer group, the position of each divide-and-conquer group is found according to the extracted divide-and-conquer information to form a divide-and-conquer chain of the corresponding rules; the rules on the same divide-and-conquer chain path are combined to form a corresponding rule group. The divide-and-conquer chain is constructed according to the necessary rule characteristics to achieve rapid layer-by-layer screening of traffic. The port divide-and-conquer group in the divide-and-conquer chain uses an array of 65536 in size. Compared with the traditional port group linked list, the search time complexity is reduced to O(1) in actual detection.

[0059] In this embodiment, the port partition group expands all possible values ​​of the port into an array, establishes an array of size 65536, reduces the time complexity to O(1), and improves the accuracy of rule detection of destination port hits.

[0060] In this embodiment, the protocol divide-and-conquer group expands all possible values ​​of the protocol into an array, establishes an array of size 256, and reduces the time complexity to o(1) to improve the hit accuracy of the rule detection protocol.

[0061] Specifically, in step S3, constructing a multi-mode aggregation graph specifically includes:

[0062] In the same rule group, according to the confidence of different types of rule options, several rule options are selected as the preliminary rule option sources for constructing the multi-mode aggregation graph;

[0063] According to the rule options selected by each rule in the same rule group, the frequency of use of the rule options is counted, and the two most frequently used rule options are selected as the data source for constructing the multi-mode intensive graph;

[0064] According to the two selected rule options, each rule selects the corresponding option and puts it into the multi-mode aggregation graph. The multi-mode aggregation graph extracts the option information according to each rule and generates the corresponding rule bitmap set. The use of the multi-mode aggregation graph avoids the current feature rule malicious traffic detection engine can only select one option feature as a multi-mode filtering restriction, and uses multiple option features to form an aggregation graph to further enhance the multi-mode filtering effect.

[0065] In this embodiment, the confidence of the rule options is based on the fingerprint frequency used by actual malicious traffic, and different confidence values ​​are set for various rule options to form a rule option confidence table. The application of the confidence table avoids the current feature rule malicious traffic detection engine can only select multi-mode option features or build a precise detection chain based on the order of option features in the rule or the configuration order of the configuration file. According to the detection frequency of option features collected in different user environments and based on big data analysis, a confidence table can be constructed, and option features can be selected and the option feature detection order can be optimized based on the confidence table, so as to more accurately encounter the option features to be detected, so as to achieve the effect of more timely detection of suspicious traffic and faster elimination of normal traffic.

[0066] Specifically, in step S4, the specific steps of constructing the precise detection chain of each rule include sorting each rule option of each rule according to the confidence level, and sequentially putting them into the precise detection chain list to form a precise detection chain.

[0067] Specifically, in step S5, the specific steps of obtaining the detection result include:

[0068] Deeply decode network traffic and extract data to be detected;

[0069] The detection data is sequentially passed through the divide-and-conquer chain, multi-mode intensive graph, and precise detection chain to obtain the final rule detection result. The precise detection chain is constructed based on the confidence of the option features, and the option features with high confidence are moved forward to promptly find and filter out the rules that do not meet the features, avoiding the current feature rule malicious traffic detection engine. Regardless of the performance of the option features, the precise detection chain can only be constructed based on the order of the option features in the rule.

[0070] Specifically, in step S6, the traffic hit by the feature rule engine is sent to a post-detection engine such as a machine learning engine for further identification and detection, and an alarm message is output based on the detection result.

[0071] In this embodiment, the detection order of each option feature in the multi-mode intensive diagram can be continuously adjusted according to the usage frequency of each option feature in the real-time detection process, thereby avoiding the disadvantage that the current feature rule malicious traffic detection engine cannot be dynamically adjusted regardless of the performance during the detection process, and can dynamically adjust the detection engine performance according to different traffic environments to achieve the optimal detection state.

[0072] In some other preferred embodiments of the present invention, a computer-readable storage medium is provided, storing a computer program. When the computer program is executed by a processor, the processor executes the steps of the method described in the above embodiment.

[0073] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0074] The above description is a detailed description of the preferred feasible embodiments of the present invention, but the embodiments are not intended to limit the scope of the patent application of the present invention. All equivalent changes or modified changes completed under the technical spirit suggested by the present invention should fall within the patent scope covered by the present invention.

Claims

1. A malicious traffic detection method based on feature rules, characterized in that: The specific steps include: Extract malicious traffic detection feature rules one by one, perform format verification, and extract relevant rule option information; Construct a rule divide-and-conquer chain based on relevant rule option information, and group the rules with the same divide-and-conquer chain path into rule groups; According to the rule grouping, select some rule options of each rule and construct the multi-mode aggregation graph of the corresponding grouping; According to the multi-mode aggregation graph, a precise detection chain of each rule is constructed; The data to be detected extracted after deep decoding of network traffic is input into the rule divide-and-conquer chain, the corresponding multi-mode concentration graph is searched, and then according to the detection result of the multi-mode concentration graph, it is input into the corresponding precise detection chain to obtain the detection result; Select the traffic that meets the characteristic rules in the detection results and input it into the post-detection engine for further identification and detection.

2. According to the method for detecting malicious traffic based on characteristic rules in claim 1, it is characterized in that: The option feature is a fingerprint feature contained in malicious traffic, and the same rule contains multiple option features.

3. A method for detecting malicious traffic based on characteristic rules according to claim 2, characterized in that: The rule divide-and-conquer chain is constructed in the order of a destination port divide-and-conquer group, a protocol divide-and-conquer group, and a detection direction divide-and-conquer group.

4. According to claim 3, a malicious traffic detection method based on characteristic rules is characterized in that: The destination port partition group expands all port values ​​into an array to establish a plurality of port arrays.

5. The method for detecting malicious traffic based on characteristic rules according to claim 3 is characterized in that: The protocol partitioning group expands all the values ​​of the protocol into an array and establishes a plurality of protocol arrays.

6. The method for detecting malicious traffic based on characteristic rules according to claim 3 is characterized in that: The construction rule grouping specifically includes: According to the requirements of each division group, the three data sources of the destination port, protocol, and detection direction of each rule are extracted in turn as division information. According to the requirements of each division group, the position of each division group is found according to the extracted division information to form a division chain of the corresponding rule. Combine the rules on the same divide-and-conquer chain path to form corresponding rule groups.

7. The method for detecting malicious traffic based on characteristic rules according to claim 1, characterized in that: Constructing a multi-mode intensive graph specifically includes: In the same rule group, according to the confidence of different types of rule options, several rule options are selected as the preliminary rule option sources for constructing the multi-mode aggregation graph; According to the rule options selected by each rule in the same rule group, the frequency of use of the rule options is counted, and the two most frequently used rule options are selected as the data source for constructing the multi-mode intensive graph; According to the two selected rule options, each rule selects the corresponding options and puts them into the multi-mode aggregation graph. The multi-mode aggregation graph extracts the option information according to each rule and generates the corresponding rule bitmap set.

8. A method for detecting malicious traffic based on characteristic rules according to claim 7, characterized in that: The confidence of the rule options is based on the fingerprint frequency used by actual malicious traffic. Different confidence values ​​are set for various rule options to form a rule option confidence table.

9. The method for detecting malicious traffic based on characteristic rules according to claim 1, characterized in that: The specific steps of constructing the precise detection chain of each rule include sorting the rule options of each rule according to the confidence level, and sequentially putting them into the precise detection chain list to form a precise detection chain.

10. The method for detecting malicious traffic based on characteristic rules according to claim 1, characterized in that: The specific steps to obtain the test results include: Deeply decode network traffic and extract data to be detected; The data to be tested goes through the divide-and-conquer chain, multi-mode intensive graph, and precise detection chain in turn to obtain the final rule detection result.

Images