A low false positive traffic analysis method based on multi-dimensional data

Abstract

The invention discloses a low false alarm traffic analysis method based on multidimensional data, which belongs to the field of data processing technology. The invention is used to solve the technical problems of poor autonomous implementation management effect and poor autonomous supervision optimization effect of low false alarm traffic analysis in existing solutions. The invention performs multidimensional data monitoring statistics and digital processing calculation on the target traffic to be identified to obtain the corresponding transmission influence degree, performs data analysis on the transmission influence degree and adaptively implements corresponding identification and analysis schemes for different target traffic. The invention performs retrospective statistics on the identification and analysis results corresponding to the target traffic, and adaptively evaluates the identification effectiveness of the identification effect corresponding to the enhanced identification and analysis scheme according to the retrospective statistical results. According to the evaluation results, the subsequent implementation and upgrade optimization of the enhanced identification and analysis scheme are adaptively managed dynamically, thereby realizing multi-level data processing and calculation analysis of the enhanced identification and analysis scheme.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a low false positive traffic analysis method based on multi-dimensional data. Background Art

[0002] Low false positive traffic analysis refers to minimizing the occurrence of false alarms during network traffic analysis, that is, reducing the situation where normal traffic is misjudged as abnormal or malicious traffic; false alarms not only waste the security team's time and resources, but may also cause unnecessary interference with legitimate business activities.

[0003] After searching, the prior art with application number CN2023112924141 and name of a network attack detection method, system and related equipment with low false alarm rate discloses that by analyzing the threat messages in the network traffic, and converting the attack behaviors that are difficult to determine in the threat messages into monitorable behaviors that are easy to perceive and determine through code weakening means, and obtaining the network attack detection results according to the actual message behaviors. Compared with the prior art, the prior art does not identify the threat messages according to the preset feature library, thereby improving the recognition rate of unknown network attacks; and based on the perceptible message encapsulation method, it can reduce the occupation of system resources by the network attack detection process and reduce the false alarm rate of network attack detection.

[0004] Although the existing low false positive traffic analysis solutions are designed to improve the accuracy of network traffic analysis, there are still some problems and challenges. They cannot perform data analysis on different traffic feature data and adaptively use traffic analysis solutions dynamically, and cannot retroactively supervise the results of different traffic analysis solutions and adaptively dynamically optimize and adjust them, resulting in poor autonomous implementation and management of low false positive traffic analysis and poor autonomous supervision and optimization effects. Summary of the invention

[0005] The purpose of the present invention is to provide a low false alarm traffic analysis method based on multidimensional data, which is used to solve the technical problems of poor autonomous implementation management effect and poor autonomous supervision and optimization effect of low false alarm traffic analysis in existing solutions.

[0006] The purpose of the present invention can be achieved through the following technical solutions:

[0007] A low false positive traffic analysis method based on multi-dimensional data, comprising:

[0008] Monitor, count and process multi-dimensional data of the target traffic to be identified, and calculate and analyze the processed data to determine the identification and analysis scheme corresponding to the target traffic; the identification and analysis scheme corresponding to the target traffic includes a conventional identification and analysis scheme or an enhanced identification and analysis scheme;

[0009] The identification and analysis results corresponding to the target traffic are traced and counted, and the identification effectiveness evaluation of the corresponding enhanced identification and analysis scheme is adaptively performed based on the traceability statistical results. The subsequent implementation and upgrade optimization of the enhanced identification and analysis scheme are adaptively managed dynamically based on the evaluation results.

[0010] Preferably, when monitoring, counting and processing multi-dimensional data of the target traffic to be identified, the data packet size, transmission protocol, initiating IP address and target IP address corresponding to the target traffic are obtained;

[0011] When the initiating IP address and the target IP address are digitized, the initiating IP address and the target IP address are respectively input into the address risk database for traversal retrieval;

[0012] If neither the initiating IP address nor the target IP address exists in the address risk database, the address impact value is set to DY1;

[0013] If the initiating IP address or the target IP address exists in the address risk database, the address impact value associated with the corresponding sample IP address is set to DY2;

[0014] If both the initiating IP address and the target IP address exist in the address risk database, the address impact value associated with the corresponding sample IP address is set to DY3.

[0015] Preferably, the digitally processed data are processed by formula Calculate the transmission impact CY corresponding to the target flow; where k = 1, 2, 3; DYk is DY1, DY2 or DY3; CZ is the data packet size corresponding to the target flow; A is the standard data packet size; CQ is the transmission impact weight corresponding to the transmission protocol of the target flow; [] is the rounding function;

[0016] When determining the identification and analysis scheme corresponding to the target traffic according to the calculated transmission impact, data analysis is performed on the transmission impact. If the transmission impact is 0, a conventional identification and analysis scheme is generated.

[0017] If the transmission impact is not 0, an enhanced identification and analysis plan is generated.

[0018] Preferably, the identification and analysis results of the identification and analysis scheme corresponding to the target traffic are traced back and counted, and if the identification and analysis result is normal, the total number of normal identifications of the corresponding identification and analysis result is increased by one;

[0019] If the recognition and analysis result is abnormal, the total number of abnormal recognitions of the corresponding recognition and analysis result will be increased by one.

[0020] Preferably, when evaluating the recognition effectiveness of the recognition effects corresponding to different recognition analysis schemes, the formula Calculate and obtain the recognition discovery rate SFi corresponding to different recognition and analysis schemes; where i = 1, 2; YFi is YF1 and YF2, which are the total number of abnormal recognitions corresponding to the conventional recognition and analysis scheme and the enhanced recognition and analysis scheme, respectively; Ni is N1 and N2, which are the total number of recognitions corresponding to the conventional recognition and analysis scheme and the enhanced recognition and analysis scheme, respectively;

[0021] The recognition discovery rate corresponding to the conventional recognition analysis scheme is marked as the first recognition discovery rate SF1, and the recognition discovery rate corresponding to the enhanced recognition analysis scheme is marked as the second recognition discovery rate SF2.

[0022] Preferably, by the formula Calculate and obtain the enhanced recognition validity QX corresponding to the enhanced recognition analysis scheme; where B is the standard error rate corresponding to the conventional recognition analysis scheme;

[0023] If the reinforcement identification validity is 0, then a reinforcement unnecessary instruction is generated;

[0024] If the reinforcement identification validity is 1, a reinforcement necessary instruction is generated.

[0025] Preferably, the negative impact corresponding to the implementation of the enhanced identification and analysis plan is calculated and analyzed according to the enhanced necessary instructions;

[0026] Count the data processing resources WZj corresponding to each implementation of the enhanced recognition and analysis plan, j = 1, 2, 3, ..., N2, and use the formula Calculate and implement the enhanced recognition impact QY corresponding to all enhanced recognition analysis schemes; C is the standard data processing resource; [] is the rounding function;

[0027] Dynamically manage the subsequent implementation and upgrade optimization of the enhanced identification analysis plan based on the impact of enhanced identification.

[0028] Preferably, data analysis is performed on the enhanced recognition impact, and if the enhanced recognition impact is 0, the subsequent implementation of the enhanced recognition analysis plan is maintained;

[0029] If the enhanced recognition impact is 1, the enhanced recognition analysis scheme is upgraded and optimized.

[0030] Compared with the existing solutions, the present invention achieves the following beneficial effects:

[0031] The present invention obtains the corresponding transmission impact by monitoring and counting the multi-dimensional data of the target traffic to be identified and performing digital processing and calculation, performs data analysis on the transmission impact and adaptively implements corresponding identification and analysis schemes for different target traffic, thereby improving the management effect of autonomous implementation of low false alarm traffic analysis.

[0032] The present invention performs retrospective statistics on the identification and analysis results corresponding to the target traffic, and adaptively evaluates the identification effectiveness of the identification effect corresponding to the enhanced identification and analysis scheme based on the retrospective statistical results. According to the evaluation results, the subsequent implementation and upgrade optimization of the enhanced identification and analysis scheme are adaptively managed dynamically, thereby realizing multi-level data processing and computational analysis of the enhanced identification and analysis scheme, and improving the autonomous supervision and optimization effect of low false alarm traffic analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The present invention will be further described below in conjunction with the accompanying drawings.

[0034] Figure 1 The present invention is a flowchart of a low false positive traffic analysis method based on multi-dimensional data.

[0035] Figure 2 This is a flowchart of the identification and analysis solution for dynamically generating target traffic in the present invention.

[0036] Figure 3 This is a flowchart for dynamically managing the subsequent implementation and upgrade optimization of the enhanced identification and analysis solution in the present invention. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] like Figure 1 As shown, the present invention is a low false positive traffic analysis method based on multi-dimensional data, comprising:

[0039] Monitor, count and process the multi-dimensional data of the target traffic to be identified, and calculate and analyze the processed data to determine the identification and analysis plan corresponding to the target traffic; including:

[0040] When monitoring, counting and processing multi-dimensional data of the target traffic to be identified, the data packet size, transmission protocol, initiating IP address and target IP address corresponding to the target traffic are obtained;

[0041] The target traffic may be network traffic, and the network traffic may be obtained by using an existing network packet capture tool. The specific acquisition steps are not described in detail here.

[0042] In addition, the transmission protocols include but are not limited to TCP, UDP and ICMP, etc. Different transmission protocols correspond to a transmission impact weight, and the transmission impact weight is used to digitally represent the security of the transmission protocol. The specific value of the transmission impact weight can be set by professional technicians in this field according to the security level;

[0043] When the initiating IP address and the target IP address are digitized, the initiating IP address and the target IP address are respectively input into the address risk database for traversal retrieval;

[0044] Among them, the address risk database pre-stores a number of sample IP addresses, which are risk addresses, constructed based on historical risk identification data of different platforms and supplemented by manual marking according to actual application conditions;

[0045] If the initiating IP address and the target IP address do not exist in the address risk database, the address impact value is set to DY1; wherein DY1 = 0;

[0046] If the initiating IP address or the target IP address exists in the address risk database, the address impact value associated with the corresponding sample IP address is set to DY2;

[0047] If both the initiating IP address and the target IP address exist in the address risk database, the address impact value associated with the corresponding sample IP address is set to DY3; DY3>DY2>0;

[0048] like Figure 2 As shown, the digitally processed data are converted into Calculate the transmission impact CY corresponding to the target traffic; where k = 1, 2, 3; DYk is DY1, DY2 or DY3; CZ is the data packet size corresponding to the target traffic; A is the standard data packet size, which is determined based on the median of all data packet sizes processed historically; CQ is the transmission impact weight corresponding to the transmission protocol of the target traffic; [] is the rounding function;

[0049] It should be noted that the transmission impact is used to integrate and calculate the data of different dimensions of the target traffic to digitally represent its transmission impact; the greater the transmission impact, the more targeted identification and analysis solutions are needed for the low false alarm analysis of the corresponding target traffic;

[0050] When determining the identification and analysis scheme corresponding to the target traffic according to the calculated transmission impact, data analysis is performed on the transmission impact. If the transmission impact is 0, a conventional identification and analysis scheme is generated.

[0051] If the transmission impact is not 0, an enhanced identification and analysis plan is generated;

[0052] It should be noted that the enhanced identification and analysis scheme can be a dynamic execution network attack detection scheme represented by taint analysis, which has the advantage of being able to discover unknown attacks; the disadvantage is that it consumes a lot of computing resources and has a slow running speed;

[0053] By implementing the enhanced identification and analysis scheme, targeted data analysis can be performed on target traffic with high transmission impact risks, thereby realizing dynamic use of traffic analysis schemes adaptively based on different traffic characteristics. Compared with the fixed traffic identification and analysis schemes in the prior art schemes, the embodiments of the present invention can effectively improve the flexibility and reliability of traffic identification and analysis.

[0054] In the embodiment of the present invention, the corresponding transmission impact is obtained by monitoring and counting the multi-dimensional data of the target traffic to be identified and digitally processing and calculating, data analysis is performed on the transmission impact, and corresponding identification and analysis schemes are adaptively implemented for different target traffic, thereby improving the management effect of autonomous implementation of low false alarm traffic analysis.

[0055] Conduct retrospective statistics on the identification and analysis results corresponding to the target traffic, and adaptively evaluate the identification effectiveness of the enhanced identification and analysis solution based on the retrospective statistical results. Adaptively dynamically manage the subsequent implementation and upgrade optimization of the enhanced identification and analysis solution based on the evaluation results; including:

[0056] The identification and analysis results of the identification and analysis scheme corresponding to the target traffic are traced back and counted. If the identification and analysis result is normal, the total number of normal identifications of the corresponding identification and analysis result is increased by one;

[0057] If the recognition and analysis result is abnormal, the total number of abnormal recognitions of the corresponding recognition and analysis result is increased by one;

[0058] When evaluating the recognition effectiveness of different recognition analysis schemes, the formula Calculate and obtain the recognition discovery rate SFi corresponding to different recognition and analysis schemes; where i = 1, 2; YFi is YF1 and YF2, which are the total number of abnormal recognitions corresponding to the conventional recognition and analysis scheme and the enhanced recognition and analysis scheme, respectively; Ni is N1 and N2, which are the total number of recognitions corresponding to the conventional recognition and analysis scheme and the enhanced recognition and analysis scheme, respectively;

[0059] The recognition discovery rate corresponding to the conventional recognition analysis scheme is marked as the first recognition discovery rate SF1, and the recognition discovery rate corresponding to the enhanced recognition analysis scheme is marked as the second recognition discovery rate SF2; and through the formula The enhanced recognition validity QX corresponding to the enhanced recognition analysis scheme is calculated; where B is the standard error rate corresponding to the conventional recognition analysis scheme, which can be determined based on the actual design requirement data corresponding to the existing conventional recognition analysis scheme, that is, the proportion of negative impacts that are subsequently caused by abnormalities that are not discovered by the conventional recognition analysis scheme;

[0060] It should be noted that enhanced recognition validity is used to digitally represent the difference between the recognition findings corresponding to the enhanced recognition analysis scheme and the recognition findings corresponding to the conventional recognition analysis scheme;

[0061] In the embodiment of the present invention, by tracing and counting the identification and analysis results corresponding to the target traffic, digitally processing the identification data corresponding to different identification and analysis schemes, and calculating the enhanced identification validity corresponding to the enhanced identification and analysis scheme, it is possible to digitally represent the identification and discovery status corresponding to the enhanced identification and analysis scheme, and provide reliable identification and discovery data support for the subsequent implementation and upgrade optimization management of the subsequent enhanced identification and analysis scheme;

[0062] If the enhanced recognition validity is 0, an enhanced unnecessary instruction is generated. Specifically, the implementation effect of the enhanced recognition analysis plan is not much different from the implementation effect of the full implementation of the conventional recognition analysis plan, and more data processing resources are generated. It is unnecessary to dynamically implement the enhanced recognition analysis plan later;

[0063] like Figure 3 As shown, if the reinforcement identification validity is 1, a reinforcement necessary instruction is generated, and the data of the negative impact corresponding to the implementation of the reinforcement identification analysis plan is calculated and analyzed according to the reinforcement necessary instruction;

[0064] It should be explained that strengthening the necessary instructions means that the implementation effect of the corresponding enhanced identification and analysis plan is greater than the implementation effect of the full implementation of the conventional identification and analysis plan; however, whether the negative impact of the implementation of the enhanced identification and analysis plan meets the identification and analysis requirements requires further retrospective analysis and dynamic management;

[0065] Statistics of the data processing resources WZj corresponding to each implementation of the enhanced identification and analysis scheme, j = 1, 2, 3, ..., N2; the data processing resources can be specifically the network traffic resources required for the target traffic to pass through the enhanced identification and analysis scheme, and through the formula Calculate and implement the enhanced recognition impact QY corresponding to all enhanced recognition and analysis schemes; C is the standard data processing resource, which can be determined according to the data processing resource requirements corresponding to the conventional recognition and analysis scheme, or according to the median of the data processing resources corresponding to all historical conventional recognition and analysis schemes; [] is the rounding function;

[0066] It should be noted that the enhanced identification impact is used to further digitally represent the impact on the use of data processing resources generated by the subsequent implementation of the enhanced identification analysis plan when necessary; the greater the enhanced identification impact, the worse the impact on the use of data processing resources generated by the implementation of the corresponding enhanced identification analysis plan;

[0067] When dynamically managing the subsequent implementation and upgrade optimization of the enhanced identification and analysis plan, data analysis of the impact of enhanced identification is performed;

[0068] If the enhanced identification impact is 0, the subsequent implementation of the enhanced identification analysis plan is maintained;

[0069] If the enhanced identification impact is 1, the enhanced identification analysis plan is upgraded and optimized to ensure that the enhanced identification impact corresponding to the subsequent implementation of the enhanced identification analysis plan is 0;

[0070] In the embodiment of the present invention, by retrospectively counting the identification and analysis results corresponding to the target traffic, and adaptively evaluating the identification effectiveness of the identification effect corresponding to the enhanced identification and analysis scheme based on the retrospective statistical results, and adaptively managing the subsequent implementation and upgrade optimization of the enhanced identification and analysis scheme based on the evaluation results, multi-level data processing and computational analysis of the enhanced identification and analysis scheme are achieved, thereby improving the autonomous supervision and optimization effect of low false alarm traffic analysis.

[0071] In addition, the formulas involved in the above are all dimensionless and numerical calculations. They are a formula that is closest to the actual situation obtained by collecting a large amount of data and simulating it with simulation software.

[0072] In the several embodiments provided by the present invention, it should be understood that the disclosed method can be implemented in other ways. For example, the above-described embodiments of the invention are only illustrative, for example, the division of modules is only a logical function division, and there may be other division methods in actual implementation.

[0073] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical modules, and may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0074] In addition, each functional module in each embodiment of the present invention may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0075] It is obvious to a person skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the essential characteristics of the present invention.

[0076] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A low false positive traffic analysis method based on multi-dimensional data, characterized in that: include: Monitor, count and process multi-dimensional data of the target traffic to be identified, and calculate and analyze the processed data to determine the identification and analysis plan corresponding to the target traffic; The identification and analysis scheme corresponding to the target traffic includes a conventional identification and analysis scheme or an enhanced identification and analysis scheme; When monitoring, counting and processing multi-dimensional data of the target traffic to be identified, the data packet size, transmission protocol, initiating IP address and target IP address corresponding to the target traffic are obtained; When the initiating IP address and the target IP address are digitized, the initiating IP address and the target IP address are respectively input into the address risk database for traversal retrieval; If neither the initiating IP address nor the target IP address exists in the address risk database, the address impact value is set to DY1; If the initiating IP address or the target IP address exists in the address risk database, the address impact value associated with the corresponding sample IP address is set to DY2; If both the initiating IP address and the target IP address exist in the address risk database, the address impact value associated with the corresponding sample IP address is set to DY3; The digitally processed data are converted into Calculate the transmission impact CY corresponding to the target traffic; where k=1, 2, 3; DYk is DY1, DY2 or DY3; CZ is the data packet size corresponding to the target traffic; A is the standard data packet size; CQ is the transmission impact weight corresponding to the transmission protocol of the target traffic; [] is the rounding function; When determining the identification and analysis scheme corresponding to the target traffic according to the calculated transmission impact, data analysis is performed on the transmission impact. If the transmission impact is 0, a conventional identification and analysis scheme is generated. If the transmission impact is not 0, an enhanced identification and analysis plan is generated; Conduct retrospective statistics on the identification and analysis results corresponding to the target traffic, and adaptively evaluate the identification effectiveness of the enhanced identification and analysis solution based on the retrospective statistical results. Adaptively dynamically manage the subsequent implementation and upgrade optimization of the enhanced identification and analysis solution based on the evaluation results. Among them, when evaluating the recognition effectiveness of the recognition effects corresponding to different recognition analysis schemes, the formula Calculate and obtain the recognition discovery rate SFi corresponding to different recognition and analysis schemes; where i=1, 2; YFi is YF1 and YF2, which are the total number of abnormal recognitions corresponding to the conventional recognition and analysis scheme and the enhanced recognition and analysis scheme, respectively; Ni is N1 and N2, which are the total number of recognitions corresponding to the conventional recognition and analysis scheme and the enhanced recognition and analysis scheme, respectively; The recognition discovery rate corresponding to the conventional recognition analysis scheme is marked as the first recognition discovery rate SF1, and the recognition discovery rate corresponding to the enhanced recognition analysis scheme is marked as the second recognition discovery rate SF2; By formula Calculate and obtain the enhanced recognition validity QX corresponding to the enhanced recognition analysis scheme; where B is the standard error rate corresponding to the conventional recognition analysis scheme; If the reinforcement identification validity is 0, then a reinforcement unnecessary instruction is generated; If the reinforcement identification validity is 1, then the reinforcement necessary instruction is generated; Calculate and analyze the negative impacts of implementing the enhanced identification and analysis program according to the necessary instructions; Count the data processing resources WZj corresponding to each implementation of the enhanced recognition and analysis plan, j = 1, 2, 3, ..., N2, and use the formula Calculate and implement the enhanced recognition impact QY corresponding to all enhanced recognition analysis schemes; C is the standard data processing resource; [] is the rounding function; Dynamically manage the subsequent implementation and upgrade optimization of the enhanced identification analysis plan based on the enhanced identification impact; Conduct data analysis on the impact of enhanced identification. If the impact of enhanced identification is 0, maintain the subsequent implementation of the enhanced identification analysis plan; If the enhanced recognition impact is 1, the enhanced recognition analysis scheme is upgraded and optimized.

2. According to the method of low false positive traffic analysis based on multi-dimensional data in claim 1, it is characterized in that: The identification and analysis results of the identification and analysis scheme corresponding to the target traffic are traced back and counted. If the identification and analysis result is normal, the total number of normal identifications of the corresponding identification and analysis result is increased by one; If the recognition analysis result is abnormal, the total number of abnormal recognitions of the corresponding recognition analysis result is increased by one.

Images