Security detection method and device for network transmission data

Abstract

The invention discloses a security detection method and device for network transmission data, and belongs to the field of data security. The method comprises the following steps: performing security detection on network transmission data by using a security detection model and a clustering model; and outputting a detection result based on comparison of detection results of the security detection model and the clustering model. When the comparison result is inconsistent, calculating a comprehensive safety coefficient, and storing the network transmission data into an experience sample pool; and respectively carrying out model optimization on the security detection model and the clustering model by utilizing samples in the empirical sample pool. According to the scheme, the supervised security detection model and the unsupervised clustering model are used for parallel detection, and bimodal high-precision detection is generated; through cross-modal conjoint analysis of two modal models, doubtful network transmission data is stored in an experience sample pool, parallel optimization of a supervised security detection model and an unsupervised clustering model is realized by using a sample of the experience sample pool, online closed-loop hot update of the models is realized, dynamic perception of novel attacks is improved, and the misjudgment rate is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data security, and in particular to a network transmission data security detection method and device. BACKGROUND

[0002] With the development of industrial internet intelligence, the complexity and concealment of network threats have significantly increased. In client and server data interaction, unsafe data may appear, and attack means have gradually evolved from traditional known vulnerability exploitation to complex multi-stage attack chain advanced threats.

[0003] Existing industrial network data security protection mainly relies on rule matching, feature signature or static model reasoning, which has certain effect on known threat detection, but when facing unknown threats, variant attacks and other advanced threats, the static model cannot adapt to the dynamic changes of the industrial environment after deployment, resulting in a lag in response to new attacks, high false positive and false negative rates, especially in large-scale deployment scenarios, manual analysis costs are extremely high.

[0004] Therefore, it is urgent to provide a new network transmission data security detection method. SUMMARY

[0005] In order to solve the problem of high false positive and false negative rates of traditional detection methods in the face of complex and variable network threats, the present application provides a network transmission data security detection method and device.

[0006] On the one hand, a network transmission data security detection method is provided, the method comprising: using a pre-trained security detection model and a clustering model to perform security detection on network transmission data; comparing the detection results of the security detection model and the clustering model, and outputting a detection result; when the comparison is inconsistent, calculating a comprehensive security coefficient and storing the network transmission data in an experience sample pool; using samples in the experience sample pool to optimize the security detection model and the clustering model respectively.

[0007] On the other hand, a network transmission data security detection device based on any method embodiment described in the specification is provided, the device comprising: a detection unit configured to use a pre-trained security detection model and a clustering model to perform security detection on network transmission data; a comparison unit configured to compare the detection results of the security detection model and the clustering model, and output a detection result; a collection unit configured to calculate a comprehensive security coefficient when the comparison is inconsistent, and store the network transmission data in an experience sample pool; An optimization unit is configured to optimize the security detection model and the clustering model respectively by using samples in the experience sample pool.

[0008] In another aspect, a computer device is provided, which includes a memory and a processor, the memory is configured to store a computer program, and the processor is configured to execute the computer program stored in the memory to implement the steps of the above method.

[0009] In another aspect, a computer readable storage medium is provided, which stores a computer program, and the computer program is executed by a processor to implement the steps of the above method.

[0010] In another aspect, a computer program product is provided, which includes a computer program, and the computer program is executed by a processor to implement the steps of the above method.

[0011] The technical solution provided by the present application can bring at least the following beneficial effects: The present application breaks through the traditional threshold static and sample annotation bottleneck, utilizes the supervised security detection model and the unsupervised clustering model to detect in parallel, generates a double-mode high-precision detection, and through the cross-modal joint analysis of the two modal models, stores the suspicious network transmission data into the experience sample pool, and utilizes the experience sample pool samples to realize the parallel optimization of the supervised security detection model and the unsupervised clustering model, realizes the online closed-loop hot update of the model, improves the dynamic perception of the new attack, and reduces the misjudgment rate. BRIEF DESCRIPTION OF DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0013] Figure 1 is a flow chart of a network transmission data security detection method provided by an embodiment of the present application; Figure 2 is a structure diagram of a network transmission data security detection device provided by an embodiment of the present application; Figure 3 is a hardware architecture diagram of a computer device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0014] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0015] The specific implementation of the above concept will be described below.

[0016] Referring to Figure 1 The security detection method for network transmission data provided by the embodiments of the present application comprises the following steps. Step 100: using a pre-trained security detection model and a clustering model to perform security detection on network transmission data; Step 102: comparing the detection results of the security detection model and the clustering model, and outputting the detection results; Step 104: when the comparison is inconsistent, calculating a comprehensive security coefficient and storing the network transmission data in an experience sample pool; Step 106: using the samples in the experience sample pool to optimize the security detection model and the clustering model respectively.

[0017] In the embodiments of the present application, the traditional threshold staticization and sample annotation dependence bottleneck are broken through, the supervised security detection model and the unsupervised clustering model are used for parallel detection to generate a double-modal high-precision detection, the suspicious network transmission data is stored in an experience sample pool through cross-modal joint analysis of the two modal models, and the supervised security detection model and the unsupervised clustering model are optimized in parallel using the samples in the experience sample pool, so as to realize online closed-loop hot updating of the model, improve dynamic perception of new attacks, and reduce the false negative rate.

[0018] The execution mode of each step shown in the above description will be described below. Figure 1

[0019] For steps 100 and 102: When the detection results of the security detection model and the clustering model are consistent, the results are directly outputted without being put into the sample pool. When the detection results of the security detection model and the clustering model are inconsistent, the result of the security detection model is outputted, the result is labeled as suspicious, and the network transmission data is put into the sample pool.

[0020] For step 104: In some embodiments, step 104 can comprise the following steps S1-S4: S1, when the comparison is inconsistent, calculating the weight factors of the security detection model and the clustering model based on the historical cumulative accuracy.​

[0021] It can be understood that the detection accuracy is improved by calculating the result reliable weights of the two models based on the accuracy.

[0022] S2, obtaining the detection results of the previous network transmission data and the security detection model and the clustering model in the same IP flow, and calculating the historical sample coefficient.

[0023] In this embodiment, step S2 can include: obtaining the detection results of the previous network transmission data and the security detection model and the clustering model in the same IP flow; If the detection results of the two models are consistent, the historical sample coefficient is 0. If the detection results of the two models are inconsistent, the comprehensive security coefficient of the previous network transmission data is obtained, and the detection results of the previous network transmission data of the previous network transmission data in the same IP flow are continuously obtained. Until the detection results of the two models of the historical network transmission data are consistent, the average value of the comprehensive security coefficients of all continuous historical network transmission data with inconsistent detection results of the two models in the same IP flow is obtained, and the historical sample coefficient is obtained.

[0024] In this embodiment, the historical sample coefficient is introduced, and the detection results of the historical network transmission data in the same IP flow are calculated to obtain the historical sample coefficient.

[0025] Specifically, the following formula is used: In the formula, is the historical sample coefficient, Y is the previous network transmission data, is the set of network transmission data with inconsistent detection results of the two models, is the comprehensive security coefficient of the historical network transmission data with inconsistent detection results of the two models, and n is the number of network transmission data with continuous inconsistent detection results.

[0026] In this embodiment, the average value of the comprehensive security coefficients of the historical network transmission data with continuous inconsistent detection results is taken as the historical sample coefficient, so as to enhance the detection effect of the advanced threat of complex multi-stage attack chain, and the detection accuracy is improved and the contingency is reduced by tracing the historical sample.

[0027] S3, based on the gradient error and probability distribution of the security detection model, the sample space distance of the clustering model, and the transmission risk parameter of the network transmission data, calculating the decay kernel, combining the weight factor and the historical sample coefficient to calculate the comprehensive security coefficient of the network transmission data.

[0028] In this embodiment, the comprehensive security coefficient is calculated by the following method: wherein, wherein, is the comprehensive security factor of the network transmission data, is the gradient error of the security detection model, is the probability distribution vector of the security detection model, represents the L2 norm square of the probability distribution vector, represents the trace of the sample space-time distance matrix of the clustering model , represents the transmission risk parameter of the network transmission data, is the decay kernel, is the historical sample coefficient, is the weight factor of the security detection model, is the weight factor of the clustering model, is the adjustment parameter, is a feature vector composed of the gradient error and the probability distribution of the security detection model, the sample space-time distance of the clustering model, and the transmission risk parameter of the network transmission data, is the weight corresponding to the feature vector.

[0029] In this embodiment, the weight factors of the security detection model and the clustering model are calculated based on the historical cumulative accuracy rate, is the adjustment parameter, used to balance the scales and influences of the transmission risk parameter and the historical sample coefficient. The logarithm of the gradient error is taken to compress the scale of the gradient error and prevent it from dominating the entire expression. The gradient error and the probability distribution of the security detection model, the sample space-time distance of the clustering model, and the transmission risk parameter of the network transmission data are combined linearly and taken to the negative exponential to form a decay kernel. When the items in the parentheses of the decay kernel increase, the comprehensive security factor will exponentially decrease.

[0030] By combining the gradient error and the probability distribution of the security detection model, the sample space-time distance of the clustering model, and the transmission risk parameter of the network transmission data into a feature column vector , and by performing a dot product with the feature vector weight vector, a weighted linear combination of different security indicators is achieved, reflecting the relative importance of each indicator in the comprehensive evaluation. The logarithm of the historical sample coefficient provides a temporal context for the current comprehensive security factor evaluation, improving accuracy.

[0031] This embodiment combines logarithmic, exponential, and quadratic forms for nonlinear fusion, which can capture the complex interactions between indicators. At the same time, it processes scalars, vectors, and matrices in multiple dimensions, making full use of different types of security information. Through weighting factors and adjustment parameters, the formula can flexibly adapt to different security needs and network environments, improve the effectiveness of the comprehensive security coefficient assessment, and further improve the optimization effect.

[0032] S4. Store the network transmission data and its comprehensive security coefficient, as well as historical network transmission data in the same IP stream, into the experience sample pool.

[0033] In this step, the network transmission data and the network transmission data with continuously inconsistent detection results in the same IP stream, along with their comprehensive security coefficients, are stored in the experience sample pool to automatically form a set of suspicious samples.

[0034] Regarding step 106: In some implementations, the security detection model is optimized using samples from an empirical sample pool, including: Based on the comprehensive safety coefficient of the samples, the probability of sample selection is calculated to draw a batch of samples from the empirical sample pool. Calculate the loss of each sample relative to its label in the security detection model, and then use the average loss of the batch of samples to optimize the security detection model.

[0035] In this embodiment, the probability of a sample being selected is calculated based on the comprehensive safety factor of each sample in the empirical sample pool: In the formula, The probability of being selected from the sample. The number of samples in the empirical sample pool. This represents the overall safety factor of the sample.

[0036] In this embodiment, the comprehensive security coefficient is used as the sample security assessment standard to calculate the sample selection probability, making it easier for relatively insecure samples to be selected. The loss of the selected relatively insecure samples in the security detection model and the sample label is used to calculate the average loss of the batch of samples and optimize the security detection model. This enables the model to automatically close the loop and update hot online, improves the dynamic perception of new attacks, and reduces the false positive rate.

[0037] In some implementations, the clustering model is optimized using samples from the empirical sample pool, including B1-B4: B1: Obtain false positive samples from the experience sample pool, and determine the influence weight of each false positive sample using the comprehensive safety factor and the original radius of each cluster in the current clustering model.

[0038] Specifically, the influence weight of each false positive sample is: In the formula, This represents the overall safety factor corresponding to the false alarm samples in this cluster. σ is a scale parameter, which is half the original radius, representing the distance from the false alarm sample to the cluster center.

[0039] It is understandable that the false positive samples of this cluster are those that have been assigned to this cluster but do not actually belong to it. The closer a false positive sample is to the center, the greater its weight.

[0040] B2, based on the false alarm samples and the total number of samples in each cluster, calculates the proposed new radius for each false alarm sample and then calculates the weighted average to obtain the proposed radius for each cluster.

[0041] In this step, the proposed new radius for each false positive sample is: In the formula, To prevent false reports of the distance from the sample to the cluster center, It is the number of false alarms per cluster. Let k be the total number of samples in the k-th cluster. To adjust the parameters, the higher the false alarm rate of the cluster, the smaller the new radius should be.

[0042] The weighted average yields the suggested radius for each cluster: In the formula, Recommended radius for each cluster For the false alarm sample of the k-th cluster, The suggested new radius for false positive samples, This represents the influence weight of false positive samples. If a cluster has 0 false positive samples, then the cluster does not need to optimize its radius and remains at its original radius.

[0043] B3. Calculate the elastic radius using the original radius and the suggested radius.

[0044] The elastic radius is: In the formula, Let be the elastic radius of the k-th cluster. Let be the original radius of the k-th cluster. Let be the suggested radius of the k-th cluster. This represents the global contraction strength.

[0045] B4. Define the shrinkage ratio vector, introduce the inter-cluster interaction matrix, and calculate the adjusted shrinkage ratio vector to obtain the final radius.

[0046] Suppose there are K clusters, define the cluster similarity matrix. For example, the reciprocal of the distance between the centers of two clusters is used as the similarity between the two clusters.

[0047] False positives may originate from intrusions from other clusters, so the boundary contraction of other clusters can affect this cluster.

[0048] Define the shrinkage ratio vector : In the formula, The suggested radius of the cluster, The original radius of the cluster. This is an element-wise division method.

[0049] Introducing the inter-cluster interaction matrix: Adjusted shrinkage ratio vector: The final radius is: In the formula, M is the inter-cluster interaction matrix, I is the identity matrix, λ is the interaction strength, S is the inter-cluster similarity matrix, v is the adjusted shrinkage ratio vector, and u is the shrinkage ratio vector. For global contraction strength, This is element-wise multiplication.

[0050] In this embodiment, by considering the number of false alarm samples in each cluster, the distance from the false alarm sample to the cluster center, and combining the modeling of the mutual influence between clusters, the cluster boundary is optimized using false alarm samples, and the shrinkage intensity is adjusted based on the distribution, distance, and mutual influence of the false alarm samples.

[0051] Please refer to Figure 2 This invention provides a network transmission data security detection device for implementing the steps of any method embodiment in the specification. The device includes: Detection unit 201 is used to perform security detection on network transmission data using a pre-trained security detection model and clustering model; The comparison unit 202 is used to compare the detection results based on the security detection model and the clustering model, and output the detection results. The collection unit 203 is used to calculate the comprehensive security factor when the comparison is inconsistent, and to store the network transmission data into the experience sample pool; The optimization unit 204 is used to optimize the security detection model and the clustering model respectively using samples in the experience sample pool.

[0052] It should be noted that the above device embodiments and method embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0053] Embodiments of this application also provide a computer device, please refer to... Figure 3 The computer device includes a processor and a memory, the memory storing at least one instruction, at least one program, code set, or instruction set, the at least one instruction, at least one program, code set, or instruction set being loaded and executed by the processor to implement the network transmission data security detection method provided in the above-described method embodiments.

[0054] Embodiments of this application also provide a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, at least one program, code set, or instruction set is loaded and executed by a processor to implement the network transmission data security detection method provided in the above-described method embodiments.

[0055] Embodiments of this application also provide a computer program product, which includes a computer program. A processor of a computer device reads the computer program from a computer-readable storage medium and executes the computer program, causing the computer device to perform any of the network transmission data security detection methods described in the above embodiments.

[0056] For ease of description, the above devices or apparatuses are described separately according to their functions, divided into various modules or units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0057] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of the embodiments of this application.

[0058] Finally, it should be noted that in this document, relational terms such as first, second, third, and fourth are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0059] The above are merely preferred embodiments of this application. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for securely detecting data transmitted over a network, characterized in that, include: Security detection of network transmission data is performed using pre-trained security detection models and clustering models; Based on the comparison of the detection results of the security detection model and the clustering model, the detection results are output; When the comparison is inconsistent, the comprehensive security factor is calculated, and the network transmission data is stored in the empirical sample pool. The security detection model and the clustering model are optimized using samples from the experience sample pool.

2. The method as described in claim 1, characterized in that, When discrepancies are found, a comprehensive security factor is calculated, and the network transmission data is stored in an empirical sample pool, including: When the comparison is inconsistent, the weight factors of the security detection model and the clustering model are calculated based on the historical cumulative accuracy. Obtain the previous network transmission data in the same IP stream and the detection results of the security detection model and the clustering model, and calculate the historical sample coefficients; Based on the gradient error and probability distribution of the security detection model, the sample spatiotemporal distance of the clustering model, and the transmission risk parameters of the network transmission data, the attenuation kernel is calculated, and the comprehensive security coefficient of the network transmission data is calculated by combining the weight factor and the historical sample coefficient. The network transmission data and its comprehensive security coefficient, along with historical network transmission data in the same IP stream, are stored in an experience sample pool.

3. The method as described in claim 2, characterized in that, The overall safety factor is calculated as follows: in, In the formula, It is the overall security factor of the data transmitted over the network. The gradient error of the security detection model is... Let be the probability distribution vector of the security detection model. The squared L2 norm of the probability distribution vector. The sample spatiotemporal distance matrix representing the clustering model traces, This represents a transmission risk parameter for the data transmitted over the network. For decay kernel, The historical sample coefficients are... The weighting factors of the security detection model are... The weighting factors of the clustering model, To adjust the parameters, The feature vector is composed of the gradient error and probability distribution of the security detection model, the spatiotemporal distance of the samples in the clustering model, and the transmission risk parameters of the network transmission data. These are the weights corresponding to the feature vectors.

4. The method as described in claim 2, characterized in that, The step of obtaining the previous network transmission data in the same IP stream and the detection results of the security detection model and the clustering model, and calculating the historical sample coefficients, includes: Obtain the previous network transmission data in the same IP stream, as well as the detection results of the security detection model and the clustering model; If the detection results of the two models are consistent, set the coefficient of the historical samples to 0; If the detection results of the two models are inconsistent, the comprehensive security coefficient of the previous network transmission data is obtained, and the detection results of the previous network transmission data in the same IP flow are obtained. Until the detection results of the two models in the acquired historical network transmission data are consistent, the average of the comprehensive security coefficients of all continuous historical network transmission data in the same IP flow with inconsistent detection results of the two models is calculated to obtain the historical sample coefficient.

5. The method as described in claim 1, characterized in that, The security detection model is optimized using samples from the empirical sample pool, including: Based on the comprehensive safety coefficient of the samples, the sample selection probability is calculated to draw a batch of samples from the empirical sample pool. Calculate the loss of each sample relative to the sample label in the security detection model, and then optimize the security detection model by calculating the average loss of the batch of samples.

6. The method as described in claim 1, characterized in that, The step of optimizing the clustering model using samples from the empirical sample pool includes: False positive samples are obtained from the experience sample pool, and the influence weight of each false positive sample is determined using the comprehensive safety coefficient and the original radius of each cluster in the current clustering model. Based on the false alarm samples and the total number of samples in each cluster, the proposed new radius for each false alarm sample is calculated, and then a weighted average is taken to obtain the proposed radius for each cluster. Calculate the elastic radius using the original radius and the suggested radius; Define a shrinkage ratio vector, introduce the inter-cluster interaction matrix, and calculate the adjusted shrinkage ratio vector to obtain the final radius.

7. The method as described in claim 6, characterized in that, The elastic radius is calculated using the following formula: in, In the formula, Let be the elastic radius of the k-th cluster. Let be the original radius of the k-th cluster. For global contraction strength, Let be the suggested radius of the k-th cluster. For the false alarm sample of the k-th cluster, The suggested new radius for false positive samples, The influence weight of false positive samples.

8. A network data transmission security detection device, used to implement the steps of the method according to any one of claims 1-7, characterized in that, include: The detection unit is used to perform security detection on network transmission data using pre-trained security detection models and clustering models. The comparison unit is used to compare the detection results based on the security detection model and the clustering model, and output the detection results. The collection unit is used to calculate the comprehensive security factor when the comparison is inconsistent, and to store the network transmission data into the experience sample pool; The optimization unit is used to optimize the security detection model and the clustering model respectively using samples from the experience sample pool.

9. A computer device, characterized in that, The computer device includes a memory and a processor. The memory is used to store computer programs, and the processor is used to execute the computer programs stored in the memory to implement the steps of the method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the steps of the method described in any one of claims 1-7.

11. A computer program product, characterized in that, Includes a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1-7.

Images