Method and apparatus for detecting security of network transmission data

Abstract

The application discloses a kind of network transmission data security detection method and device, belong to data security field.Method includes: using security detection model and clustering model to carry out security detection to network transmission data;Based on the detection result comparison of security detection model and clustering model, output detection result;When comparison is inconsistent, calculate integrated safety coefficient, and store this network transmission data in experience sample pool;Using sample in experience sample pool, model optimization is carried out to security detection model and clustering model respectively.This scheme utilizes supervised security detection model and unsupervised clustering model parallel detection, generates double mode high-precision detection;Through the cross-modal joint analysis of two modal models, the network transmission data of doubtful is stored in experience sample pool, and supervised security detection model and unsupervised clustering model are optimized in parallel using experience sample pool sample, realize the online closed loop hot update of model, improve the dynamic perception of new attack, with low misjudgment rate.

Description

Technical Field

[0001] This invention relates to the field of data security technology, and in particular to a method and apparatus for secure detection of data transmitted over a network. Background Technology

[0002] With the intelligent development of the Industrial Internet, the complexity and concealment of cyber threats have significantly increased. Insecure data may appear during data interaction between clients and servers, and attack methods have gradually evolved from traditional exploitation of known vulnerabilities to sophisticated threats involving complex, multi-stage attack chains.

[0003] Existing industrial network data security protection mainly relies on rule matching, feature signatures, or static model reasoning. While these methods are effective in detecting known threats, they are less effective against unknown threats and advanced threats such as mutated attacks. Static models, once deployed, cannot perceive or adapt to changes in the dynamic industrial environment in real time, leading to delayed responses to new attacks and high false positive and false negative rates. In particular, manual analysis is extremely costly in large-scale deployment scenarios.

[0004] Therefore, there is an urgent need to provide a new method for secure detection of network transmission data. Summary of the Invention

[0005] To address the problem of high false positive and false negative rates in traditional detection methods when facing complex and ever-changing network threats, this invention provides a method and apparatus for security detection of network transmission data.

[0006] On the one hand, a method for securely detecting network transmission data is provided, the method comprising:

[0007] Security detection of network transmission data is performed using pre-trained security detection models and clustering models;

[0008] Based on the comparison of the detection results of the security detection model and the clustering model, the detection results are output;

[0009] When the comparison is inconsistent, the comprehensive security factor is calculated, and the network transmission data is stored in the empirical sample pool.

[0010] The security detection model and the clustering model are optimized using samples from the experience sample pool.

[0011] On the other hand, a network transmission data security detection device based on the steps described in any method embodiment of the specification is provided, the device comprising:

[0012] The detection unit is used to perform security detection on network transmission data using pre-trained security detection models and clustering models.

[0013] The comparison unit is used to compare the detection results based on the security detection model and the clustering model, and output the detection results.

[0014] The collection unit is used to calculate the comprehensive security factor when the comparison is inconsistent, and to store the network transmission data into the experience sample pool;

[0015] The optimization unit is used to optimize the security detection model and the clustering model respectively using samples from the experience sample pool.

[0016] On the other hand, a computer device is provided, the computer device including a memory and a processor, the memory for storing a computer program, and the processor for executing the computer program stored in the memory to implement the steps of the method described above.

[0017] On the other hand, a computer-readable storage medium is provided, wherein a computer program is stored therein, and when the computer program is executed by a processor, it implements the steps of the method described above.

[0018] On the other hand, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the method described above.

[0019] The technical solution provided by this invention can bring at least the following beneficial effects:

[0020] Breaking through the bottlenecks of traditional static thresholding and sample labeling dependence, this method utilizes parallel detection with supervised security detection models and unsupervised clustering models to generate dual-modal high-precision detection. Through cross-modal joint analysis of the two models, questionable network transmission data is stored in an empirical sample pool. The empirical sample pool samples are then used to achieve parallel optimization of the supervised security detection model and the unsupervised clustering model, enabling online closed-loop hot updates of the model. This improves the dynamic perception of new attacks and reduces the false positive rate. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart of a network transmission data security detection method according to an embodiment of the present invention;

[0023] Figure 2 This is a structural diagram of a network data transmission security detection device according to an embodiment of the present invention;

[0024] Figure 3 This is a hardware architecture diagram of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0026] The following describes the specific implementation of the above concept.

[0027] Please refer to Figure 1 This invention provides a method for secure detection of network transmission data, the method comprising:

[0028] Step 100: Perform security checks on network transmission data using pre-trained security detection and clustering models;

[0029] Step 102: Compare the detection results based on the security detection model and the clustering model, and output the detection results;

[0030] Step 104: When the comparison is inconsistent, calculate the comprehensive security factor and store the network transmission data into the experience sample pool;

[0031] Step 106: Optimize the security detection model and clustering model using samples from the experience sample pool.

[0032] In this embodiment of the invention, the bottlenecks of traditional static thresholding and sample labeling dependence are overcome. Supervised security detection model and unsupervised clustering model are used for parallel detection to generate dual-modal high-precision detection. Through cross-modal joint analysis of the two modal models, suspicious network transmission data is stored in an empirical sample pool. The samples in the empirical sample pool are used to achieve parallel optimization of the supervised security detection model and the unsupervised clustering model, realizing online closed-loop hot update of the model, improving the dynamic perception of new attacks, and reducing the false positive rate.

[0033] The following description Figure 1 The execution method for each step is shown.

[0034] For steps 100 and 102:

[0035] When the detection results of the security detection model and the clustering model are consistent, the result is output directly without being added to the sample pool. When the detection results of the security detection model and the clustering model are inconsistent, the result of the security detection model is output, the result is labeled as questionable, and the network transmission data is added to the empirical sample pool.

[0036] Regarding step 104:

[0037] In some implementations, step 104 may include the following steps S1-S4:

[0038] S1, when the comparison is inconsistent, calculate the weight factors of the security detection model and the clustering model based on the historical cumulative accuracy.

[0039] It is understandable that by calculating the reliability weights of the results from the two models based on their accuracy, detection accuracy can be improved.

[0040] S2, obtain the previous network transmission data and the detection results of the security detection model and clustering model in the same IP flow, and calculate the historical sample coefficients.

[0041] In this embodiment, step S2 may include:

[0042] Obtain the previous network transmission data and the detection results of the security detection model and clustering model in the same IP stream;

[0043] If the detection results of the two models are consistent, set the coefficient of the historical samples to 0;

[0044] If the detection results of the two models are inconsistent, the comprehensive security coefficient of the previous network transmission data is obtained, and the detection results of the previous network transmission data in the same IP flow are obtained.

[0045] Until the detection results of the two models in the acquired historical network transmission data are consistent, the average of the comprehensive security coefficients of all continuous historical network transmission data in the same IP flow with inconsistent detection results of the two models is calculated to obtain the historical sample coefficient.

[0046] In this embodiment, historical sample coefficients are introduced, which are calculated by comparing the detection results of historical network transmission data in the same IP stream with those of two models.

[0047] The specific formula is as follows:

[0048]

[0049] In the formula, Here, Y represents the historical sample coefficients, and Y represents the previous network transmission data. This is a set of network transmission data for which the detection results of the two models are inconsistent. The comprehensive security coefficient for historical network transmission data with inconsistent detection results between the two models is denoted by n, where n is the number of consecutive network transmission data with inconsistent detection results.

[0050] In this embodiment, the average of the comprehensive security coefficients of historical network transmission data with inconsistent detection results is used as the historical sample coefficient to enhance the detection effect on advanced threats with complex multi-stage attack chains. At the same time, tracing historical samples can improve detection accuracy and reduce randomness.

[0051] S3 calculates the attenuation kernel based on the gradient error and probability distribution of the security detection model, the spatiotemporal distance of the samples in the clustering model, and the transmission risk parameters of the network transmission data. It then calculates the comprehensive security coefficient of the network transmission data by combining the weighting factor and the historical sample coefficient.

[0052] In this embodiment, the overall safety factor is calculated as follows:

[0053]

[0054] in,

[0055]

[0056]

[0057]

[0058] In the formula, It is the overall security factor of the data transmitted over the network. For the gradient error of the safety detection model, Let be the probability distribution vector of the security detection model. The squared L2 norm of the probability distribution vector. The spatiotemporal distance matrix of samples representing the clustering model traces, This represents a transmission risk parameter for the data transmitted over the network. For decay kernel, For historical sample coefficients, As weighting factors for the security detection model, Weighting factors in clustering models To adjust the parameters, The feature vector is composed of the gradient error and probability distribution of the security detection model, the spatiotemporal distance of samples in the clustering model, and the transmission risk parameters of the network's transmitted data. These are the weights corresponding to the feature vectors.

[0059] In this embodiment, The weight factors for the security detection model and the clustering model are calculated based on historical cumulative accuracy. To adjust parameters, used to balance the scale and impact of transmission risk parameters and historical sample coefficients. Gradient error. Taking the logarithm is to compress the scale of the gradient error and prevent it from dominating the entire expression. The gradient error and probability distribution of the security detection model, the spatiotemporal distance of the samples in the clustering model, and the transmission risk parameters of the network transmission data are linearly combined and negatively exponentially taken to form a decay kernel. When the term within the brackets of the decay kernel increases, the overall security coefficient will decrease exponentially.

[0060] A feature column vector is constructed by combining four factors: the gradient error and probability distribution of the security detection model, the spatiotemporal distance of samples in the clustering model, and the transmission risk parameters of the network data. By performing a dot product with the feature vector and weight vector, a weighted linear combination of different safety indicators is achieved, reflecting the relative importance of each indicator in the comprehensive evaluation. Introducing the logarithm of historical sample coefficients provides a temporal context for the current comprehensive safety factor assessment, improving accuracy.

[0061] This embodiment combines logarithmic, exponential, and quadratic forms for nonlinear fusion, which can capture the complex interactions between indicators. At the same time, it processes scalars, vectors, and matrices in multiple dimensions, making full use of different types of security information. Through weighting factors and adjustment parameters, the formula can flexibly adapt to different security needs and network environments, improve the effectiveness of the comprehensive security coefficient assessment, and further improve the optimization effect.

[0062] S4. Store the network transmission data and its comprehensive security coefficient, as well as historical network transmission data in the same IP stream, into the experience sample pool.

[0063] In this step, the network transmission data and the network transmission data with continuously inconsistent detection results in the same IP stream, along with their comprehensive security coefficients, are stored in the experience sample pool to automatically form a set of suspicious samples.

[0064] Regarding step 106:

[0065] In some implementations, the security detection model is optimized using samples from an empirical sample pool, including:

[0066] Based on the comprehensive safety coefficient of the samples, the probability of sample selection is calculated to draw a batch of samples from the empirical sample pool.

[0067] Calculate the loss of each sample relative to its label in the security detection model, and then use the average loss of the batch of samples to optimize the security detection model.

[0068] In this embodiment, the probability of a sample being selected is calculated based on the comprehensive safety factor of each sample in the empirical sample pool:

[0069]

[0070] In the formula, The probability of being selected from the sample. The number of samples in the empirical sample pool. This represents the overall safety factor of the sample.

[0071] In this embodiment, the comprehensive security coefficient is used as the sample security assessment standard to calculate the sample selection probability, making it easier for relatively insecure samples to be selected. The loss of the selected relatively insecure samples in the security detection model and the sample label is used to calculate the average loss of the batch of samples and optimize the security detection model. This enables the model to automatically close the loop and update hot online, improves the dynamic perception of new attacks, and reduces the false positive rate.

[0072] In some implementations, the clustering model is optimized using samples from the empirical sample pool, including B1-B4:

[0073] B1: Obtain false positive samples from the experience sample pool, and determine the influence weight of each false positive sample using the comprehensive safety factor and the original radius of each cluster in the current clustering model.

[0074] Specifically, the influence weight of each false positive sample is:

[0075]

[0076] In the formula, This represents the overall safety factor corresponding to the false alarm samples in this cluster. σ is a scale parameter, which is half the original radius, representing the distance from the false alarm sample to the cluster center.

[0077] It is understandable that the false positive samples of this cluster are those that have been assigned to this cluster but do not actually belong to it. The closer a false positive sample is to the center, the greater its weight.

[0078] B2, based on the false alarm samples and the total number of samples in each cluster, calculates the proposed new radius for each false alarm sample and then calculates the weighted average to obtain the proposed radius for each cluster.

[0079] In this step, the proposed new radius for each false positive sample is:

[0080]

[0081] In the formula, To prevent false reports of the distance from the cluster center, It is the number of false alarms per cluster. Let k be the total number of samples in the k-th cluster. To adjust the parameters, the higher the false alarm rate of the cluster, the smaller the new radius should be.

[0082] The weighted average yields the suggested radius for each cluster:

[0083]

[0084] In the formula, Recommended radius for each cluster For the false alarm sample of the k-th cluster, The suggested new radius for false positive samples, This represents the influence weight of false positive samples. If a cluster has 0 false positive samples, then the cluster does not need to optimize its radius and remains at its original radius.

[0085] B3. Calculate the elastic radius using the original radius and the suggested radius.

[0086] The elastic radius is:

[0087]

[0088] In the formula, Let be the elastic radius of the k-th cluster. Let be the original radius of the k-th cluster. Let be the suggested radius of the k-th cluster. This represents the global contraction strength.

[0089] B4. Define the shrinkage ratio vector, introduce the inter-cluster interaction matrix, and calculate the adjusted shrinkage ratio vector to obtain the final radius.

[0090] Suppose there are K clusters, define the cluster similarity matrix. For example, the reciprocal of the distance between the centers of two clusters is used as the similarity between the two clusters.

[0091] False positives may originate from intrusions from other clusters, so the boundary contraction of other clusters can affect this cluster.

[0092] Define the shrinkage ratio vector :

[0093]

[0094] In the formula, The suggested radius of the cluster, The original radius of the cluster, This is an element-wise division method.

[0095] Introducing the inter-cluster interaction matrix:

[0096] M=I+λS

[0097] Adjusted shrinkage ratio vector:

[0098]

[0099] The final radius is:

[0100]

[0101] In the formula, M is the inter-cluster interaction matrix, I is the identity matrix, λ is the interaction strength, S is the inter-cluster similarity matrix, v is the adjusted shrinkage ratio vector, and u is the shrinkage ratio vector. For global contraction strength, This is element-wise multiplication.

[0102] In this embodiment, by considering the number of false alarm samples in each cluster, the distance from the false alarm sample to the cluster center, and combining the modeling of the mutual influence between clusters, the cluster boundary is optimized using false alarm samples, and the shrinkage intensity is adjusted based on the distribution, distance, and mutual influence of the false alarm samples.

[0103] Please refer to Figure 2 This invention provides a network transmission data security detection device for implementing the steps of any method embodiment in the specification. The device includes:

[0104] Detection unit 201 is used to perform security detection on network transmission data using a pre-trained security detection model and clustering model;

[0105] The comparison unit 202 is used to compare the detection results based on the security detection model and the clustering model, and output the detection results.

[0106] The collection unit 203 is used to calculate the comprehensive security factor when the comparison is inconsistent, and to store the network transmission data into the experience sample pool;

[0107] The optimization unit 204 is used to optimize the security detection model and the clustering model respectively using samples in the experience sample pool.

[0108] It should be noted that the above device embodiments and method embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0109] Embodiments of this application also provide a computer device, please refer to... Figure 3 The computer device includes a processor and a memory, the memory storing at least one instruction, at least one program, code set, or instruction set, the at least one instruction, at least one program, code set, or instruction set being loaded and executed by the processor to implement the network transmission data security detection method provided in the above-described method embodiments.

[0110] Embodiments of this application also provide a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, at least one program, code set, or instruction set is loaded and executed by a processor to implement the network transmission data security detection method provided in the above-described method embodiments.

[0111] Embodiments of this application also provide a computer program product, which includes a computer program. A processor of a computer device reads the computer program from a computer-readable storage medium and executes the computer program, causing the computer device to perform any of the network transmission data security detection methods described in the above embodiments.

[0112] For ease of description, the above devices or apparatuses are described separately according to their functions, divided into various modules or units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0113] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of the embodiments of this application.

[0114] Finally, it should be noted that in this document, relational terms such as first, second, third, and fourth are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0115] The above are merely preferred embodiments of this application. It should be noted that those skilled in the art can make several improvements and modifications without departing from the principles of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A method for detecting security of network transmission data, characterized by, include: Security detection of network transmission data is performed using pre-trained security detection models and clustering models; Based on the comparison of the detection results of the security detection model and the clustering model, the detection results are output; When the comparison is inconsistent, the comprehensive security factor is calculated, and the network transmission data is stored in the empirical sample pool. The security detection model and the clustering model are optimized using samples from the empirical sample pool, respectively. When discrepancies are found, a comprehensive security factor is calculated, and the network transmission data is stored in an empirical sample pool, including: When the comparison is inconsistent, the weight factors of the security detection model and the clustering model are calculated based on the historical cumulative accuracy. Obtain the previous network transmission data in the same IP stream and the detection results of the security detection model and the clustering model, and calculate the historical sample coefficients; Based on the gradient error and probability distribution of the security detection model, the spatiotemporal distance of the samples in the clustering model, and the transmission risk parameters of the network transmission data, the attenuation kernel is calculated, and the comprehensive security coefficient of the network transmission data is calculated by combining the weight factor and the historical sample coefficient. The network transmission data and its comprehensive security coefficient, along with historical network transmission data in the same IP stream, are stored in an experience sample pool.

2. The method of claim 1, wherein, The overall safety factor is calculated as follows: in, In the formula, It is the overall security factor of the data transmitted over the network. The gradient error of the security detection model is... Let be the probability distribution vector of the security detection model. The squared L2 norm of the probability distribution vector. The sample spatiotemporal distance matrix representing the clustering model traces, This represents a transmission risk parameter for the data transmitted over the network. For decay kernel, The historical sample coefficients are... The weighting factors of the security detection model are... The weighting factors of the clustering model, To adjust the parameters, The feature vector is composed of the gradient error and probability distribution of the security detection model, the spatiotemporal distance of the samples in the clustering model, and the transmission risk parameters of the network transmission data. These are the weights corresponding to the feature vectors.

3. The method of claim 1, wherein, The step of obtaining the previous network transmission data in the same IP stream and the detection results of the security detection model and the clustering model, and calculating the historical sample coefficients, includes: Obtain the previous network transmission data in the same IP stream, as well as the detection results of the security detection model and the clustering model; If the detection results of the two models are consistent, set the coefficient of the historical samples to 0; If the detection results of the two models are inconsistent, the comprehensive security coefficient of the previous network transmission data is obtained, and the detection results of the previous network transmission data in the same IP flow are obtained. Until the detection results of the two models in the acquired historical network transmission data are consistent, the average of the comprehensive security coefficients of all continuous historical network transmission data in the same IP flow with inconsistent detection results of the two models is calculated to obtain the historical sample coefficient.

4. The method as described in claim 1, characterized in that, The security detection model is optimized using samples from the empirical sample pool, including: Based on the comprehensive safety coefficient of the samples, the sample selection probability is calculated to draw a batch of samples from the empirical sample pool. Calculate the loss of each sample relative to the sample label in the security detection model, and then optimize the security detection model by calculating the average loss of the batch of samples.

5. The method as described in claim 1, characterized in that, The step of optimizing the clustering model using samples from the empirical sample pool includes: False positive samples are obtained from the experience sample pool, and the influence weight of each false positive sample is determined using the comprehensive safety coefficient and the original radius of each cluster in the current clustering model. Based on the false alarm samples and the total number of samples in each cluster, the proposed new radius for each false alarm sample is calculated, and then a weighted average is taken to obtain the proposed radius for each cluster. Calculate the elastic radius using the original radius and the suggested radius; Define a shrinkage ratio vector, introduce the inter-cluster interaction matrix, and calculate the adjusted shrinkage ratio vector to obtain the final radius.

6. The method of claim 5, wherein, The elastic radius is calculated using the following formula: in, In the formula, Let be the elastic radius of the k-th cluster. Let be the original radius of the k-th cluster. For global contraction strength, Let be the suggested radius of the k-th cluster. For the false alarm sample of the k-th cluster, The suggested new radius for false positive samples, The influence weight of false positive samples.

7. A network transmission data security detection apparatus for implementing the steps of the method according to any one of claims 1 to 6, characterized in that, include: The detection unit is used to perform security detection on network transmission data using pre-trained security detection models and clustering models. The comparison unit is used to compare the detection results based on the security detection model and the clustering model, and output the detection results. The collection unit is used to calculate the comprehensive security factor when the comparison is inconsistent, and to store the network transmission data into the experience sample pool; The optimization unit is used to optimize the security detection model and the clustering model respectively using samples from the experience sample pool.

8. A computer device, comprising: The computer device includes a memory and a processor. The memory is used to store computer programs, and the processor is used to execute the computer programs stored in the memory to implement the steps of the method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the steps of the method described in any one of claims 1-6.

10. A computer program product, characterised in that, Includes a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1-6.

Images