Data security management method and device, equipment, storage medium and program product

Abstract

The application provides a data security management and control method, device, equipment, storage medium and program product. The method comprises the following steps: obtaining network flow data to be detected in a target detection scene and a training sample library of the target detection scene, wherein the training sample library comprises sample feature data and sample attack types of a plurality of training samples; preprocessing the network flow data to obtain first feature data of the network flow data; inputting the first feature data into a pre-trained first model to obtain second feature data output by the first model, wherein the first model is used for extracting attack features in the first feature data through a convolution operation to generate the second feature data; calculating a first Euclidean distance between the second feature data and the sample feature data of the training sample; performing type matching on the sample attack types of the training sample and the first Euclidean distance to determine a target attack type of the network flow data. The application has high detection accuracy for network attacks.

Description

Technical Field

[0001] This invention relates to the field of data security technology, and in particular to a data security management method, apparatus, equipment, storage medium, and program product. Background Technology

[0002] In the context of today's increasingly severe cybersecurity situation, intrusion detection systems (IDS) are widely used as a key technology for ensuring information security. However, existing intrusion detection technologies have many problems. For example, when an IDS uses network monitoring mode, it may collect a large amount of network traffic data for analysis. If the system has security vulnerabilities or is improperly configured during the storage, transmission, or processing of this data, it may be exploited by attackers, leading to the theft and leakage of sensitive data.

[0003] Existing intrusion detection systems typically employ multiple detection methods, including signature-based detection, anomaly detection, stateful detection, and protocol analysis. However, each method has inherent limitations. For example, anomaly detection often relies on statistical methods; setting the threshold too low can lead to numerous false positives, while setting it too high can result in numerous false negatives, potentially causing data leaks. Similarly, signature-based detection primarily depends on a database of known intrusion behavior signatures. However, with the continuous emergence of new attack methods, this database cannot be updated in a timely manner, leading to the inability to detect unknown and complex intrusion behaviors. Furthermore, while protocol analysis methods can deeply analyze network protocol behavior, they are extremely difficult to analyze when dealing with complex protocols and encrypted communications, easily resulting in false positives or false negatives, further exacerbating the risk of data leaks. Summary of the Invention

[0004] The purpose of this invention is to provide a data security management method, apparatus, device, storage medium, and program product to solve the problem that the detection accuracy of network attacks in the prior art is low, leading to the risk of data leakage.

[0005] To address the aforementioned technical problems, embodiments of the present invention provide a data security management method, comprising:

[0006] Obtain network traffic data to be detected in the target detection scenario and a training sample library for the target detection scenario, wherein the training sample library includes sample feature data and sample attack types of multiple training samples;

[0007] The network traffic data is preprocessed to obtain the first feature data of the network traffic data;

[0008] The first feature data is input into a pre-trained first model to obtain the second feature data output by the first model. The first model is trained based on the training samples in the target detection scenario. The first model is used to extract attack features from the first feature data through convolution operations to generate the second feature data.

[0009] Calculate the first Euclidean distance between the second feature data and the sample feature data of the training samples;

[0010] The target attack type of the network traffic data is determined by matching the sample attack type of the training samples with the first Euclidean distance.

[0011] Optionally, the preprocessing of the network traffic data to obtain the first feature data of the network traffic data includes:

[0012] The network traffic data is parsed and statistically analyzed to obtain the first feature vector of the network traffic data;

[0013] The first feature vector is normalized and encoded to obtain the second feature vector;

[0014] The second feature vector is subjected to dimensionality reduction processing to obtain the third feature vector;

[0015] The third feature vector is converted into a grayscale image to obtain the first feature data.

[0016] Optionally, calculating the first Euclidean distance between the second feature data and the sample feature data of the training samples includes:

[0017] Obtain the first number of training samples for each of the aforementioned attack types in the training sample library, and obtain the total number of training samples in the training sample library;

[0018] Based on the first quantity and the total quantity, the type weight value of the training sample is determined, wherein a training sample of one type of attack corresponds to one type weight value, and the size of the first quantity is inversely proportional to the size of the type weight value;

[0019] Based on the type weight value and the pre-set dimension weight value, a first Euclidean distance is calculated between the second feature data and each of the sample feature data, wherein the dimension weight value is used to represent the importance of each feature dimension in the second feature data and the sample feature data.

[0020] Optionally, the method further includes:

[0021] For each feature dimension, calculate the total difference between the second feature data and the multiple sample feature data;

[0022] The total difference is subjected to power-law decay and normalization to obtain the dimension weight value of each dimension feature.

[0023] Optionally, determining the target attack type of the network traffic data by performing type matching based on the sample attack type of the training samples and the first Euclidean distance includes:

[0024] Sort the first Euclidean distances in ascending order to generate the first sequence;

[0025] The first K training samples corresponding to the first Euclidean distance in the first sequence are selected to form the nearest neighbor set of the network traffic data, where K is an integer greater than 0;

[0026] The target attack type of the network traffic data is determined based on the sample attack type of the training samples in the nearest neighbor set.

[0027] Optionally, the method further includes:

[0028] Obtain the original training dataset, which includes multiple first traffic data and a first attack type of the first traffic data;

[0029] The data in the original training dataset is preprocessed to obtain the first training dataset;

[0030] The first convolutional neural network model is trained based on the first training dataset to generate the second model;

[0031] Based on the training samples in the target detection scenario and the second model, the second convolutional neural network model is trained by knowledge transfer to obtain the first model.

[0032] Optionally, the step of performing knowledge transfer training on the second convolutional neural network model based on the training samples in the target detection scene and the second model to obtain the first model includes:

[0033] The second convolutional neural network model is initialized based on the network parameters of the second model;

[0034] Based on the training samples in the target detection scenario, the initialized second convolutional neural network model is iteratively trained using connection pruning and conjugate gradient algorithms to obtain the first model. In each iteration of the model training process, the connection pruning technique is used to prune the network structure of the second convolutional neural network model, and the conjugate gradient algorithm is used to determine the training direction of the model based on the pruned network structure.

[0035] Optionally, the method further includes:

[0036] The training samples are input into the first model to obtain the sample feature data output by the first model. The model is used to extract attack features from the training samples through convolution and connection pruning operations to generate the sample feature data.

[0037] This invention also provides a data security management and control device, comprising:

[0038] The first acquisition module is used to acquire network traffic data to be detected in the target detection scenario and a training sample library of the target detection scenario, wherein the training sample library includes sample feature data and sample attack types of multiple training samples.

[0039] The first processing module is used to preprocess the network traffic data to obtain the first feature data of the network traffic data;

[0040] The second processing module is used to input the first feature data into a pre-trained first model to obtain the second feature data output by the first model. The first model is trained based on the training samples in the target detection scenario. The first model is used to extract attack features from the first feature data through convolution operations to generate the second feature data.

[0041] The first calculation module is used to calculate the first Euclidean distance between the second feature data and the sample feature data of the training sample;

[0042] The first matching module is used to match the sample attack type of the training sample with the first Euclidean distance to determine the target attack type of the network traffic data.

[0043] This invention also provides a network device, including: a processor, a memory, and a program stored in the memory and executable on the processor, wherein the program, when executed by the processor, implements the data security management method as described in any of the preceding embodiments.

[0044] This invention also provides a readable storage medium, comprising: a program stored on the readable storage medium, wherein when the program is executed by a processor, it implements the steps of the data security management method as described in any of the preceding claims.

[0045] This invention also provides a computer program product, including computer instructions, which, when executed by a processor, implement the steps of the data security management method as described in any of the preceding embodiments.

[0046] At least one of the above technical solutions of the present invention has the following beneficial effects:

[0047] The above scheme provides a data security management method. First, it acquires network traffic data to be detected in a target detection scenario and a training sample library for the target detection scenario. The training sample library includes sample feature data and sample attack types of multiple training samples. Then, it preprocesses the network traffic data to obtain first feature data. Next, it inputs the first feature data into a pre-trained first model to obtain second feature data output by the first model. The first model is trained based on training samples in the target detection scenario and is used to extract attack features from the first feature data through convolution operations to generate the second feature data. It also calculates a first Euclidean distance between the second feature data and the sample feature data of the training samples. Finally, it performs type matching between the sample attack types of the training samples and the first Euclidean distance to determine the target attack type of the network traffic data. The first model in the above scheme is trained based on the target detection scenario, which can significantly improve the model's generalization ability and convergence speed, thereby improving the model's performance. Therefore, the attack features in the second feature data extracted by the first model are more accurate. Finally, based on the first Euclidean distance between the second feature data and the sample feature data of the training samples, the target attack type of the network traffic data is determined, which can effectively improve the detection accuracy of network attacks and avoid the risk of data leakage. Attached Figure Description

[0048] Figure 1 This is a flowchart illustrating the data security management method according to an embodiment of the present invention;

[0049] Figure 2 This is a schematic diagram of the data security management device according to an embodiment of the present invention. Detailed Implementation

[0050] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0051] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0052] like Figure 1 As shown, this embodiment of the invention provides a data security management method, including:

[0053] Step S101: Obtain network traffic data to be detected in the target detection scenario and a training sample library for the target detection scenario, wherein the training sample library includes sample feature data and sample attack types of multiple training samples.

[0054] In step S101, network traffic devices are deployed to capture data packets passing through the network interface in real time, i.e., network traffic data. The network traffic data includes all relevant network behavior information, such as source address, destination address, port number, protocol type, service type, packet length, and duration.

[0055] Step S102: Preprocess the network traffic data to obtain the first feature data of the network traffic data;

[0056] Step S103: Input the first feature data into the pre-trained first model to obtain the second feature data output by the first model. The first model is trained based on the training samples in the target detection scenario. The first model is used to extract the attack features in the first feature data through convolution operations to generate the second feature data.

[0057] Step S104: Calculate the first Euclidean distance between the second feature data and the sample feature data of the training samples;

[0058] Step S105: Match the sample attack type of the training sample with the first Euclidean distance to determine the target attack type of the network traffic data.

[0059] In this embodiment of the invention, the target detection scenario in step S101 can be a common general scenario, a special scenario (e.g., a small sample environment), or a scenario corresponding to a specific task (e.g., specific heterogeneous data or novel attack detection). The training samples used to train the first model are constructed based on the target detection scenario, and their sample attack types are pre-labeled. The first model trained using these training samples can significantly improve the model's generalization ability and convergence speed, thereby improving model performance. The sample feature data is obtained by extracting the attack features of the training samples. In one optional implementation, the attack features of the training samples are extracted using the first model to obtain the sample feature data.

[0060] The aforementioned data security management method first preprocesses the network traffic data to be processed to obtain first feature data. Then, it uses a first model to extract attack features from the first feature data to obtain second feature data. Since the first model is trained based on the target detection scenario, the extracted attack features, i.e., the second feature data, will be more accurate. Finally, based on the first Euclidean distance between the second feature data and the sample feature data of the training samples, the target attack type of the network traffic data is determined. The above method can effectively improve the detection accuracy of network attacks and avoid the risk of data leakage.

[0061] In one embodiment, optionally, the preprocessing of the network traffic data to obtain first feature data of the network traffic data includes:

[0062] The network traffic data is parsed and statistically analyzed to obtain the first feature vector of the network traffic data;

[0063] The first feature vector is normalized and encoded to obtain the second feature vector;

[0064] The second feature vector is subjected to dimensionality reduction processing to obtain the third feature vector;

[0065] The third feature vector is converted into a grayscale image to obtain the first feature data.

[0066] In this embodiment of the invention, the data preprocessing method in step S102 is described. After obtaining the original network traffic data, the network traffic data is first parsed and statistically analyzed to obtain a first feature vector. The specific operations include: on the one hand, directly parsing and extracting basic network behavior information from the network traffic data, such as connection duration, protocol type, and number of bytes transmitted; on the other hand, statistically analyzing the network traffic data by setting a time window (e.g., the past 2 seconds) or connection count (e.g., the first 100 connections) to calculate complex statistical features such as "the proportion of connections with the same service" and "the error rate of Synchronize Sequence Numbers (SYN)". Finally, these features are combined to form a complete first feature vector describing the network connection behavior. The first feature vector is composed of features of multiple dimensions, and its data types include, but are not limited to, symbolic, binary, and numerical types.

[0067] Then, the first feature vector is normalized and encoded sequentially to obtain the second feature vector. The first feature vector is then expanded into an M-dimensional second feature vector. The encoding methods include, but are not limited to, Boolean expansion, binary encoding, and one-hot encoding. Specifically, for symbolic features, one-hot encoding is performed; for binary features, Boolean expansion is performed; and for numerical features, normalization and binary encoding are performed.

[0068] Secondly, the second feature vector is subjected to dimensionality reduction processing to obtain the third feature vector. Specifically, a pre-set target dimension information is obtained, and the second feature vector is subjected to dimensionality reduction processing based on this dimension information to obtain a K-dimensional third feature vector. The target dimension information is determined based on the variance coefficient of the training samples during preprocessing.

[0069] Finally, the third feature vector is transformed into grayscale Figure 2 The first feature data is obtained by filling the missing dimensional matrix with 0s. The first feature data is then input into the first model. This method retains only the dimension with the most dispersed feature distribution and removes redundant or low-information features. This reduces both the dimensionality and the risk of overfitting. It also eliminates the need for an additional dimensionality reduction model, has low computational cost, and is easy to parallelize.

[0070] In one embodiment, optionally, calculating the first Euclidean distance between the second feature data and the sample feature data of the training samples includes:

[0071] Obtain the first number of training samples for each of the aforementioned attack types in the training sample library, and obtain the total number of training samples in the training sample library;

[0072] Based on the first quantity and the total quantity, the type weight value of the training sample is determined, wherein a training sample of one type of attack corresponds to one type weight value, and the size of the first quantity is inversely proportional to the size of the type weight value;

[0073] Based on the type weight value and the pre-set dimension weight value, a first Euclidean distance is calculated between the second feature data and each of the sample feature data, wherein the dimension weight value is used to represent the importance of each feature dimension in the second feature data and the sample feature data.

[0074] In this embodiment of the invention, step S104 is described. This embodiment of the invention provides a new method for calculating weighted Euclidean distance, which introduces type weight values ​​and dimension weight values ​​in the weighted Euclidean distance calculation. The specific method is as follows:

[0075] First, weights are assigned based on the number of training samples of each type, and the type weight value of the training samples for each type of attack is calculated as follows:

[0076]

[0077] in, Indicates the type of sample attack. The first number of training samples, This represents the total number of training samples. Indicates the type of sample attack. The type weights of the training samples are used to amplify the influence of the minority class.

[0078] Then, based on the type weight value and pre-set dimension weight values Calculate the first Euclidean distance between the second feature data and each sample feature data, using the following formula:

[0079]

[0080] in, This represents the i-th second feature data. This represents the sample feature data of the j-th training sample. express and The first Euclidean distance between them This indicates the sample attack type corresponding to the j-th training sample. Type weight value, Represents the i-th second feature data. 3D feature data, Indicates the first The first of the sample feature data 3D feature data, This represents the feature dimension of the second feature data and the sample feature data. This represents the d-th feature dimension and its weight value (which can be considered as the importance of the feature, highlighting important features or amplifying the influence of a certain feature dimension; it can be predefined or set empirically).

[0081] In addition, to ensure the validity of the weights, a type weight value was introduced in the above calculation process. and dimension weight values The normalization constraints are as follows:

[0082]

[0083]

[0084] Where C represents the sample attack type The total number, Indicates the type of sample attack. The type weight values ​​of the training samples; This represents the total number of feature dimensions. This represents the dimension weight value of the d-th feature dimension.

[0085] The above method introduces type weight values. and dimension weight values It significantly improves robustness to class imbalance and heterogeneous feature data, giving greater attention to minority class samples during distance calculation, effectively alleviating the bias problem of the model towards the majority class under imbalanced sample conditions. At the same time, the introduction of dimensional weight values ​​can automatically adjust the importance of different features in the discrimination process, thereby improving the classification accuracy and generalization ability. It maintains a high recognition rate, especially when detecting low-frequency attack types. It overcomes the performance degradation of traditional methods in high-dimensional sparse data or environments with large intra-class variance differences, thus having better adaptability and practical application value.

[0086] In one embodiment, optionally, the method further includes:

[0087] For each feature dimension, calculate the total difference between the second feature data and the multiple sample feature data;

[0088] The total difference is subjected to power-law decay and normalization to obtain the dimension weight value of each dimension feature.

[0089] In this embodiment of the invention, a method for calculating dimension weight values ​​is provided. For each feature dimension, the total difference between the second feature data and multiple sample feature data is calculated. This total difference reflects the distinguishing power of the dimension on the current sample. Then, the total difference is power-decimated and its reciprocal is taken, so that dimensions with smaller differences (higher similarity) receive larger initial weights. Finally, the reciprocal results of all dimensions are normalized so that the sum of the weights of all dimensions is 1, thereby obtaining the dimension weight value. The calculation formula is as follows:

[0090]

[0091] in, This represents the dimensional weight value of the d-th feature dimension. p represents the total number of training samples, and p represents the total number of feature dimensions. This represents the attenuation coefficient, used for adjustment. and The degree to which the distance between them affects the weights This represents a small constant, used to prevent division by zero. Represents the i-th second feature data. 3D feature data, Indicates the first The first of the sample feature data 3D feature data, Represents the i-th second feature data. 3D feature data, Indicates the first The first of the sample feature data Dimensional feature data.

[0092] In one implementation, optionally, the step of determining the target attack type of the network traffic data by performing type matching based on the sample attack type of the training samples and the first Euclidean distance includes:

[0093] Sort the first Euclidean distances in ascending order to generate the first sequence;

[0094] The first K training samples corresponding to the first Euclidean distance in the first sequence are selected to form the nearest neighbor set of the network traffic data, where K is an integer greater than 0;

[0095] The target attack type of the network traffic data is determined based on the sample attack type of the training samples in the nearest neighbor set.

[0096] In this embodiment of the invention, based on the K-Nearest Neighbor (KNN) algorithm and the first Euclidean distance, the K sample feature data most similar to the second feature data are selected. Then, based on the sample attack types of the training samples corresponding to the K sample feature data, the target attack type of the network traffic data is determined. The specific method is as follows:

[0097] First, sort the first Euclidean distances in ascending order to generate the first sequence;

[0098] Then, select the first K training samples corresponding to the first Euclidean distance in the first sequence, and form the nearest neighbor set of the network traffic data to be processed from these training samples;

[0099] Finally, a weighted vote is calculated based on the attack types of the training samples in the nearest neighbor set to obtain multiple attack types. Cumulative voting weight will determine the attack type with the highest cumulative voting weight. As a type of attack targeting network traffic data, the attack type The formula for calculating cumulative voting weight is as follows;

[0100]

[0101] in, This represents the j-th training sample in the nearest neighbor set. The tag, express The corresponding sample attack type is indicated. The voting weight of the j-th training sample in the nearest neighbor set for the sample attack type c, specifically, It can be determined based on the inverse transformation of the first function, or by type weight values. Sure.

[0102] The above method combines the ease of implementation of KNN with the weighted distance mechanism proposed in this invention. By comprehensively considering the distribution of attack types and the importance of features, the final determined target attack types are more robust, especially in scenarios with imbalanced training samples or complex feature distributions, where it has stronger classification ability and anti-interference ability.

[0103] In one embodiment, optionally, the method further includes:

[0104] Obtain the original training dataset, which includes multiple first traffic data and a first attack type of the first traffic data;

[0105] The data in the original training dataset is preprocessed to obtain the first training dataset;

[0106] The first convolutional neural network model is trained based on the first training dataset to generate the second model;

[0107] Based on the training samples in the target detection scenario and the second model, the second convolutional neural network model is trained by knowledge transfer to obtain the first model.

[0108] In this embodiment of the invention, two 1D-CNNs, namely a first model and a second model, are constructed and trained. The second model is trained on a general training set, and its core function is to learn the common low-level feature representations in network traffic and establish basic discrimination capabilities. The first model is obtained by knowledge transfer training based on the second model according to the target detection scenario, which can improve the generalization ability and training efficiency of the second model under small sample and heterogeneous data. The specific construction and training method of the second model is as follows:

[0109] First, the original training dataset, i.e., the general training set, is obtained. The original training dataset includes multiple sets of first traffic data and the first attack types of the first traffic data. The first traffic data consists of data packets captured by network traffic devices passing through network interfaces, including all relevant network behavior information, such as source address, destination address, port number, protocol type, service type, packet length, and duration. The first attack types are pre-labeled, and the first attack types of the first traffic data in the original training dataset cover a variety of basic attack types.

[0110] Then, the data in the original training dataset is preprocessed to obtain the first training dataset. The preprocessing method is the same as that for network traffic data, including: parsing and statistically analyzing the data in the original training dataset to obtain the fourth feature vector; normalizing and encoding the fourth feature vector to obtain the fifth feature vector; reducing the dimension of the fifth feature vector to obtain the sixth feature vector; and converting the sixth feature vector into a two-dimensional grayscale image to obtain the first training dataset.

[0111] Finally, the first training dataset is input into the first convolutional neural network model for training to obtain the second model. The specific training steps are as follows:

[0112] First, initialize the network parameters of the first convolutional neural network model, including the convolutional kernel weights and bias terms, usually using random initialization methods such as Glorot initialization or He initialization;

[0113] Then, the forward propagation algorithm is used to calculate the outputs of the convolutional and pooling layers of the first convolutional neural network model. The convolutional layers apply convolutional kernels to the input data through a sliding window mechanism to extract local features, while the pooling layers reduce the data dimensionality and retain important features through downsampling. Max pooling or average pooling methods are typically used. The calculation process of the convolutional and pooling layers is as follows:

[0114]

[0115] In the first formula Indicates the first The first in the layer The output of each neuron These are the features of the previous layer after pooling. Convolution kernel weights (corresponding to the first) The input channel and the first (convolution kernel between each output channel) For bias terms, It is an activation function, such as the Corrected Linear Unit (ReLU), used to introduce nonlinearity; in the second formula These are the feature values ​​after pooling operations. This is the output of the convolution from the previous layer. For pooling downsampling operations, the upper layer With the spatial size reduced, there are two main types of pooling methods: max pooling and average pooling. This scheme uses the max pooling method. This is the scaling factor on that channel, which can be used for feature normalization. The bias term after pooling is g(⋅), which is a linear function or identity function (no nonlinearity is introduced after pooling).

[0116] The first training sample in the first training dataset of the time series is processed using one-dimensional convolution. This treats the data of the first training sample as an "image" for spatial feature extraction. The convolutional layer is responsible for extracting local features, while the pooling layer performs downsampling to reduce feature dimensionality and reduce overfitting. This step no longer relies on traditional manual feature design. By combining pooling operations and nonlinear transformations, it improves the ability to detect abnormal traffic. The forward propagation algorithm is used to calculate the outputs of the convolutional and pooling layers. The convolutional layer applies a convolutional kernel to the input data through a sliding window mechanism to extract local features, while the pooling layer reduces data dimensionality and retains important features through downsampling, typically using max pooling or average pooling methods.

[0117] Furthermore, the first convolutional neural network model is trained based on the backpropagation algorithm, and the model prediction error is minimized based on the loss function to update the weights and biases. The specific implementation is as follows:

[0118] First, the loss function is constructed using the cross-entropy loss function with regularization, as follows:

[0119]

[0120] in, This represents the first training sample. This indicates the true label of the first training sample, which is the first attack type. Represents the regularization coefficient. Indicates the number of samples. It is a general representation of network parameters; in convolutional layers, it specifically refers to the convolution kernel weights in the forward propagation formula. , The model number is represented by the first... The layer output uniformly represents the output of the first layer in the neural network. The final output (activation value) of a layer, whether it is a convolutional layer (outputting a feature map after convolution and activation functions) or a pooling layer (outputting a feature map after downsampling), as long as it is a layer in the network structure, the output data that has been calculated and is ready to be passed to the next layer is represented by this symbol.

[0121] Then, gradient descent is used to apply the first... Layer weights and bias The updated formula is as follows:

[0122]

[0123]

[0124] in, This is the learning rate.

[0125] Furthermore, in the gradient calculation and update process, the gradient propagation process starts from the output layer to calculate the error term. And propagate backward layer by layer, where the error term for the output layer neurons... The calculation formula is as follows:

[0126]

[0127] in, Indicates the Lth layer. The weighted input of each neuron, It is the derivative of the activation function. Indicates the first Layer output, The learning rate;

[0128] For the error term of the hidden layer neurons The calculation formula is as follows:

[0129]

[0130] in, Indicates the first The weighted input of the j-th neuron in the layer, Indicates the first +1 floor Error term of each neuron, This indicates the distance from the j-th neuron in the current layer to the j-th neuron. +1 floor The weights of each neuron, It is the derivative of the activation function.

[0131] Furthermore, taking convolutional layers as an example, the parameter update formula is expanded to show that the convolutional kernel update is achieved by accumulating the gradient of the local receptive field, as follows:

[0132]

[0133]

[0134] in, This represents the input data received by the current convolutional layer, which is the data from the previous layer (the first convolutional layer). The output feature map of a layer (the first layer of the network) is equivalent to the original input only if that layer is the first layer of the network. This indicates a convolution operation.

[0135] Furthermore, L2 norm (L2-Norm, L2) regularization directly affects the gradient term. The regularization update step is used to accelerate convergence and prevent overfitting, as shown in the following formula:

[0136]

[0137] After the backpropagation is completed, the process repeats with the first training sample of the next batch until the convergence condition is met or the set number of rounds is reached.

[0138] The backpropagation implementation of the above model training process combines the backpropagation process of the standard convolutional neural network and integrates a regularization mechanism to achieve efficient optimization of the convolutional kernel parameters in the network structure.

[0139] It should be noted that the first training dataset is in the form of two-dimensional grayscale images. The core logic of training the first convolutional neural network model is to treat the network traffic feature vector as an "image" to assist in the organization of spatial features. However, in actual operation, the characteristics of the first convolutional neural network model in processing sequential data are utilized to map the convolution axis to the time series or arrangement order of features, thereby capturing temporal correlations while extracting local spatial features, rather than performing traditional two-dimensional pixel-level convolution.

[0140] In one embodiment, optionally, the step of performing knowledge transfer training on the second convolutional neural network model based on the training samples in the target detection scene and the second model to obtain the first model includes:

[0141] The second convolutional neural network model is initialized based on the network parameters of the second model;

[0142] Based on the training samples in the target detection scenario, the initialized second convolutional neural network model is iteratively trained using connection pruning and conjugate gradient algorithms to obtain the first model. In each iteration of the model training process, the connection pruning technique is used to prune the network structure of the second convolutional neural network model, and the conjugate gradient algorithm is used to determine the training direction of the model based on the pruned network structure.

[0143] In this embodiment of the invention, the second convolutional neural network model is trained and optimized using connection pruning technology and the conjugate gradient algorithm to generate the first model. The training steps of the first model are described below:

[0144] Step 1: Parameter initialization. Copy the network parameters (including convolutional kernel weights and bias terms) of the trained second model to initialize the second convolutional neural network model. This enables the new model to have basic discrimination capabilities against network attacks, thereby reducing the number of training rounds and accelerating the convergence speed.

[0145] Step 2: Construct the objective function and define the new loss function. ,in It is the vector of parameters to be optimized. It is a cross-entropy loss function that includes a regularization term. It is used to measure the difference between the predicted output and the true label, while controlling the model complexity to avoid overfitting.

[0146] Step 3: Initialize the conjugate gradient algorithm, setting the initial search point as... The initial search direction is Initialize the number of iteration steps ;

[0147] Step 4: Optimize the model through feature extraction, connection pruning, and conjugate gradient methods. Specifically:

[0148] Feature extraction refers to the process of transforming input data into a high-dimensional feature representation using convolutional layers (extracting local features) and pooling layers (downsampling) during model training.

[0149] Connection pruning is a dynamic process implemented in each iteration (or each batch of forward propagation) during training. Specifically, it involves randomly discarding some connections between neurons while extracting features. This strategy is used throughout the entire transfer learning training phase and aims to prevent overfitting by constantly changing the connection structure, thereby enhancing the model's robustness in feature extraction. During training, some network connections are randomly discarded to reduce overfitting and improve the model's generalization ability. This process is equivalent to introducing a regularization strategy during training to strengthen the model's robustness under different substructures.

[0150] Conjugate gradient optimization is based on the initial search point, initial search direction, and initial number of iterations. It iteratively updates the search direction and performs the following steps until the accuracy or maximum number of iterations is met:

[0151] (1) Online search step size The formula for minimizing the objective function in the current search direction is as follows:

[0152]

[0153] (2) Update the current search point using the following formula: ;

[0154] (3) Calculate the new gradient, using the following formula: The new β value is then calculated using either the Fletcher-Reeves formula or the Polak-Ribière formula, as follows:

[0155]

[0156] in, Indicates the first gradient of step, Indicates the first The gradient of the step;

[0157] (4) Update the search direction .

[0158] Step 5: When the objective function changes tend to stabilize or reaches the specified error threshold, the training process ends and the first model is generated. At this point, the first model has a strong ability to learn attack features and has completed its adaptation to the test dataset.

[0159] The training process of the first model described above achieves knowledge transfer from the source model and fine-tunes the parameters in the feature space through the conjugate gradient optimization algorithm, so that the model can better adapt to the experimental data, improve detection accuracy and training efficiency, avoid overfitting, and enhance the practicality of the model.

[0160] It should be noted that the first model acts as a feature extractor when applied, transforming the input data into feature vectors containing depth and spatiotemporal information.

[0161] In one embodiment, optionally, the method further includes:

[0162] The training samples are input into the first model to obtain the sample feature data output by the first model. The model is used to extract attack features from the training samples through convolution and connection pruning operations to generate the sample feature data.

[0163] In this embodiment of the invention, the method of obtaining the sample feature data of the training samples is described. The training samples are input into the first model to obtain the sample feature data output by the first model. The format of the sample feature data is the same as that of the second feature data. It should be noted that the sample feature data can be obtained by inputting the training samples into the first model after the first model has been trained, or it can be generated and stored during the process of training the first model using the training samples.

[0164] In one embodiment, optionally, the original data is preprocessed to obtain first data, wherein the original data may be the original data of training samples used to train the first model, or data from the original training dataset used to train the second model. The preprocessing method includes:

[0165] First, the raw data is parsed and statistically analyzed to obtain the first vector. In one optional implementation, each first vector contains 41 valid features. The meaning of each dimension is given below in order of the number of feature bits (the data type of each dimension is in parentheses below):

[0166] 1. Duration (continuous), meaning: the duration of the connection (in seconds);

[0167] 2. protocol_type (symbol), meaning: Protocol type: Transmission Control Protocol (TCP), User Datagram Protocol (UDP), or Internet Control Message Protocol (ICMP).

[0168] 3. service (symbol), meaning: the network service of the target host, such as: Hypertext Transfer Protocol (HTTP), File Transfer Protocol (FTP), or Simple Mail Transfer Protocol (SMTP), etc.;

[0169] 4. flag (symbol), meaning: connection status flag, such as: session completed normally (Session Finished, SF), connection rejected (Rejected, REJ), or connection reset due to timeout (Reset On Timeout, RSTO), etc.

[0170] 5. src_bytes (continuous), meaning: the number of bytes transferred from the source host to the destination host;

[0171] 6. dst_bytes (contiguous), meaning: the number of bytes transferred from the target host to the source host;

[0172] 7. land (Boolean): Whether the source, destination addresses, and ports are the same;

[0173] 8. wrong_fragment (continuous), meaning: number of faulty fragments;

[0174] 9. urgent (continuous), meaning: number of urgent packages;

[0175] 10. hot (continuous), meaning: the number of times sensitive files and directories have been accessed;

[0176] 11. num_failed_logins (consecutive), meaning: the number of failed login attempts;

[0177] 12. logged_in (Boolean), meaning: whether the login was successful;

[0178] 13. num_compromised (continuous), meaning: the number of times the target has been compromised;

[0179] 14. root_shell (Boolean), meaning: whether to obtain superuser command line access permissions (root shell);

[0180] 15. su_attempted (boolean), meaning: whether a user switch (su) attempt was made;

[0181] 16. num_root (continuous), meaning: the number of times root has been accessed;

[0182] 17. num_file_creations (continuous), meaning: number of files created;

[0183] 18. num_shells (continuous), meaning: the number of shells used;

[0184] 19. num_access_files (contiguous), meaning: number of access control files;

[0185] 20. num_outbound_cmds (continuous), meaning: the number of outbound commands is always 0;

[0186] 21. is_host_login (Boolean), meaning: whether the user is a member of the "hot" list for login;

[0187] 22. is_guest_login (Boolean), meaning: whether a guest account (guest) is logged in;

[0188] 23. count (continuous), meaning: the number of connections to the same host as the current connection within the past 2 seconds;

[0189] 24. srv_count (continuous), meaning: the number of services that have the same service as the current connection in the past 2 seconds;

[0190] 25. serror_rate (continuous), meaning: the percentage of connections that experience SYN errors;

[0191] 26. srv_serror_rate (continuous), meaning: the percentage of services that experience SYN errors;

[0192] 27. rerror_rate (continuous), meaning: the proportion of connections that experience REJ errors;

[0193] 28. srv_rerror_rate (continuous), meaning: the percentage of services that experience REJ errors;

[0194] 29. same_srv_rate (continuous), meaning: the proportion of connections using the same service;

[0195] 30. diff_srv_rate (continuous), meaning: the connection ratio of different services;

[0196] 31. srv_diff_host_rate (continuous), meaning: the service ratio of different hosts;

[0197] 32. dst_host_count (continuous), meaning: the number of connections with the same destination address in the first 100 connections;

[0198] 33. dst_host_srv_count (continuous), meaning: the number of times the target host provides the same service in the first 100 connections;

[0199] 34. dst_host_same_srv_rate (continuous), meaning: the proportion of the target host with the same service;

[0200] 35. dst_host_diff_srv_rate (continuous), meaning: the proportion of different services on the target host;

[0201] 36. dst_host_same_src_port_rate (continuous), meaning: the proportion of the same source port on the target host;

[0202] 37. dst_host_srv_diff_host_rate (continuous), meaning: the service ratio of different hosts on the target host;

[0203] 38. dst_host_serror_rate (continuous), meaning: the percentage of SYN errors occurring on the target host;

[0204] 39. dst_host_srv_serror_rate (continuous), meaning: the percentage of SYN errors that occur on the target host service;

[0205] 40. dst_host_rerror_rate (continuous), meaning: the percentage of REJ errors occurring on the target host;

[0206] 41. dst_host_srv_rerror_rate (continuous), meaning: the percentage of REJ errors that occur on the target host service.

[0207] The 42nd feature is the tag. The 42nd bit of the tag represents the attack type, indicating whether the connection belongs to normal or one of the four main attack types (Denial of Service (DoS), Probing Attack (Probe), Remote-to-Local Attack (R2L), and User-to-Root Attack (U2R)) and its 37 sub-attack types.

[0208] It should be noted that the above embodiments are also applicable to the first feature vector of the network traffic data to be detected, but the first feature vector does not include the 42nd bit label feature.

[0209] Then, the first vector is normalized and encoded sequentially to obtain the second vector. The first vector is then expanded into an M-dimensional second vector. The encoding methods include, but are not limited to, Boolean expansion, binary encoding, and one-hot encoding. In one optional implementation, the "41-dimensional features and 38-bit label features" are mapped to a 454-dimensional input vector. All numerical values ​​are true values. Normalization The vectors are expanded using 10-bit binary or one-hot encoding methods and then concatenated into a 454-dimensional vector. The vector mapping method is shown in the table below:

[0210]

[0211] The following examples further illustrate the encoding method: For symbolic features, one-hot encoding is used. For example, if the second dimension's protocol_type is tcp, it is encoded as a 3-bit one-hot encoding [0,1,0]; if the third dimension's protocol_type is http, it is encoded as a 70-bit one-hot encoding; and if the fourth dimension's flag is SF, it is encoded as an 11-bit one-hot encoding (a total of 11 connection status flags, such as SF, REJ, etc.). For binary features, Boolean expansion is used, for example, 0... 0,1 1. For numerical features, normalization is performed first, followed by 10-bit binary encoding. For example, duration=181 is first normalized to obtain 0.120, and then 10-bit binary encoding is performed to obtain 0001111010. For attack types, one-hot encoding is used. For example, if the attack type is normal, it is encoded as [0,…,0,1,0,…].

[0212] Next, the second vector is subjected to dimension reduction processing to obtain the third vector. In one optional implementation, the dimension of the second vector is reduced according to the variance coefficient, and the variance coefficient of each dimension is... The calculation formula is as follows:

[0213]

[0214] in, For the first The standard deviation of the dimensional vector on the training set. For the first The mean of a dimensional vector.

[0215] Finally, according to Sort values ​​in descending order and retain Find the largest K-dimensional vector and transform the retained K-dimensional vector into... grayscale Figure 2 The matrix is ​​preprocessed with zeros to fill any missing bits, generating training samples for training the first model or the original training dataset for training the second model.

[0216] like Figure 2 As shown, this embodiment of the invention also provides a data security management and control device, including:

[0217] The first acquisition module 201 is used to acquire network traffic data to be detected in the target detection scenario and a training sample library of the target detection scenario, wherein the training sample library includes sample feature data and sample attack types of multiple training samples.

[0218] The first processing module 202 is used to preprocess the network traffic data to obtain the first feature data of the network traffic data;

[0219] The second processing module 203 is used to input the first feature data into a pre-trained first model to obtain the second feature data output by the first model. The first model is trained based on the training samples in the target detection scenario. The first model is used to extract attack features from the first feature data through convolution operations to generate the second feature data.

[0220] The first calculation module 204 is used to calculate the first Euclidean distance between the second feature data and the sample feature data of the training sample;

[0221] The first matching module 205 is used to match the sample attack type of the training sample with the first Euclidean distance to determine the target attack type of the network traffic data.

[0222] Optionally, the first processing module 202 includes:

[0223] The first processing submodule is used to parse and statistically analyze the network traffic data to obtain the first feature vector of the network traffic data;

[0224] The second processing submodule is used to normalize and encode the first feature vector to obtain the second feature vector.

[0225] The third processing submodule is used to perform dimension reduction processing on the second feature vector to obtain the third feature vector;

[0226] The fourth processing submodule is used to convert the third feature vector into a grayscale image to obtain the first feature data.

[0227] Optionally, the first computing module 204 includes:

[0228] The first acquisition submodule is used to acquire the first number of training samples of each of the sample attack types in the training sample library, and to acquire the total number of training samples in the training sample library;

[0229] The first determining submodule is used to determine the type weight value of the training sample based on the first quantity and the total quantity, wherein a training sample of one type of attack corresponds to one type weight value, and the size of the first quantity is inversely proportional to the size of the type weight value;

[0230] The first calculation submodule is used to calculate the first Euclidean distance between the second feature data and each of the sample feature data according to the type weight value and the preset dimension weight value, wherein the dimension weight value is used to represent the importance of each feature dimension in the second feature data and the sample feature data.

[0231] Optionally, the device further includes:

[0232] The second calculation module is used to calculate the total difference between the second feature data and the multiple sample feature data for each feature dimension;

[0233] The third calculation module is used to perform power-law decay and normalization on the total difference to obtain the dimension weight value of each dimension feature.

[0234] Optionally, the first matching module 205 includes:

[0235] The first sorting submodule is used to sort the first Euclidean distance in ascending order to generate a first sequence;

[0236] The first selection submodule is used to select the first K training samples corresponding to the first Euclidean distance in the first sequence to form the nearest neighbor set of the network traffic data, where K is an integer greater than 0;

[0237] The second determining submodule is used to determine the target attack type of the network traffic data based on the sample attack type of the training samples in the nearest neighbor set.

[0238] Optionally, the device further includes:

[0239] The second acquisition module is used to acquire the original training dataset, which includes multiple first traffic data and the first attack type of the first traffic data.

[0240] The third processing module is used to preprocess the data in the original training dataset to obtain the first training dataset;

[0241] The first training module is used to train the first convolutional neural network model based on the first training dataset to generate the second model;

[0242] The second training module is used to perform knowledge transfer training on the second convolutional neural network model based on the training samples in the target detection scenario and the second model, so as to obtain the first model.

[0243] Optionally, the second training module includes:

[0244] The first initialization submodule is used to initialize the second convolutional neural network model according to the network parameters of the second model;

[0245] The first training submodule is used to iteratively train the initialized second convolutional neural network model based on the training samples in the target detection scene, using connection pruning technology and conjugate gradient algorithm, to obtain the first model. In each iteration of the model training process, the connection pruning technology is used to prune the network structure of the second convolutional neural network model, and the conjugate gradient algorithm is used to determine the training direction of the model based on the pruned network structure.

[0246] Optionally, the device further includes:

[0247] The fourth processing module is used to input the training samples into the first model and obtain the sample feature data output by the first model, wherein the first model is used to extract the attack features in the training samples through convolution and connection pruning operations to generate the sample feature data.

[0248] It should be noted that the embodiments of this device are devices corresponding to the embodiments of the above methods. All implementations in the embodiments of the above methods are applicable to the embodiments of this device and can achieve the same technical effect.

[0249] This invention also provides a network device, including: a processor, a memory, and a program stored in the memory and executable on the processor. When the program is executed by the processor, it implements the data security management method as described above and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0250] This invention also provides a readable storage medium, comprising: a program stored on the readable storage medium, wherein when the program is executed by a processor, it implements the steps of the data security management method described in any of the preceding claims, and achieves the same technical effect; therefore, to avoid repetition, it will not be described again here. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0251] This invention also provides a computer program product, including computer instructions. When the computer instructions are executed by a processor, they implement the steps of the data security management method described in any of the preceding claims and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0252] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0253] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A data security management and control method, characterized in that, include: Obtain network traffic data to be detected in the target detection scenario and a training sample library for the target detection scenario, wherein the training sample library includes sample feature data and sample attack types of multiple training samples; The network traffic data is preprocessed to obtain the first feature data of the network traffic data; The first feature data is input into a pre-trained first model to obtain the second feature data output by the first model. The first model is trained based on the training samples in the target detection scenario. The first model is used to extract attack features from the first feature data through convolution operations to generate the second feature data. Calculate the first Euclidean distance between the second feature data and the sample feature data of the training samples; The target attack type of the network traffic data is determined by matching the sample attack type of the training samples with the first Euclidean distance.

2. The data security management method according to claim 1, characterized in that, The preprocessing of the network traffic data to obtain the first feature data of the network traffic data includes: The network traffic data is parsed and statistically analyzed to obtain the first feature vector of the network traffic data; The first feature vector is normalized and encoded to obtain the second feature vector; The second feature vector is subjected to dimensionality reduction processing to obtain the third feature vector; The third feature vector is converted into a grayscale image to obtain the first feature data.

3. The data security management method according to claim 1, characterized in that, The calculation of the first Euclidean distance between the second feature data and the sample feature data of the training samples includes: Obtain the first number of training samples for each of the aforementioned attack types in the training sample library, and obtain the total number of training samples in the training sample library; Based on the first quantity and the total quantity, the type weight value of the training sample is determined, wherein a training sample of one type of attack corresponds to one type weight value, and the size of the first quantity is inversely proportional to the size of the type weight value; Based on the type weight value and the pre-set dimension weight value, a first Euclidean distance is calculated between the second feature data and each of the sample feature data, wherein the dimension weight value is used to represent the importance of each feature dimension in the second feature data and the sample feature data.

4. The data security management method according to claim 3, characterized in that, The method further includes: For each feature dimension, calculate the total difference between the second feature data and the multiple sample feature data; The total difference is subjected to power-law decay and normalization to obtain the dimension weight value of each dimension feature.

5. The data security management method according to claim 1, characterized in that, The step of determining the target attack type of the network traffic data by performing type matching based on the sample attack type of the training samples and the first Euclidean distance includes: Sort the first Euclidean distances in ascending order to generate the first sequence; The first K training samples corresponding to the first Euclidean distance in the first sequence are selected to form the nearest neighbor set of the network traffic data, where K is an integer greater than 0; The target attack type of the network traffic data is determined based on the sample attack type of the training samples in the nearest neighbor set.

6. The data security management method according to claim 1, characterized in that, The method further includes: Obtain the original training dataset, which includes multiple first traffic data and a first attack type of the first traffic data; The data in the original training dataset is preprocessed to obtain the first training dataset; The first convolutional neural network model is trained based on the first training dataset to generate the second model; Based on the training samples in the target detection scenario and the second model, the second convolutional neural network model is trained by knowledge transfer to obtain the first model.

7. The data security management method according to claim 6, characterized in that, The step of performing knowledge transfer training on the second convolutional neural network model based on the training samples in the target detection scene and the second model to obtain the first model includes: The second convolutional neural network model is initialized based on the network parameters of the second model; Based on the training samples in the target detection scenario, the initialized second convolutional neural network model is iteratively trained using connection pruning and conjugate gradient algorithms to obtain the first model. In each iteration of the model training process, the connection pruning technique is used to prune the network structure of the second convolutional neural network model, and the conjugate gradient algorithm is used to determine the training direction of the model based on the pruned network structure.

8. The data security management method according to claim 1, characterized in that, The method further includes: The training samples are input into the first model to obtain the sample feature data output by the first model. The model is used to extract attack features from the training samples through convolution and connection pruning operations to generate the sample feature data.

9. A data security management and control device, characterized in that, include: The first acquisition module is used to acquire network traffic data to be detected in the target detection scenario and a training sample library of the target detection scenario, wherein the training sample library includes sample feature data and sample attack types of multiple training samples. The first processing module is used to preprocess the network traffic data to obtain the first feature data of the network traffic data; The second processing module is used to input the first feature data into a pre-trained first model to obtain the second feature data output by the first model. The first model is trained based on the training samples in the target detection scenario. The first model is used to extract attack features from the first feature data through convolution operations to generate the second feature data. The first calculation module is used to calculate the first Euclidean distance between the second feature data and the sample feature data of the training sample; The first matching module is used to match the sample attack type of the training sample with the first Euclidean distance to determine the target attack type of the network traffic data.

10. A network device, characterized in that, include: A processor, a memory, and a program stored in the memory and executable on the processor, wherein the program, when executed by the processor, implements the data security management method as described in any one of claims 1 to 8.

11. A readable storage medium, characterized in that, include: The readable storage medium stores a program that, when executed by a processor, implements the steps of the data security management method as described in any one of claims 1 to 8.

12. A computer program product, characterized in that, It includes computer instructions, which, when executed by a processor, implement the steps of the data security management method as described in any one of claims 1 to 8.

Images