Sample generation method and device, electronic equipment and readable storage medium
By constructing and adjusting the hypersphere to generate balanced data sets and using the generative adversarial network to optimize samples, the data category imbalance problem is solved, and the training effect and recognition accuracy of the detection model are improved.
Patent Information
- Application Number
- CN202411770695.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-04
- Publication Date
- 2025-04-11
AI Technical Summary
In machine learning and deep learning, data category imbalance causes the model to tend toward most classes when predicting, affecting the training effect of the model.
By obtaining the initial dataset, building a hypersphere, adjusting its coverage to exclude a few class samples, generating a balanced dataset, and inputting it to generate an adversarial network to further optimize the samples.
The training effect and recognition accuracy of the detection model are improved, ensuring that the number of samples of different categories in the dataset is more balanced.
Smart Images
Figure CN119807737A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and in particular to a sample generation method, device, electronic device and readable storage medium. Background Art
[0002] In the application of machine learning and deep learning, sample generation is a crucial link. Samples are crucial for model training and evaluation, and the quality of sample generation directly affects the performance and effect of the model. However, in many practical applications, data is often unbalanced, that is, the number of samples in some categories is much larger than that in other categories. The imbalance of samples may cause the model to be biased towards the majority class when predicting.
[0003] For example, when using deep learning technology for network intrusion detection, network traffic data usually has a serious class imbalance problem, with the vast majority of normal traffic and a small amount of abnormal traffic. If the collected data is not balanced and directly used for model training, it will affect the training effect of the model. Summary of the invention
[0004] The embodiment of the present application provides a sample generation method, which can improve the training effect of the detection model and thus improve the recognition accuracy of the detection model.
[0005] In a first aspect, an embodiment of the present application discloses a sample generation method, the method comprising:
[0006] Acquire an initial data set; the initial data set includes a first sample and a second sample; the number of the first samples is less than or equal to a first preset threshold;
[0007] Based on the first sample, construct at least one first hypersphere;
[0008] In a case where the second sample is included in the coverage of the first hypersphere, reconstructing the first hypersphere until the second sample is no longer included in the coverage of the first hypersphere;
[0009] When the second sample is not included in the coverage of the first hypersphere, generating a first data set based on the first sample and the first hypersphere;
[0010] Inputting the initial data set and the first data set into a generative adversarial network to generate a target data set;
[0011] The target data set is used to train the detection model, and the trained detection model is tested.
[0012] Optionally, constructing at least one first hypersphere based on the first sample includes:
[0013] For each first sample, calculating a first distance between the first sample and each sample in the initial data set;
[0014] Adding samples in the initial data set whose first distance to the first sample meets a preset condition to a candidate sample set;
[0015] A first hypersphere is constructed based on the first sample and samples in the candidate sample set.
[0016] Optionally, when the second sample is not included in the coverage of the first hypersphere, generating a first data set based on the first sample and the first hypersphere includes:
[0017] Generate a random number sequence; the square of each random number in the random number sequence is less than 1;
[0018] A first data set is generated based on the first sample, the random number sequence, and the radius of the first hypersphere; the first data set includes at least one data subset; and one first sample corresponds to one data subset.
[0019] Optionally, the using the target data set to train the detection model and testing the trained detection model includes:
[0020] For a sample data matrix corresponding to each sample data in the target data set, the sample data matrix is input into a capsule network, and based on each capsule in the capsule network, a first output of each capsule is obtained;
[0021] In each round of iteration, the coupling coefficient of the dynamic routing mechanism is determined according to the protocol score, and the number of iterations is recorded; the protocol score is used to indicate the relevance of the sample data in the capsule network;
[0022] Determining a second output of the capsule network based on the first output of each capsule in the capsule network and a coupling coefficient of the dynamic routing mechanism;
[0023] Updating the protocol score and performing the next round of iteration;
[0024] When the number of iterations reaches a preset threshold, the iteration is stopped, and a feature vector corresponding to the sample data is obtained based on the second output of the capsule network;
[0025] Based on the feature vector, predict a first classification result corresponding to the sample data;
[0026] According to the first classification result and the actual classification result corresponding to the sample data, various parameters of the detection model are updated.
[0027] Optionally, the method further comprises:
[0028] After the first round of iteration is completed, the first output of each capsule in the capsule network is updated based on the self-attention mechanism, and the next round of iteration is performed.
[0029] Optionally, updating the protocol score and performing the next round of iteration includes:
[0030] Performing a transposition operation on the second output of the capsule network;
[0031] Calculate the product of the transposed second output and the first output of each capsule to obtain a first operation result;
[0032] Adding the first operation result to the protocol score to obtain an updated protocol score;
[0033] The next iteration is performed based on the updated agreement score.
[0034] Optionally, after the first round of iteration is completed, the first output of each capsule in the capsule network is updated based on the self-attention mechanism, and the next round of iteration is performed based on the updated first output, including:
[0035] Encoding the first output by using a self-attention mechanism to obtain a sequence feature corresponding to the first output;
[0036] The sequence feature is superimposed on the first output of each capsule to obtain an updated first output.
[0037] In a second aspect, an embodiment of the present application discloses a sample generation device, the device comprising:
[0038] An acquisition module, configured to acquire an initial data set; the initial data set includes a first sample and a second sample; the number of the first samples is less than or equal to a first preset threshold;
[0039] A first construction module, configured to construct at least one first hypersphere based on the first sample;
[0040] A second construction module is used for reconstructing the first hypersphere when the second sample is included in the coverage of the first hypersphere, until the second sample is no longer included in the coverage of the first hypersphere;
[0041] A first generating module, configured to generate a first data set based on the first sample and the first hypersphere when the second sample is not included in the coverage of the first hypersphere;
[0042] A second generating module, configured to input the initial data set and the first data set into a generative adversarial network to generate a target data set;
[0043] The training module is used to train the detection model using the target data set and test the detection model after the training is completed.
[0044] In a third aspect, an embodiment of the present application discloses an electronic device, which includes a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other through the communication bus; the memory is used to store executable instructions, and the executable instructions enable the processor to execute the sample generation method as described above.
[0045] In a fourth aspect, an embodiment of the present application discloses a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the aforementioned sample generation method is implemented.
[0046] The embodiments of the present application include the following advantages:
[0047] Acquire an initial data set; the initial data set includes a first sample and a second sample; based on the first sample, construct at least one first hypersphere; when the second sample is included in the coverage of the first hypersphere, reconstruct the first hypersphere until the second sample is not included in the coverage of the first hypersphere; when the second sample is not included in the coverage of the first hypersphere, generate a first data set based on the first sample and the first hypersphere, so that the number of samples of different categories in the data set used for training the model is more balanced. Input the initial data set and the first data set into the generative adversarial network to generate a target data set, further optimize the generated samples, and improve the training effect of the detection module. Use the target data set to train the detection model, and test the detection model after training to improve the training effect of the detection model, thereby improving the accuracy of the detection model in identifying the sample data category. BRIEF DESCRIPTION OF THE DRAWINGS
[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.
[0049] Figure 1 is a flow chart of steps of an embodiment of a sample generation method of the present invention;
[0050] Figure 2 is a structural block diagram of a sample generating device of the present invention;
[0051] Figure 3 It is a structural block diagram of an electronic device provided by an example of the present invention. DETAILED DESCRIPTION
[0052] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0053] The terms "first", "second", etc. in the specification and claims of the present invention are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable when appropriate, so that the embodiments of the present invention can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, the term "and / or" in the specification and claims is used to describe the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. In the embodiments of the present invention, the term "multiple" refers to two or more, and other quantifiers are similar.
[0054] Method Embodiment
[0055] Reference Figure 1 , shows a flow chart of a sample generation method embodiment of the present application, the method specifically comprising the following steps:
[0056] Step S101: Acquire an initial data set.
[0057] The initial data set includes a first sample and a second sample; the number of the first samples is less than or equal to a first preset threshold.
[0058] The sample generation method provided in this application can be applied to the fields of machine learning and deep learning to generate sample-balanced training sets for machine learning models and deep learning models. For example: in the field of network intrusion detection, the sample generation method is used to generate network traffic data, and the generated traffic data is used to train the intrusion detection model, and the abnormal network traffic data is identified by the intrusion detection model. In the field of natural language processing, the sample generation method can be used for text generation tasks, such as automatically generating news summaries, stories, poems, etc., and the generated text is used to train the language model. In the field of computer vision, the sample generation method can be used to generate new images, and the generated images are used to train the recognition model.
[0059] The sample generation method provided in this application can also be used to simulate real data and provide a large amount of data for analysis. For example, in the financial field, the sample generation method can be used to simulate the financial market. By generating random sample data that conforms to a specific distribution, the fluctuation of stock prices can be simulated. In the field of risk management, the sample generation method can be used to generate risk data under different scenarios to help enterprises and institutions conduct risk assessment and management.
[0060] Among them, in the initial data set, the number of samples of different categories is unbalanced. The first sample refers to the minority class sample in the initial data set, and the second sample refers to the majority class sample in the initial data set. The number of samples of different categories in the initial data set is counted, and the category with fewer samples is the minority class sample, that is, the first sample.
[0061] The first preset threshold refers to a value less than half of the total number of samples in the initial data set, and the samples whose number of samples is less than or equal to the first preset threshold are the first samples. For example, if the number of samples in the initial data set is 100, the first preset threshold is a value less than 50, and the samples whose number of samples is less than or equal to 49 are the first samples.
[0062] Step S102: construct at least one first hypersphere based on the first sample.
[0063] In n-dimensional space, a hypersphere is an n-1-dimensional sphere. For example, in two-dimensional space, a hypersphere is a circle; in three-dimensional space, it is a sphere.
[0064] It should be noted that at least one sample is selected from the first samples as the center point of a first hypersphere, and each first hypersphere corresponds to a different center point. The center point can be the mean, median, mode of the sample, or the center calculated by a clustering algorithm, such as the K-means algorithm and the DBSCAN algorithm.
[0065] For each selected center point, calculate the distance between the center point and all other points in the first sample. According to actual needs, select one of the distances as the radius of the first hypersphere. The radius can be the maximum distance or the average distance. With the selected center point as the center and the calculated radius as the radius, a hypersphere is constructed in n-dimensional space.
[0066] Step S103: When the second sample is included in the coverage of the first hypersphere, reconstruct the first hypersphere until the second sample is no longer included in the coverage of the first hypersphere.
[0067] It should be noted that, for each first hypersphere, the distance between each second sample and the center point of the first hypersphere is calculated. If there is at least one distance less than or equal to the radius corresponding to the first hypersphere, it means that the second sample is included in the coverage range of the first hypersphere.
[0068] The formula for calculating the distance between the second sample and the center point of the first hypersphere is as follows:
[0069] ||y j -x m ||≤r……(1)
[0070] In formula (1), y j represents a second sample, X m represents the center point of the first hypersphere, and r represents the radius of the first hypersphere.
[0071] If the second sample is included in the coverage of the first hypersphere, the first hypersphere is reconstructed. Based on each constructed first hypersphere, it is determined whether the coverage of the first hypersphere includes the second sample; if the second sample is included in the coverage of the first hypersphere, the first hypersphere is reconstructed until the coverage of the first hypersphere does not include the second sample.
[0072] It should be noted that, when the second sample is included in the coverage of the first hypersphere, an optimization algorithm is used to adjust the center point and / or radius of the first hypersphere, and the first hypersphere is reconstructed based on the adjusted center point and / or radius. The optimization algorithm may include: gradient descent method, genetic algorithm, particle swarm optimization algorithm.
[0073] Exemplarily, based on a set of two-dimensional sample points, the center point and radius of the first hypersphere are adjusted using the gradient descent method so that there is no second sample within the coverage of the first hypersphere. The mean square error is selected as the loss function, and the calculation formula of the loss function is as follows:
[0074]
[0075] In formula (2), r is the radius of the circle, (c x ,c y ) is the coordinate of the first hypersphere, (x i ,y i ) is the coordinate of the ith data point, and N is the total number of sample points.
[0076] Randomly initialize the radius r and center coordinates (c x ,c y ), the loss function is about r, c x 、c y Calculate partial derivatives respectively to get gradients. Use gradient descent method to update r and c x 、c y , the update formula is:
[0077]
[0078] In formulas (3), (4), and (5), α is the learning rate, which controls the step size of each update, repeatedly calculates the gradient and updates r and c x 、c y , until the loss function converges below the preset threshold or reaches the preset number of iterations.
[0079] Step S104: When the second sample is not included in the coverage of the first hypersphere, generate a first data set based on the first sample and the first hypersphere.
[0080] The first data set is generated based on the first sample and the first hypersphere according to a preset rule. The preset rule may be generating points uniformly distributed on the first hypersphere and within the hypersphere.
[0081] Step S105: input the initial data set and the first data set into a generative adversarial network to generate a target data set.
[0082] The initial dataset is original data, a real data sample for training the generative adversarial network, a first dataset, a dataset generated based on the first sample and the first hypersphere, and is input into the generative adversarial network as additional information or conditions.
[0083] Among them, Generative Adversarial Network (GAN) is a deep learning framework that generates new data samples through adversarial training. GAN is mainly composed of two key parts: Generator and Discriminator. Among them, the generator accepts random noise and possible conditional information as input, and outputs generated data. Random noise is usually a high-dimensional vector, and the possible conditional information can be the information in the first data set. The discriminator is used to accept data as input and output a probability value, indicating the possibility that the data is real data. The data received by the discriminator can be real data or data generated by the generator.
[0084] Training a generative adversarial network can include the following steps:
[0085] Step 11: Input the initial data set into the discriminator as a real data sample;
[0086] Step 12: Provide the random noise and the information in the first data set as input to the generator to generate false data;
[0087] Step 13: Use the discriminator to classify real data and fake data and calculate the loss function;
[0088] Step 14: Update the parameters of the generator and discriminator through the back propagation algorithm;
[0089] Step 15: Repeat the above process until the generator can generate fake data that is close to the real data.
[0090] After the training is completed, the generator generates new data samples as the target data set.
[0091] Step S106: train the detection model using the target data set, and test the trained detection model.
[0092] Among them, the detection model can be a deep learning model such as a Convolutional Neural Network (CNN) and a Long Short-Term Memory Network (LSTM).
[0093] Divide the target dataset into training and test sets, load the training set into the detection model, and set training parameters such as learning rate, batch size, and number of training rounds. Run the training script to enable the detection model to learn the features and patterns in the training set. Monitor the training progress and performance of the model through indicators such as loss function and accuracy. After training, load the test set into the detection model, use the detection model to predict the test set, and calculate evaluation indicators such as precision, recall, F1 score, etc., and analyze the performance of the model based on the test results.
[0094] In an embodiment of the present application, an initial data set is obtained; the initial data set includes a first sample and a second sample; based on the first sample, at least one first hypersphere is constructed; when the second sample is included in the coverage of the first hypersphere, the first hypersphere is reconstructed until the second sample is not included in the coverage of the first hypersphere; when the second sample is not included in the coverage of the first hypersphere, a first data set is generated based on the first sample and the first hypersphere, so that the number of samples of different categories in the data set used for training the model is more balanced. The initial data set and the first data set are input into the generative adversarial network to generate a target data set, and the generated samples are further optimized to improve the training effect of the detection module. The detection model is trained using the target data set, and the detection model after training is tested to improve the training effect of the detection model, thereby improving the accuracy of the detection model in identifying the sample data category.
[0095] Optionally, constructing at least one first hypersphere based on the first sample includes:
[0096] Step 21: For each first sample, calculate a first distance between the first sample and each sample in the initial data set;
[0097] Step 22: adding the samples in the initial data set whose first distance from the first sample meets a preset condition to the candidate sample set;
[0098] Step 23: construct a first hypersphere based on the first sample and samples in the candidate sample set.
[0099] It should be noted that the first distance between the first sample and each sample in the initial data set may be calculated in the following ways: Euclidean distance, Manhattan distance, and Mahalanobis distance. For example, the Euclidean distance is used to measure the absolute distance between the first sample and each sample in the initial data set, and the Mahalanobis distance is used to measure the covariance distance between the first sample and each sample in the initial data set.
[0100] The preset condition may be less than or equal to a preset distance threshold, or a preset number of samples may be selected from low to high according to the first distance.
[0101] There are k first samples x i , the preset condition is that the distance is less than or equal to the preset distance threshold, based on each first sample x i , calculate the first distance between the first sample and each sample in the initial data set, add the samples in the initial data set that are less than or equal to the preset distance threshold to the candidate sample set, and obtain k candidate sample sets X ik The preset distance threshold corresponding to each first sample may be the same or different.
[0102] By x i With X ik One of the samples in , constructs a hypersphere C, where the radius of the hypersphere C is calculated as follows:
[0103]
[0104] In formula (6), r is the radius of the first hypersphere C.
[0105] It should be noted that, when there is a second sample y j If the second sample y falls within the coverage of the hypersphere C, j Add a noise sample set CS, and if CS is a non-empty set, select the distance x from CS i The most recent sample y p ,y p The calculation formula is as follows:
[0106]
[0107] Connect x i With y p The formed straight line is used as the diameter to reconstruct the first hypersphere C1.
[0108] In an embodiment of the present application, for each first sample, a first distance between the first sample and each sample in the initial data set is calculated; samples in the initial data set whose first distance to the first sample meets a preset condition are added to a candidate sample set; based on the first sample and the samples in the candidate sample set, a first hypersphere is constructed so that the hypersphere generates more diverse samples, which helps the detection model to fully learn the characteristics of the traffic data and improve the training effect of the detection model.
[0109] Optionally, when the second sample is not included in the coverage of the first hypersphere, generating a first data set based on the first sample and the first hypersphere includes:
[0110] Step 31, generating a random number sequence; the square of each random number in the random number sequence is less than 1;
[0111] Step 32: Generate a first data set based on the first sample, the random number sequence, and the radius of the first hypersphere; the first data set includes at least one data subset; one first sample corresponds to one data subset.
[0112] The first hypersphere is based on the first sample x i And the sample x that meets the preset conditions k Constructed, the center point of the first hypersphere is x m , x m The calculation formula is as follows:
[0113]
[0114] The product of the radius of the first hypersphere and the random number is calculated to obtain a second operation result; and the center point of the first hypersphere and the second operation result are added to obtain a new sample.
[0115] Specifically, the way to generate new samples is as follows:
[0116]
[0117] In formula (9), x syn is the new sample generated, r is the radius of the first hypersphere, v = {v 1 ,...,v i ,...,v d},v i is a random number, and ||v|| 2 <1. The new samples obtained by the above generation method can ensure that the new samples are within the coverage of the first hypersphere.
[0118] It should be noted that for each first sample, a new sample can be generated based on the first hypersphere corresponding to the first sample and a random number; for each first sample, a corresponding series of new samples can be generated based on the first hypersphere corresponding to the first sample and a series of random numbers, and a series of new samples are aggregated to obtain a data subset corresponding to the first sample. The data subsets corresponding to each first sample are aggregated to obtain a first data set. Among them, the number of random numbers affects the number of new samples in the generated first data set.
[0119] In the embodiment of the present application, a random number sequence is generated; the square of each random number in the random number sequence is less than 1. Based on the first sample, the random number sequence, and the radius of the first hypersphere, a first data set is generated; the first data set contains at least one data subset; one of the first samples corresponds to one of the data subsets, ensuring that the generated new samples are within the coverage of the hypersphere and that the expected number of samples is obtained, reducing the noise in the sample set, and improving the training effect and classification accuracy of the detection model.
[0120] Optionally, the using the target data set to train the detection model and testing the trained detection model includes:
[0121] Step 41: for each sample data matrix corresponding to each sample data in the target data set, input the sample data matrix into the capsule network, and obtain the first output of each capsule based on each capsule in the capsule network;
[0122] Step 42: In each round of iteration, the coupling coefficient of the dynamic routing mechanism is determined according to the protocol score, and the number of iterations is recorded; the protocol score is used to indicate the relevance of the sample data in the capsule network;
[0123] Step 43: determining a second output of the capsule network based on the first output of each capsule in the capsule network and the coupling coefficient of the dynamic routing mechanism;
[0124] Step 44: Update the protocol score and perform the next round of iteration;
[0125] Step 45: When the number of iterations reaches a preset threshold, the iteration is stopped, and a feature vector corresponding to the sample data is obtained based on the second output of the capsule network;
[0126] Step 46: predicting a first classification result corresponding to the sample data based on the feature vector;
[0127] Step 47: Update various parameters of the detection model according to the first classification result and the true classification result corresponding to the sample data.
[0128] Among them, the capsule network is a neural network structure. The capsule network is usually composed of multiple capsule layers to form a hierarchical structure, such as low-level capsules and high-level capsules. The capsule layers are connected to each other through a dynamic routing mechanism. Each capsule layer contains multiple capsules, and each capsule is composed of a group of neurons. Each capsule is used to capture specific features of the input data. The connection weights between different capsules are determined by the dynamic routing mechanism to achieve feature selection and combination. The output of the capsule is a vector. The length of the vector represents the probability of the feature's existence, and the direction of the vector represents the feature information.
[0129] In the embodiment of the present application, the second output of the capsule network is equivalent to the output of the high-level capsule in the capsule network, and each capsule refers to a capsule in the low-level capsule.
[0130] Among them, the agreement score is used to represent the similarity or consistency between the output of the lower-level capsule and the prediction vector of the higher-level capsule. The role of the agreement score is to guide the dynamic routing algorithm to ensure that information can be transmitted between capsules along the most relevant or consistent path. The coupling coefficient represents the connection strength or weight between the output vector of the lower-level capsule and the higher-level capsule. The weight determines the proportion of the output vector of the lower-level capsule when it is passed to the higher-level capsule. In the dynamic routing process, the agreement score is used as the basis for updating the coupling coefficient. Specifically, the capsule network calculates the consistency between the output vector of each lower-level capsule and the prediction vector of the higher-level capsule based on the agreement score. Based on the consistency, the capsule network dynamically adjusts the coupling coefficient so that the output vector of the lower-level capsule that is more consistent or more relevant to the higher-level capsule has a greater weight when it is passed to the higher-level capsule. For example, the coupling coefficient corresponding to the agreement score can be obtained by the softmax function, and the formula is as follows:
[0131] c = softmax(g)(10)
[0132] In formula (10), g is the protocol score and c is the coupling coefficient of the dynamic routing mechanism.
[0133] It should be noted that before the sample data is input into the capsule network, the sample data is vectorized using embedding technology. Among them, embedding technology is the process of mapping high-dimensional data to low-dimensional vector space through mathematical transformation so that the data can be processed and analyzed more efficiently. Common embedding technologies include: Entity Embedding or One-Hot Encoding combined with Embedding Layer. For example, traffic data includes detailed information of network sessions, source IP address, destination IP address, protocol type, data transmission volume, etc. The traffic data is converted into vector representation using embedding technology:
[0134] E=[e1,e2,…,e i ]∈R i×n (11)
[0135] In formula (11), e i Represents the vector formed after the i-th byte data is encoded.
[0136] It should be noted that, in each capsule, the first output is obtained by multiplying the first conversion matrix and the sample data matrix, and the calculation formula is as follows:
[0137] z o=EW i (12)
[0138] In formula (12), E is the sample data matrix, W i The first transformation matrix, z i is the first output of the i-th capsule.
[0139] In each round of iteration, each capsule uses the current corresponding coupling coefficient as the weight, and performs weighted summation on the first output corresponding to each capsule to obtain the prediction vector of the capsule network. The calculation formula is as follows:
[0140]
[0141] In formula (13), v i is the i-th prediction vector of the capsule network, c i is the coupling coefficient corresponding to the ith capsule, z i is the first output of the i-th capsule, and m is the total number of capsules in the capsule network.
[0142] A nonlinear squeezing function is used to keep the length of the prediction vector between 0 and 1 to obtain the second output of the capsule network. The calculation formula is as follows:
[0143]
[0144] In formula (14), v i is the i-th prediction vector of the capsule network, o i It is the second output corresponding to the i-th capsule network.
[0145] In the embodiment of the present application, the protocol score may be updated by using the dot product or cosine similarity between the prediction vector of the capsule network and the first output of each capsule.
[0146] It should be noted that the second output of the capsule network is used as the input of the fully connected layer to obtain the final feature representation. The calculation formula is as follows:
[0147] q i =ReLU(o i W i ') (15)
[0148] In formula (15), o i is the second output corresponding to the i-th capsule network, W i is the second transformation matrix corresponding to the fully connected layer, and the eigenvector corresponding to the sample data is q=[q1,…q k ]
[0149] In the embodiment of the present application, a softmax function can be used to output the probability of the sample data category. The function of the softmax function is to map the features corresponding to the sample data to a probability value of 0-1. The calculation formula is as follows:
[0150] P = softmax(q)(16)
[0151] In formula (16), P is the calculated probability matrix, and the category corresponding to the maximum value in P is the first classification result corresponding to the sample data.
[0152] It should be noted that, when the sample data is traffic data, the categories corresponding to the sample data may include: illegal attempts to interrupt or interfere with the normal operation of the host or network (Denial-Of-Service, DOS), remote unauthorized users illegally obtaining user privileges of the local host (Remote to Local, R2L), local unauthorized users illegally obtaining privileges of the local super user or administrator (User to Root, U2R), illegal scanning of hosts or networks, looking for vulnerabilities, searching system configurations or network topologies (Surveillance or probe, Probe), and normal (Normal).
[0153] For example, P = [0.5, 0.2, 0.1, 0.15, 0.05], and the corresponding category list is [Dos, R2L, Probe, U2R, Normal], which means that the values identified as Dos, R2L, Probe, U2R, and Normal are 0.5, 0.2, 0.1, 0.15, and 0.05, respectively. The first element 0.5 is the maximum value, and the corresponding traffic type is Dos. The first classification result corresponding to the input traffic is Dos attack traffic. If P = [0.1, 0.15, 0.1, 0.05, 0.6], the maximum value in P is 0.6, and the corresponding traffic type is Normal, then the first classification result corresponding to the input traffic is normal.
[0154] In an embodiment of the present application, a capsule network is used as a deep learning model, and each capsule in the capsule network outputs a vector, rather than just a scalar in a traditional neural network. The vector contains information such as the location, size, direction, and probability of existence of the feature, and can more comprehensively characterize the input data. The capsule network retains more spatial hierarchical information by grouping features with spatial relationships into capsules, which is very important for understanding the internal structure and spatial relationships of complex objects. The capsule network can automatically adjust the weight distribution between capsules through a dynamic routing mechanism, so that it has good robustness to deformation and perspective changes of the input data.
[0155] Optionally, the method further comprises:
[0156] After the first round of iteration is completed, the first output of each capsule in the capsule network is updated based on the self-attention mechanism, and the next round of iteration is performed.
[0157] Among them, the self-attention mechanism is a mechanism used in neural networks that can dynamically capture the dependencies between elements at different positions in a sequence and generate new sequence representations based on these dependencies. The self-attention mechanism generates new feature representations by calculating the mutual dependencies between sequence elements, usually through Query-Key-Value decomposition and attention weight allocation.
[0158] It should be noted that the first output of each capsule can be represented as a vector. These output vectors will be used as the input of the self-attention mechanism to initialize the Query, Key, and Value matrices by linearly transforming the input vectors.
[0159] For the Query matrix corresponding to the output vector of each capsule, the dot product between the output vector of each capsule and the Key matrix corresponding to the output vectors of all other capsules is calculated to obtain the attention score. A scaling factor is introduced to scale the dot product result, and the scaled attention score is normalized using the softmax function to obtain the attention weight. According to the attention weight, the Value matrix corresponding to the output vectors of all capsules is weighted and summed to obtain the updated output vector matrix.
[0160] In the embodiment of the present application, the self-attention mechanism can capture the global dependencies in the sequence data. Through the self-attention mechanism, the capsule can focus on the most relevant part of the input data, thereby extracting richer features. The self-attention mechanism allows the capsule to dynamically adjust its weight according to different parts of the input data to achieve weighted fusion of features, which helps the capsule network to better understand and represent the complex structure and relationship of the input data, thereby improving the accuracy of classifying sample data.
[0161] Optionally, updating the protocol score and performing the next round of iteration includes:
[0162] Step 51: performing a transposition operation on the second output of the capsule network;
[0163] Step 52: Calculate the product of the transposed second output and the first output of each capsule to obtain a first operation result;
[0164] Step 53: superimpose the first operation result on the protocol score to obtain an updated protocol score;
[0165] Step 54: Perform the next iteration according to the updated protocol score.
[0166] It should be noted that the second output of the capsule network is equivalent to the output of the high-level capsule in the capsule network, and each capsule refers to the capsule in the low-level capsule.
[0167] The matching degree between the output of the capsule network and the expected output is evaluated by calculating the product of the transposed second output and the first output of each capsule. The agreement score reflects the connection strength or consistency between each capsule in the lower layer and the higher layer capsule. The connection strength is updated by superimposing the first operation result on the original agreement score, so that more matching connections are strengthened and mismatching connections are suppressed.
[0168]
[0169] In formula (17), g i is the agreement score corresponding to the ith capsule, z i The first output corresponding to the i-th capsule in the lower layer capsule, o i is the i-th output corresponding to the high-level capsule.
[0170] In an embodiment of the present application, a transposition operation is performed on the second output of the capsule network; the product of the transposed second output and the first output of each capsule is calculated to obtain a first operation result, which can reflect the degree of matching between the output of the capsule network and the expected output; the first operation result is superimposed on the protocol score to obtain an updated protocol score; the connection strength is updated by superimposing the first operation result on the original protocol score, so that more matching connections are strengthened and unmatched connections are suppressed, thereby allowing the capsule network to focus on more important features of the sample data.
[0171] Optionally, after the first round of iteration is completed, the first output of each capsule in the capsule network is updated based on the self-attention mechanism, and the next round of iteration is performed based on the updated first output, including:
[0172] Step 61: Encode the first output using a self-attention mechanism to obtain a sequence feature corresponding to the first output;
[0173] Step 62: superimpose the sequence feature on the first output of each capsule to obtain an updated first output.
[0174] Among them, encoding the first output using the self-attention mechanism refers to generating a corresponding query vector, key vector and value vector for the first output, calculating the attention score between the query vector and all key vectors, normalizing the attention score to obtain the attention weight, and using the attention weight to perform weighted summation on the value vector to obtain the final output vector. The output vector is the result of encoding the input vector through the self-attention mechanism.
[0175] Combined with the result after encoding by the self-attention mechanism, the residual structure is used to update the first output. The calculation formula is as follows:
[0176] z i =z i +SA(z i ) (18)
[0177] In formula (18), SA refers to the self-attention mechanism, z i Refers to the first output corresponding to the i-th capsule.
[0178] The residual structure refers to the difference between the original input and the transformed output. In the formula, SA(z i ) is z i The difference between the output after the update and the output before the update.
[0179] In an embodiment of the present application, the first output is encoded using a self-attention mechanism to obtain a sequence feature corresponding to the first output; the sequence feature is superimposed on the first output of each capsule to obtain an updated first output, and the sequence features of the sample data are taken into account to improve the efficiency and accuracy of the detection model in reading the sample data for classification.
[0180] In summary, an initial data set is obtained; the initial data set includes a first sample and a second sample; based on the first sample, at least one first hypersphere is constructed; when the second sample is included in the coverage of the first hypersphere, the first hypersphere is reconstructed until the second sample is not included in the coverage of the first hypersphere; when the second sample is not included in the coverage of the first hypersphere, a first data set is generated based on the first sample and the first hypersphere, so that the number of samples of different categories in the data set used for training the model is more balanced. The initial data set and the first data set are input into the generative adversarial network to generate a target data set, and the generated samples are further optimized to improve the training effect of the detection module. The detection model is trained using the target data set, and the detection model after training is tested to improve the training effect of the detection model, thereby improving the recognition accuracy of the detection model.
[0181] Device Embodiment
[0182] like Figure 2 FIG. 1 is a block diagram showing a structure of a sample generation device of the present application, wherein the device comprises:
[0183] An acquisition module, configured to acquire an initial data set; the initial data set includes a first sample and a second sample; the number of the first samples is less than or equal to a first preset threshold;
[0184] A first construction module, configured to construct at least one first hypersphere based on the first sample;
[0185] A second construction module is used for reconstructing the first hypersphere when the second sample is included in the coverage of the first hypersphere, until the second sample is no longer included in the coverage of the first hypersphere;
[0186] A first generating module, configured to generate a first data set based on the first sample and the first hypersphere when the second sample is not included in the coverage of the first hypersphere;
[0187] A second generating module, configured to input the initial data set and the first data set into a generative adversarial network to generate a target data set;
[0188] The training module is used to train the detection model using the target data set and test the detection model after the training is completed.
[0189] Optionally, the first building block includes:
[0190] A calculation module, configured to calculate, for each first sample, a first distance between the first sample and each sample in the initial data set;
[0191] An adding module, configured to add samples in the initial data set whose first distance from the first sample meets a preset condition to a candidate sample set;
[0192] A construction submodule is used to construct a first hypersphere based on the first sample and samples in the candidate sample set.
[0193] Optionally, the first generating module includes:
[0194] A first generating submodule, used for generating a random number sequence; the square of each random number in the random number sequence is less than 1;
[0195] The second generating submodule is used to generate a first data set based on the first sample, the random number sequence, and the radius of the first hypersphere; the first data set includes at least one data subset; one first sample corresponds to one data subset.
[0196] Optionally, the training module includes:
[0197] An input module, for inputting a sample data matrix corresponding to each sample data in a target data set into a capsule network, and obtaining a first output of each capsule in the capsule network based on each capsule;
[0198] A first determination module is used to determine the coupling coefficient of the dynamic routing mechanism according to the protocol score in each round of iteration, and record the number of iterations; the protocol score is used to indicate the relevance of the sample data in the capsule network;
[0199] A second determination module, configured to determine a second output of the capsule network based on the first output of each capsule in the capsule network and a coupling coefficient of the dynamic routing mechanism;
[0200] A scoring update module, used to update the protocol score and perform the next round of iteration;
[0201] An output module, configured to stop iteration when the number of iterations reaches a preset threshold, and obtain a feature vector corresponding to the sample data based on the second output of the capsule network;
[0202] A prediction module, used for predicting a first classification result corresponding to the sample data based on the feature vector;
[0203] A parameter updating module is used to update various parameters of the detection model according to the first classification result and the actual classification result corresponding to the sample data.
[0204] Optionally, the device further comprises:
[0205] The output updating module is used to update the first output of each capsule in the capsule network based on the self-attention mechanism after the first round of iteration is completed, and perform the next round of iteration.
[0206] Optionally, the update module includes:
[0207] A transposition module, used to perform a transposition operation on the second output of the capsule network;
[0208] A product module, used to calculate the product of the transposed second output and the first output of each capsule to obtain a first operation result;
[0209] A first superposition module, configured to superimpose the first operation result on the protocol score to obtain an updated protocol score;
[0210] The iteration module is used to perform the next round of iteration according to the updated agreement score.
[0211] Optionally, the update output module includes:
[0212] an encoding module, configured to encode the first output by using a self-attention mechanism to obtain a sequence feature corresponding to the first output;
[0213] The second superposition module is used to superimpose the sequence feature on the first output of each capsule to obtain an updated first output.
[0214] In summary, the sample generation device provided by the present application can obtain an initial data set; the initial data set includes a first sample and a second sample; based on the first sample, at least one first hypersphere is constructed; when the second sample is included in the coverage of the first hypersphere, the first hypersphere is reconstructed until the second sample is not included in the coverage of the first hypersphere; when the second sample is not included in the coverage of the first hypersphere, a first data set is generated based on the first sample and the first hypersphere, so that the number of samples of different categories in the data set used for training the model is more balanced. The initial data set and the first data set are input into the generative adversarial network to generate a target data set, and the generated samples are further optimized to improve the training effect of the detection module. The detection model is trained using the target data set, and the detection model after training is tested to improve the training effect of the detection model, thereby improving the accuracy of the detection model in identifying the sample data category.
[0215] The sample generation device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or it can be other devices other than the terminal. Exemplarily, the electronic device can be a GPU BOX, a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.
[0216] The sample generation device provided in the embodiment of the present application can achieve Figure 1 To avoid repetition, the various processes implemented by the method embodiment are not described here.
[0217] Alternatively, if Figure 3As shown, an embodiment of the present application also provides an electronic device, including a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the various steps of the above-mentioned sample generation method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.
[0218] In an embodiment of the present application, the memory may be used to store software programs and various data. The memory may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory may include a volatile memory or a non-volatile memory, or the memory may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory in the embodiments of the present application includes but is not limited to these and any other suitable types of memory.
[0219] The processor may include one or more processing units; optionally, the processor integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to the operating system, user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor.
[0220] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned sample generation method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.
[0221] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.
[0222] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the sample generation method embodiment described above, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0223] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises one..." does not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the method and device in the embodiment of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted or combined. In addition, the features described with reference to certain examples may be combined in other examples.
[0224] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the relevant technology, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.
[0225] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.
Claims
1. A sample generation method, characterized in that: The method comprises: Acquire an initial data set; the initial data set includes a first sample and a second sample; the number of the first samples is less than or equal to a first preset threshold; Based on the first sample, construct at least one first hypersphere; In a case where the second sample is included in the coverage of the first hypersphere, reconstructing the first hypersphere until the second sample is no longer included in the coverage of the first hypersphere; When the second sample is not included in the coverage of the first hypersphere, generating a first data set based on the first sample and the first hypersphere; Inputting the initial data set and the first data set into a generative adversarial network to generate a target data set; The target data set is used to train the detection model, and the trained detection model is tested.
2. The method according to claim 1, characterized in that: The step of constructing at least one first hypersphere based on the first sample comprises: For each first sample, calculating a first distance between the first sample and each sample in the initial data set; Adding samples in the initial data set whose first distance to the first sample meets a preset condition to a candidate sample set; A first hypersphere is constructed based on the first sample and samples in the candidate sample set.
3. The method according to claim 1, characterized in that: The step of generating a first data set based on the first sample and the first hypersphere when the second sample is not included in the coverage of the first hypersphere comprises: Generate a random number sequence; the square of each random number in the random number sequence is less than 1; A first data set is generated based on the first sample, the random number sequence, and the radius of the first hypersphere; the first data set includes at least one data subset; and one first sample corresponds to one data subset.
4. The method according to claim 1, characterized in that: The using the target data set to train the detection model and testing the trained detection model includes: For a sample data matrix corresponding to each sample data in the target data set, the sample data matrix is input into a capsule network, and based on each capsule in the capsule network, a first output of each capsule is obtained; In each round of iteration, the coupling coefficient of the dynamic routing mechanism is determined according to the protocol score, and the number of iterations is recorded; the protocol score is used to indicate the relevance of the sample data in the capsule network; Determining a second output of the capsule network based on the first output of each capsule in the capsule network and a coupling coefficient of the dynamic routing mechanism; Updating the protocol score and performing the next round of iteration; When the number of iterations reaches a preset threshold, the iteration is stopped, and a feature vector corresponding to the sample data is obtained based on the second output of the capsule network; Based on the feature vector, predict a first classification result corresponding to the sample data; According to the first classification result and the actual classification result corresponding to the sample data, various parameters of the detection model are updated.
5. The method according to claim 4, characterized in that The method further comprises: After the first round of iteration is completed, the first output of each capsule in the capsule network is updated based on the self-attention mechanism, and the next round of iteration is performed.
6. The method according to claim 4, characterized in that The updating of the protocol score and the next round of iteration include: Performing a transposition operation on the second output of the capsule network; Calculate the product of the transposed second output and the first output of each capsule to obtain a first operation result; Adding the first operation result to the protocol score to obtain an updated protocol score; The next iteration is performed based on the updated agreement score.
7. The method according to claim 4, characterized in that After the first round of iteration is completed, the first output of each capsule in the capsule network is updated based on the self-attention mechanism, and the next round of iteration is performed based on the updated first output, including: Encoding the first output by using a self-attention mechanism to obtain a sequence feature corresponding to the first output; The sequence feature is superimposed on the first output of each capsule to obtain an updated first output.
8. A sample generating device, characterized in that: The device comprises: An acquisition module, configured to acquire an initial data set; the initial data set includes a first sample and a second sample; the number of the first samples is less than or equal to a first preset threshold; A first construction module, configured to construct at least one first hypersphere based on the first sample; A second construction module is used for reconstructing the first hypersphere when the second sample is included in the coverage of the first hypersphere, until the second sample is no longer included in the coverage of the first hypersphere; A first generating module, configured to generate a first data set based on the first sample and the first hypersphere when the second sample is not included in the coverage of the first hypersphere; A second generating module, configured to input the initial data set and the first data set into a generative adversarial network to generate a target data set; The training module is used to train the detection model using the target data set and test the detection model after the training is completed.
9. An electronic device, characterized in that: The electronic device includes a processor, a memory, a communication interface and a communication bus, and the processor, the memory and the communication interface communicate with each other through the communication bus; the memory is used to store executable instructions, and the executable instructions enable the processor to execute the sample generation method according to any one of claims 1 to 7.
10. A readable storage medium, characterized in that: The readable storage medium stores a program or an instruction, and when the program or the instruction is executed by a processor, the sample generation method according to any one of claims 1 to 7 is implemented.