Industrial internet anomaly detection method based on deep generative adversarial network

Abstract

The invention relates to the field of data supervision and detection, and provides an industrial internet anomaly detection method based on a deep generative adversarial network, which is characterized by comprising the following steps: S10, preprocessing an intrusion detection data set based on a numerical value type to obtain a real sample; s20, a DeepGAN sampling model is trained; and S30, outputting a sampling result and carrying out anomaly detection on the sampling result based on the detection model. According to the oversampling method based on the deep generative adversarial network, the problem that the performance of a detector is reduced due to data class imbalance is solved, an existing oversampling method is utilized, the oversampling method based on the deep generative adversarial network is designed, the model training process is improved, and the quality of generated samples is improved. And finally, the detection performance of the detection model is effectively improved.

Description

Technical Field

[0001] The present invention belongs to the field of data supervision detection, and in particular relates to an industrial Internet anomaly detection method based on deep generative adversarial networks. Background Art

[0002] Intrusion Detection System (IDS) is an active defense technology in the field of network security. IDS can identify known attack patterns, abnormal behaviors, or unauthorized activities by collecting and analyzing information from network traffic, system logs, and application data. An effective IDS can alert administrators in a timely manner and provide detailed information about the threat so that countermeasures can be taken quickly to reduce security risks and potential losses. In the past few decades, many studies have taken the approach of combining traditional machine learning and deep learning to apply to IDS.

[0003] However, in network intrusion detection, the dataset is usually class-imbalanced, with the number of benign samples (i.e., majority class) far exceeding the number of malicious samples (i.e., minority class). Due to the scarcity of malicious samples and the serious consequences of their misclassification, it becomes crucial to accurately detect these minority class samples. However, class-imbalanced datasets pose challenges to the training of detection classifiers, which may cause minority class samples to be misclassified or ignored, thereby reducing the accuracy of minority class classification. Therefore, strengthening the classification ability of minority class samples and improving the overall classification accuracy are important challenges facing intrusion detection systems. Summary of the invention

[0004] In view of the above-mentioned defects of the prior art, the present invention proposes an industrial Internet anomaly detection method based on a deep generative adversarial network. The technical solution steps designed by the present invention include:

[0005] S10: preprocess the intrusion detection dataset based on the numerical type to obtain real samples;

[0006] S20: Train the DeepGAN sampling model;

[0007] S30: Output the sampling results and perform anomaly detection on the sampling results based on the detection model.

[0008] Preferably, the S10 includes:

[0009] The intrusion detection data set includes numerical data and non-numerical data;

[0010] The formula for scaling numerical data is as follows:

[0011]

[0012] In the formula, x is the eigenvalue of the numerical data, x′ is the eigenvalue of the numerical data after standardization, and x max is the maximum value of the characteristic of numerical data, x min It is the minimum characteristic value of numerical data;

[0013] One-hot encoding is used for each column of non-numeric data.

[0014] Preferably, the S10 further includes:

[0015] Divide the intrusion detection data set, divide 70% of the data volume of the intrusion detection data set into a training data set, and divide the remaining 30% of the data volume of the intrusion detection data set into a test data set, so as to obtain real samples of the training data set and real samples of the test data set.

[0016] Preferably, the S10 further includes:

[0017] Calculate the imbalance rate of the training data set and the test data set respectively. The formula is as follows:

[0018]

[0019] Where IR is the imbalance rate, N majority is the number of majority classes, N minority is the number of minority classes;

[0020] Based on the imbalance ratio of the training dataset and the test dataset, (N majority -N minority ) minority class samples.

[0021] Preferably, the DeepGAN sampling model includes:

[0022] Generator, discriminator, auxiliary classifier and information extraction module;

[0023] The generator is used to input a real sample and output a generated sample;

[0024] The discriminator is used to distinguish between real samples and generated samples;

[0025] The auxiliary classifier is used to predict the category label of a given sample and correspond the generated sample to the category label of the given sample;

[0026] The information extraction module is used to perform feature processing on the hidden layer outputs of the generator, the discriminator and the auxiliary classifier.

[0027] Preferably, the S20 includes the following steps:

[0028] S2001: The discriminator is fixed to train the generator. The generator adjusts the weight of the generator by minimizing the judgment error of the discriminator on the generated samples.

[0029] S2002: The generator is fixed to train the discriminator. The discriminator adjusts the weight of the discriminator by minimizing the Wasserstein distance between the real sample and the generated sample.

[0030] S2003: Add the loss of the auxiliary classifier to the loss of the generator, predict the category label of the given sample, and correspond the generated sample to the category label of the given sample;

[0031] S2004: The generator loss and the discriminator loss continuously optimize and adjust their respective weights through back propagation until the discriminator cannot determine the source of the real sample, completing the training of the DeepGAN sampling model.

[0032] Preferably, the generator is used to input a real sample and output a generated sample, including:

[0033] The generator inputs 30-dimensional random noise z and category label y, and extracts features through the hidden layer. The information extraction module further extracts the dependencies between time based on the features extracted in the hidden layer. The extracted data results are used as the input of the subsequent output layer.

[0034] In the output layer, the generator is divided into numerical vectors and non-numerical vectors according to the numerical type; for numerical vector output, the generator directly generates numerical output through a linear layer; for non-numerical vector output, the generator sets a separate dense layer for each non-numerical column so that each non-numerical column can generate independent output; the numerical vector output and the non-numerical vector output are concatenated to output a complete generated sample.

[0035] Preferably, the dense layer uses a Gumbel-Softmax activation function to add noise from a Gumbel distribution to the non-numerical vector output. For a vector x representing the non-normalized logarithmic probabilities of k categories of non-numerical vectors, Gumbel-Softmax is applied to each element x of the vector x. i , the formula is as follows:

[0036]

[0037] In the formula, g1,…,g k are independent and identically distributed samples drawn from Gumbel(0,1), and τ is the temperature coefficient.

[0038] Preferably, the discriminator is used to distinguish between real samples and generated samples and includes:

[0039] The input of the discriminator is the category label y and the generated sample, and the classification is performed according to the numerical type of the generated sample; for the numerical vector, the discriminator adds Gaussian noise with a mean of 0, representing the m numerical vectors x of the data sample, and adds noise to each vector xi, so that where z1,z2...z m Obtained from N(0, 0.01), noise is added to the real sample and the generated sample respectively; for non-numerical vectors, the non-numerical vectors are processed by a separate embedding layer for dimensionality reduction, and the dimension is dynamically adjusted to dimension d according to the original dimension k of one-hot, dimension d = min([k / 3], 20);

[0040] The numerical vector and non-numerical vector are taken as the input of the hidden layer of the discriminator. The numerical vector and non-numerical vector are learned based on nonlinear transformation, and the global information is integrated through the information extraction module to extract the features that distinguish the real samples from the generated samples. The output layer outputs the Wasserstein distance between the real samples and the generated samples.

[0041] Preferably, the information extraction module is used to perform feature processing on the hidden layer outputs of the generator, the discriminator and the auxiliary classifier, including:

[0042] The information extraction module includes a position encoder and a Transformer encoder, which are applied after the hidden layers of the generator, the discriminator and the auxiliary classifier. The data input to the information extraction module is passed through multiple sets of weight matrices W Q , W K and W V Perform linear changes to generate query vector Q, key vector K and value vector V. T Dot product calculation to calculate the similarity relationship between different features;

[0043] The Q and K T The dot product calculation formula is as follows:

[0044]

[0045] Beneficial effects: The present invention proposes an industrial Internet anomaly detection method based on a deep generative adversarial network. Aiming at the problem of detector performance degradation caused by data class imbalance, the present invention utilizes the existing oversampling method to design an oversampling method based on a deep generative adversarial network, introduces a Transformer-based information extraction module to help the model extract global information, so as to fully learn the distribution characteristics of minority class data, and takes into account that the intrusion detection data set contains both numerical variables and non-numerical variables, adopts a differentiated processing strategy to ensure that the model can effectively model these two types of variables at the same time. Finally, the loss functions of WGAN-GP and CGAN are combined, and auxiliary classifier loss and Relativistic Loss are introduced to improve the model training process and enhance the quality of generated samples. Finally, the detection performance of the detection model is effectively improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a flow chart of a preferred embodiment of the present invention;

[0047] Figure 2 It is a schematic diagram of the generator structure of a preferred embodiment of the present invention;

[0048] Figure 3 1 is a schematic diagram of the discriminator structure of a preferred embodiment of the present invention;

[0049] Figure 4 It is a schematic diagram of the structure of an auxiliary classifier of a preferred embodiment of the present invention;

[0050] Figure 5 It is a schematic diagram of the information extraction module structure of a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0051] The embodiments of the present invention are described in detail below. The following embodiments are implemented on the premise of the technical solution of the present invention, and detailed implementation methods and specific operation processes are given, but the protection scope of the present invention is not limited to the following embodiments.

[0052] The present invention designs an industrial Internet anomaly detection method based on deep generative adversarial network, such as Figure 1 As shown, the technical solution includes the following steps, specifically including:

[0053] S10: preprocess the intrusion detection dataset based on the numerical type to obtain real samples;

[0054] S20: Train the DeepGAN sampling model;

[0055] S30: Output the sampling results and perform anomaly detection on the sampling results based on the detection model.

[0056] Preferably, the S10 includes:

[0057] The intrusion detection data set includes numerical data and non-numerical data;

[0058] The formula for scaling numerical data is as follows:

[0059]

[0060] In the formula, x is the eigenvalue of the numerical data, x′ is the eigenvalue of the numerical data after standardization, and x max is the maximum value of the characteristic of numerical data, x min It is the minimum characteristic value of numerical data;

[0061] One-hot encoding is used for each column of non-numeric data.

[0062] Specifically, industrial control system data includes non-numeric data such as "service", "state", and "proto". The GAN model cannot effectively process non-numeric data, mainly because the GAN model generator and discriminator require data to be differentiable and cannot directly generate non-numeric data. For non-numeric data, one-hot encoding technology is used for each column of non-numeric data, and the softmax activation method is used in the generator. In this way, the processed data is not only easy to calculate the distance between features, but also retains the information of the original non-numeric data attributes.

[0063] Preferably, the S10 further includes:

[0064] Divide the intrusion detection data set, divide 70% of the data volume of the intrusion detection data set into a training data set, and divide the remaining 30% of the data volume of the intrusion detection data set into a test data set, so as to obtain real samples of the training data set and real samples of the test data set.

[0065] Preferably, the S10 further includes:

[0066] Calculate the imbalance rate of the training data set and the test data set respectively. The formula is as follows:

[0067]

[0068] Where IR is the imbalance rate, N majority is the number of majority classes, N minority is the number of minority classes;

[0069] Based on the imbalance ratio of the training dataset and the test dataset, (N majority -N minority ) minority class samples.

[0070] Specifically, the intrusion detection dataset of this application uses 7 public industrial control system network security datasets, namely NSL-KDD, Firewall, UNSW-NB15, Power, SWaT, Gas Pipeline, and BATADAL. The detailed information of the dataset is shown in Table 1. The dataset is divided into 70% training datasets for the training process of the detection model and 30% test datasets for evaluating the performance of the model. After data division, both datasets will maintain the original class imbalance. The imbalance rate (IR) can reflect the degree of class imbalance of the dataset.

[0071] Table 1

[0072]

[0073] Among them, Instances is the total number of samples in the data set, Features is the number of data sample features, #Maj is the number of majority samples, #Min is the number of minority samples, #Num is the number of numerical sample features, #Non-num is the number of non-numerical sample features, and IR is the imbalance ratio.

[0074] Preferably, the DeepGAN sampling model includes:

[0075] Generator, discriminator, auxiliary classifier and information extraction module;

[0076] The generator is used to input a real sample and output a generated sample;

[0077] The discriminator is used to distinguish between real samples and generated samples;

[0078] The auxiliary classifier is used to predict the category label of a given sample and correspond the generated sample to the category label of the given sample;

[0079] The information extraction module is used to perform feature processing on the hidden layer outputs of the generator, the discriminator and the auxiliary classifier.

[0080] Preferably, the S20 includes the following steps:

[0081] S2001: The discriminator is fixed to train the generator. The generator adjusts the weight of the generator by minimizing the judgment error of the discriminator on the generated samples.

[0082] S2002: The generator is fixed to train the discriminator. The discriminator adjusts the weight of the discriminator by minimizing the Wasserstein distance between the real sample and the generated sample.

[0083] S2003: Add the loss of the auxiliary classifier to the loss of the generator, predict the category label of the given sample, and correspond the generated sample to the category label of the given sample;

[0084] S2004: The generator loss and the discriminator loss continuously optimize and adjust their respective weights through back propagation until the discriminator cannot determine the source of the real sample, completing the training of the DeepGAN sampling model.

[0085] Preferably, if Figure 2 As shown, the generator is used to input real samples and output generated samples including:

[0086] The generator inputs 30-dimensional random noise z and category label y, and extracts features through the hidden layer. The information extraction module further extracts the dependencies between time based on the features extracted from the hidden layer. The extracted data results are used as the input of the subsequent output layer.

[0087] In the output layer, the generator is divided into numerical vectors and non-numerical vectors according to the numerical type; for numerical vector output, the generator directly generates numerical output through a linear layer; for non-numerical vector output, the generator sets a separate dense layer for each non-numerical column so that each non-numerical column can generate independent output; the numerical vector output and the non-numerical vector output are concatenated to output a complete generated sample.

[0088] Preferably, the dense layer uses a Gumbel-Softmax activation function to add noise from a Gumbel distribution to the non-numerical vector output. For a vector x representing the non-normalized logarithmic probabilities of k categories of non-numerical vectors, Gumbel-Softmax is applied to each element x of the vector x. i , the formula is as follows:

[0089]

[0090] In the formula, g1,…,g k are independent and identically distributed samples drawn from Gumbel(0,1), and τ is the temperature coefficient.

[0091] Specifically, by adding noise from the Gumbel distribution to non-numeric outputs, the generator does not simply copy the input noise, but is able to generate discrete values ​​close to the real data distribution.

[0092] Preferably, if Figure 3 As shown, the discriminator is used to distinguish between real samples and generated samples and includes:

[0093] The input of the discriminator is the category label y and the generated sample, and the classification is performed according to the numerical type of the generated sample; for the numerical vector, the discriminator adds Gaussian noise with a mean of 0, representing the m numerical vectors x of the data sample, and adds noise to each vector xi, so that where z1,z2...z m Obtained from N(0, 0.01), noise is added to the real sample and the generated sample respectively; for non-numerical vectors, the non-numerical vectors are processed by a separate embedding layer for dimensionality reduction, and the dimension is dynamically adjusted to dimension d according to the original dimension k of one-hot, dimension d = min([k / 3], 20);

[0094] The numerical vector and non-numerical vector are taken as the input of the hidden layer of the discriminator. The numerical vector and non-numerical vector are learned based on nonlinear transformation, and the global information is integrated through the information extraction module to extract the features that distinguish the real samples from the generated samples. The output layer outputs the Wasserstein distance between the real samples and the generated samples.

[0095] Specifically, adding noise to real samples can ensure that the discriminator does not make judgments based solely on precise values ​​that are different from real data. Adding noise to generated samples can ensure that the generator does not learn to imitate noise and strengthen the output of values ​​that are the same as the real data distribution. The role of the embedding layer is to convert the high-dimensional sparse one-hot encoding into a low-dimensional dense representation, which not only reduces the amount of calculation and the number of parameters, but also helps to extract the potential relationship between features. The dimension is dynamically adjusted according to the original dimension k of the one-hot vector to ensure that the dimension d = min ([k / 3], 20). Based on experience, 20 dimensions are sufficient to capture the main features of the data. The non-numerical vector part and the numerical vector part processed by the embedding layer are used as the input of the hidden layer together. In the hidden layer, the data will undergo a series of nonlinear transformations to learn the complex patterns of the data. Through the information extraction module, the global information is further integrated to extract features that are useful for distinguishing real samples from generated samples. Finally, the output layer will output a one-dimensional scalar, which represents the probability that the input data is a real sample (actually an approximation of the Wasserstein distance). Since the goal of DeepGAN is to directly optimize the Wasserstein distance rather than perform probability estimation, no activation function is required in the output layer.

[0096] In addition, if Figure 4 As shown, the auxiliary classifier is used to predict the category label of a given sample and correspond the generated sample to the category label of the given sample, including:

[0097] When the auxiliary classifier processes data samples, its input processing flow is consistent with that of the discriminator. Its structure consists of two fully connected layers, both of which use LeaklyRelu as the activation function. After passing through these two fully connected layers, the data will flow into the information extraction module for further feature processing. Finally, the data passes through an output layer, where Sigmoid is used as the activation function to generate a class probability. This class probability represents the probability that a given sample X belongs to category y, AC(X) = P(y|X). This design aims to encourage the generator to generate samples belonging to category y through the prediction ability of the auxiliary classifier, thereby improving the generator's generation ability.

[0098] Preferably, if Figure 5 As shown, the information extraction module is used to perform feature processing on the hidden layer outputs of the generator, the discriminator and the auxiliary classifier, including:

[0099] The information extraction module includes a position encoder and a Transformer encoder, which are applied after the hidden layers of the generator, the discriminator and the auxiliary classifier. The data input to the information extraction module is passed through multiple sets of weight matrices W Q , W K and W V Perform linear changes to generate query vector Q, key vector K and value vector V. T Dot product calculation to calculate the similarity relationship between different features;

[0100] The Q and K T The dot product calculation formula is as follows:

[0101]

[0102] Specifically, data sampling based on the DeepGAN model mainly includes a generator, a discriminator, an auxiliary classifier, and an information extraction module embedded in the generator and the discriminator. After the information extraction module is applied to the hidden layers of the generator, the discriminator, and the auxiliary classifier, the output of the hidden layer is further processed, so that the model can effectively capture the dependencies and important features between time.

[0103] In addition, in the information extraction module, the Transformer encoder part introduces position encoding to explicitly represent the position of each element in the sequence. Considering the temporal nature of industrial control system data, the position encoder is combined with the Transformer encoder in the DeepGAN architecture to form an information extraction module. The core of the Transformer's working principle lies in the multi-head attention mechanism, which uses scaled dot product attention to capture the key dependencies in the input data. First, the input data is passed through multiple sets of weight matrices W. Q ,WK and W V Perform linear changes to generate Q (query), K (key) and V (value). Then, use the formula to calculate the value of Q and K. T The dot product calculation can calculate the similarity relationship between different features. In order to prevent the dot product result from being too large and causing the Softmax method to enter the saturation zone, the dot product result is divided by a scaling factor, which is usually the square root of the key feature dimension. Then the scaled dot product matrix is ​​normalized by the Softmax method to obtain the attention weight. According to the formula attention weight, the value matrix is ​​weighted summed to obtain the output head of each attention head j The outputs of multiple attention heads are concatenated using the formula, and the entire multi-head attention output is calculated through linear transformation.

[0104] Among them, the attention weight performs weighted summation on the value matrix, and the formula is as follows:

[0105] head i =Attention(QW i Q , KW i K , VW i V )

[0106] The outputs of multiple attention heads are concatenated using the formula:

[0107] MultiHead(Q,K,V)=Concat(head1,...,head n )

[0108] The output obtained by the multi-head attention mechanism cannot be directly passed to the fully connected neural network, but will be processed by the Add&Norm layer first. The Add&Norm layer consists of two parts, Add and Norm. The Add part is to add the residual block X on the basis of the original input. The purpose of adding the residual block is to prevent the problem of degradation during the training of the deep neural network. The Norm part is mainly for normalization, the purpose is to speed up the training speed of the model and improve the stability of the training. Its calculation formula is as follows:

[0109] LayerNorm(X+MultiHeadAttention(X))

[0110] LayerNorm(X+FeedForward(X))

[0111] Next, the output after processing by the Add&Norm layer is passed to the FeedForward network. FeedForward is a two-layer fully connected neural network that first performs a linear transformation, then applies the Relu nonlinear activation function, and finally performs a linear transformation again so that the model can learn more complex feature representations. Through the filtering effect of the Relu function, the model can selectively retain important feature information and ignore irrelevant parts. Finally, after the transformation of the FeedForward network, the model will get a new output.

[0112] Preferably, the sampling results are output in S30 and anomaly detection is performed on the sampling results based on a detection model, including four common detection models: support vector machine (SVM), XGBoost, random forest, LightGBM, and fully connected neural network (FCNN).

[0113] Preferably, the present invention also provides evaluation index design, including evaluating the performance of the DeepGAN method using F-measure (Table 2) and G-mean index (Table 3) for evaluating the anomaly detection performance. In order to verify the superiority of the experimental results, the performance of DeepGAN is compared with SMOTE, Borderline-SMOTE, K-means SMOTE, Geometric SMOTE, GAN, CWGAN and no sampling method.

[0114] Table 2

[0115]

[0116]

[0117] Table 3

[0118]

[0119]

[0120] The preferred specific embodiments of the present invention are described in detail above. It should be understood that ordinary technicians in the field can make many modifications and changes based on the concept of the present invention without creative work. Therefore, all technical solutions that can be obtained by technicians in the technical field based on the concept of the present invention through logical analysis, reasoning or limited experiments on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. An industrial Internet anomaly detection method based on deep generative adversarial network, characterized in that: The following steps are involved: S10: preprocess the intrusion detection dataset based on the numerical type to obtain real samples; S20: Train the DeepGAN sampling model; S30: Output the sampling results and perform anomaly detection on the sampling results based on the detection model.

2. According to claim 1, the industrial Internet anomaly detection method based on deep generative adversarial network is characterized in that: The S10 includes: The intrusion detection data set includes numerical data and non-numerical data; The formula for scaling numerical data is as follows: In the formula, x is the eigenvalue of the numerical data, x′ is the eigenvalue of the numerical data after standardization, and x max is the maximum value of the characteristic of numerical data, x min It is the minimum characteristic value of numerical data; One-hot encoding is used for each column of non-numeric data.

3. According to the industrial Internet anomaly detection method based on deep generative adversarial network according to claim 1, it is characterized in that: The S10 further includes: Divide the intrusion detection data set, divide 70% of the data volume of the intrusion detection data set into a training data set, and divide the remaining 30% of the data volume of the intrusion detection data set into a test data set, so as to obtain real samples of the training data set and real samples of the test data set.

4. According to claim 3, the industrial Internet anomaly detection method based on deep generative adversarial network is characterized in that: The S10 further includes: Calculate the imbalance rate of the training data set and the test data set respectively. The formula is as follows: Where IR is the imbalance rate, N majority is the number of majority classes, N minority is the number of minority classes; Based on the imbalance ratio of the training dataset and the test dataset, (N majority -N minority ) minority class samples.

5. According to claim 2, the method for industrial Internet anomaly detection based on deep generative adversarial network is characterized in that: The DeepGAN sampling model includes: Generator, discriminator, auxiliary classifier and information extraction module; The generator is used to input a real sample and output a generated sample; The discriminator is used to distinguish between real samples and generated samples; The auxiliary classifier is used to predict the category label of a given sample and correspond the generated sample to the category label of the given sample; The information extraction module is used to perform feature processing on the hidden layer outputs of the generator, the discriminator and the auxiliary classifier.

6. The industrial Internet anomaly detection method based on deep generative adversarial network according to claim 5 is characterized in that: The S20 includes the following steps: S2001: The discriminator is fixed to train the generator. The generator adjusts the weight of the generator by minimizing the judgment error of the discriminator on the generated samples. S2002: The generator is fixed to train the discriminator. The discriminator adjusts the weight of the discriminator by minimizing the Wasserstein distance between the real sample and the generated sample. S2003: Add the loss of the auxiliary classifier to the loss of the generator, predict the category label of the given sample, and correspond the generated sample to the category label of the given sample; S2004: The generator loss and the discriminator loss continuously optimize and adjust their respective weights through back propagation until the discriminator cannot determine the source of the real sample, completing the training of the DeepGAN sampling model.

7. The industrial Internet anomaly detection method based on deep generative adversarial network according to claim 5 is characterized in that: The generator is used to input real samples and output generated samples, including: The generator inputs 30-dimensional random noise z and category label y, and extracts features through the hidden layer. The information extraction module further extracts the dependencies between time based on the features extracted from the hidden layer. The extracted data results are used as the input of the subsequent output layer. In the output layer, the generator is divided into numerical vectors and non-numerical vectors according to the numerical type; for numerical vector output, the generator directly generates numerical output through a linear layer; for non-numerical vector output, the generator sets a separate dense layer for each non-numerical column so that each non-numerical column can generate independent output; the numerical vector output and the non-numerical vector output are concatenated to output a complete generated sample.

8. The industrial Internet anomaly detection method based on deep generative adversarial network according to claim 7 is characterized in that: The dense layer uses the Gumbel-Softmax activation function to add noise from the Gumbel distribution to the non-numeric vector output. For a vector x representing the non-normalized log-probability of k-class non-numeric vectors, Gumbel-Softmax is applied to each element of the vector x. i , the formula is as follows: In the formula, g1,…,g k are independent and identically distributed samples drawn from Gumbel(0,1), and τ is the temperature coefficient.

9. The industrial Internet anomaly detection method based on deep generative adversarial network according to claim 5 is characterized in that: The discriminator is used to distinguish between real samples and generated samples and includes: The input of the discriminator is the category label y and the generated sample, and the classification is performed according to the numerical type of the generated sample; for the numerical vector, the discriminator adds Gaussian noise with a mean of 0, representing the m numerical vectors x of the data sample, and adds noise to each vector xi, so that where z1,z2...z m Obtained from N(0, 0.01), noise is added to the real sample and the generated sample respectively; for non-numerical vectors, the non-numerical vectors are processed by a separate embedding layer for dimensionality reduction, and the dimension is dynamically adjusted to dimension d according to the original dimension k of one-hot, dimension d = min([k / 3], 20); The numerical vector and non-numerical vector are taken as the input of the hidden layer of the discriminator. The numerical vector and non-numerical vector are learned based on nonlinear transformation, and the global information is integrated through the information extraction module to extract the features that distinguish the real samples from the generated samples. The output layer outputs the Wasserstein distance between the real samples and the generated samples.

10. The industrial Internet anomaly detection method based on deep generative adversarial network according to claim 5 is characterized in that: The information extraction module is used to perform feature processing on the hidden layer outputs of the generator, the discriminator and the auxiliary classifier, including: The information extraction module includes a position encoder and a Transformer encoder, which are applied after the hidden layers of the generator, the discriminator and the auxiliary classifier. The data input to the information extraction module is passed through multiple sets of weight matrices W Q , W T and W V Perform linear changes to generate query vector Q, key vector K and value vector V. T Dot product calculation to calculate the similarity relationship between different features; The Q and K T The dot product calculation formula is as follows:

Images