Industrial equipment anomaly detection method and system based on skip-GANopen

Abstract

The invention relates to the field of industrial equipment anomaly detection, in particular to an industrial equipment anomaly detection method and system based on ski-GANopen, and the method comprises the following steps: obtaining a data set, preprocessing the data set, and dividing the preprocessed data set to obtain a training set and a test set; constructing a deep learning model based on ski-GANopen, and constructing a deep learning model based on the ski-GANopen; training the deep learning model on the training set, and adjusting model parameters to obtain a trained deep learning model; and inputting the test set into the trained deep learning model for testing to obtain a detection result, and evaluating the detection result. According to the detection method and system provided by the invention, the abnormity of the industrial equipment can be detected and identified, so that a factory is further helped to detect the running state of the equipment, the loss caused by equipment faults is avoided, and the production efficiency is improved.

Description

Technical Field

[0001] The present application relates to the technical field of industrial anomaly detection, and in particular to a method and system for detecting anomalies of industrial equipment based on skip-GANomaly. Background Art

[0002] Since the intelligent manufacturing system (IMS) is large in scale and highly complex, it is susceptible to a variety of factors, so reliability and risk management are required. Anomaly detection is a technology that monitors the status of equipment in real time. It collects data to determine whether the equipment is working properly, detects faults in time, and provides relevant information for life prediction. Among them, surface defects are the most intuitive manifestation of equipment abnormalities, so the abnormality detection of equipment can be achieved by performing surface defect detection on key parts of the equipment. Surface defect detection refers to the detection of scratches, defects, foreign body occlusion, color pollution, holes and other defects on the surface of the sample, so as to obtain a series of relevant information such as the type, contour, position, size, etc. of the surface defects of the tested sample. Manual defect detection used to be the mainstream method, but this method is inefficient; the detection results are easily affected by human subjective factors and cannot meet the requirements of real-time detection. It has gradually been replaced by other methods.

[0003] Traditional anomaly detection methods automatically extract features and quantify anomalies by building neural networks. As systems become larger and more complex, the performance indicators that need to be monitored are becoming more diverse and large-scale. Traditional anomaly detection systems cannot adapt well to the needs, and deep learning-based system anomaly detection has become a research hotspot. Deep learning algorithms use neural networks to automatically learn features, and can extract and learn features of more complex data than traditional anomaly detection methods. With the advantages of strong learning ability and high adaptability, they are becoming more and more popular in the field of anomaly detection and have been applied to various tasks. Summary of the invention

[0004] The embodiments of the present application provide a method and system for detecting anomalies of industrial equipment based on skip-GANomaly, so as to solve the problems of insufficient detection accuracy and low maintenance efficiency existing in industrial anomaly detection.

[0005] To solve the above technical problems, in the first aspect, an embodiment of the present application provides an industrial equipment anomaly detection method based on skip-GANomaly, comprising the following steps: first, obtaining a data set, preprocessing the data set, and dividing the preprocessed data set into a training set and a test set; second, constructing a deep learning model based on skip-GANomaly; third, inputting the training set into the deep learning model for model training, adjusting the model parameters, and obtaining a trained deep learning model; finally, inputting the test set into the trained deep learning model for testing, obtaining detection results, and evaluating the detection results.

[0006] In some exemplary embodiments, the data set is preprocessed, including: randomly cropping the images in the data set; and performing data format processing on the cropped images to obtain a preprocessed data set.

[0007] In some exemplary embodiments, the skip-GANomaly-based deep learning model is an improved deep learning model with a skip-connected encoder-decoder network structure; the skip-GANomaly-based deep learning model learns data features through a generator and reconstructs an input image to obtain a generated image, compares and scores the generated image with the real image through a discriminator, and calculates an anomaly score based on the loss to obtain an anomaly detection result.

[0008] In some exemplary embodiments, a skip-GANomaly-based deep learning model is constructed, including: defining a generator network structure; the generator network structure includes an encoder unit and a decoder unit; connecting the encoder unit and the decoder unit in the generator network structure to obtain a skip connection network structure, and completing a forward propagation function to determine a flow path of data in the network; defining a discriminator network structure; completing a forward propagation function of the discriminator to determine a flow path of data in the network.

[0009] In some exemplary embodiments, the encoder unit G E It includes a two-dimensional convolution and four identical convolution layers to extract data features; the input of each convolution layer is set with a LeakyReLU activation function, and the output is set with batch normalization; the relationship between the input and output of the encoder unit is as follows:

[0010] z=G E (x)

[0011] Among them, G E is the encoder unit, x is the encoder unit G E The input of z is the encoder unit G E Output:

[0012] The calculation formula of the LeakyReLU activation function is as follows:

[0013] f(x)=max(0,x)+α·min(0,x),0<α<<1

[0014] Where x is the input value, and f(x) is the output of the LeakyReLU activation function. The LeakyReLU function introduces a very small slope α when the input is negative to prevent the gradient from disappearing. When the input is non-negative, the output is the same as the input.

[0015] Decoder unit G D It includes four deconvolution layers, three of which have the same structure, with ReLU activation function at the input and batch normalization at the output; the remaining deconvolution layer has Tanh activation function only at the output; the relationship between the input and output of the decoder unit is as follows:

[0016]

[0017] Among them, G D is the decoder unit, z is the decoder unit G D Input, is the decoder unit G D Output:

[0018] The calculation formula of the ReLU activation function is as follows:

[0019] f(x)=max(0,x)

[0020] Where x is the input value and f(x) is the output of the ReLU activation function. The ReLU function outputs 0 when the input is negative and the output is the same as the input when the input is non-negative.

[0021] The calculation formula of the Tanh activation function is as follows:

[0022]

[0023] Where x is the input value and Tanh(x) is the output of the Tanh activation function; the Tanh function converts its input to the (-1,1) interval;

[0024] The discriminator includes a two-dimensional convolution and four identical convolution layers, with a LeakyReLU activation function at its input and a batch normalization at its output; the last layer of the discriminator is composed of a two-dimensional convolution and a sigmoid activation function;

[0025] The calculation formula of the sigmoid activation function is as follows:

[0026]

[0027] Among them, x is the input value of the discriminator, sigmoid(x) is the output of the sigmoid function; the sigmoid activation function smoothly converts the input to between (0,1).

[0028] In some exemplary embodiments, the generator is optimized by a total loss function obtained by adding three loss functions according to certain weights. The three loss functions are adversarial loss function, contextual loss function and potential loss function. The calculation formula of the adversarial loss function is as follows:

[0029]

[0030] Among them, x represents the real image input, represents the fake image generated by the generator, and D represents the discriminator;

[0031] The calculation formula of the context loss function is as follows:

[0032]

[0033] Among them, x represents the real image input, Represents the fake image generated by the generator;

[0034] The potential loss function is calculated as follows:

[0035]

[0036] Among them, x represents the real image input, represents the fake graph generated by the generator, and f represents a function, which is equivalent to the last convolutional layer of the discriminator, for x and Perform feature extraction;

[0037] The calculation formula of the generator's total loss function is as follows:

[0038]

[0039] Among them, λ adv , con and λ lat Represent the weights of the three loss functions respectively.

[0040] In some exemplary embodiments, the test set is input into the trained deep learning model for testing to obtain the detection result, including: inputting the test set into the trained deep learning model for testing, calculating the anomaly score of each sample and then classifying and predicting, and evaluating the detection result; the calculation process of the anomaly score includes: first, for each input test image Calculate its anomaly score

[0041]

[0042] in, is a reconstruction score based on the contextual loss function, which is used to measure the contextual similarity between the input image and the generated image; represents the potential representation score based on the potential loss function, which is used to measure the difference between the input image and the generated image; λ is a weighting parameter that controls the relative importance of the score function; the abnormality score of all input test images is Composition vector a:

[0043]

[0044] Finally, the feature of vector A is scaled to between (0,1) to obtain the final anomaly score of each corresponding test image. The calculation formula is as follows:

[0045]

[0046] Among them, min(A) represents the minimum value in vector A, and max(A) represents the maximum value in vector A.

[0047] In some exemplary embodiments, the preprocessed test set data is input into the optimal network model for testing, and the test results are evaluated, including: drawing ROC curves and PRC curves based on the test results and the true results; calculating AUC, best_accuracy, best_threshold and best_F1_score based on the test results and the true results; evaluating the model based on the drawn ROC curve, PRC curve and the calculated AUC, best_accuracy, best_threshold, best_F1_score; the vertical axis of the coordinate system of the ROC curve is TPR (true positive rate), the maximum value is 1, and the horizontal axis is FPR (false positive rate), the maximum value is 1, and the calculation formulas of TPR and FPR are as follows:

[0048]

[0049] Among them, TP is the number of positive samples correctly identified, FN is the number of positive samples misclassified as negative samples, P is the number of positive samples, FP is the number of negative samples misclassified as positive samples, TN is the number of negative samples correctly identified, and N is the number of negative samples. The larger the area under the ROC curve, i.e., AUC, the higher the accuracy of the model.

[0050] The horizontal axis of the PRC curve coordinate system is the recall rate (Recall), the maximum value is 1, and the vertical axis is the precision rate (Precision), the maximum value is 1. The calculation formulas for the recall rate and precision rate are as follows:

[0051]

[0052] The meaning of each symbol is the same as described above. The larger the area under the PRC curve, i.e., the AP, the better the classification performance of the model; best_accuracy refers to the maximum value of the accuracy obtained in the test round. Accuracy refers to the ratio of the number of correct predictions in positive samples and negative samples to the total number. Its calculation formula is as follows:

[0053]

[0054] Best_F1_score refers to the maximum value of the F1_score obtained in the test round. F1_score is the harmonic mean of recall and precision. Its calculation formula is as follows:

[0055]

[0056] Best_threshold refers to the optimal threshold value, which is calculated here using the method provided by sklearn, that is, after calculating the F1_score corresponding to all possible thresholds, the threshold corresponding to the maximum F1_score is selected as the optimal threshold.

[0057] In the second aspect, an embodiment of the present application also provides an industrial equipment anomaly detection system based on skip-GANomaly, including: a data set acquisition module, a model construction module, a model training module, a model testing module and a detection evaluation module connected in sequence; the data set acquisition module is used to acquire a data set, preprocess the data set, and divide the preprocessed data set into a training set and a test set; the model construction module is used to build a deep learning model based on skip-GANomaly; the model training module is used to input the training set into the deep learning model for model training, adjust the model parameters, and obtain a trained deep learning model; the model testing module is used to input the test set into the trained deep learning model for testing to obtain detection results; the detection evaluation module is used to evaluate the detection results.

[0058] In some exemplary embodiments, the model building module includes a generator structure definition unit and a discriminator structure definition unit; the generator network structure is used to define the encoder and the decoder, and connect the encoder part and the decoder part to obtain a jump connection network structure, and complete the forward propagation function to determine the flow path of the data in the network; the discriminator structure definition unit is used to define the discriminator network structure, and complete the forward propagation function of the discriminator to determine the flow path of the data in the network; the input of the discriminator network structure is the true graph and the generator generated graph, and the final detection result is obtained.

[0059] The technical solution provided by the embodiments of the present application has at least the following advantages:

[0060] An embodiment of the present application provides an industrial equipment anomaly detection method based on skip-GANomaly, comprising the following steps: first, obtaining a data set, preprocessing the data set, and dividing the preprocessed data set into a training set and a test set; then, building a deep learning model based on skip-GANomaly; secondly, inputting the training set into the deep learning model for model training, adjusting the model parameters, and obtaining a trained deep learning model; finally, inputting the test set into the trained deep learning model for testing, obtaining a detection result, and evaluating the detection result.

[0061] The aero-engine prediction and health management solution provided by this application adopts a deep learning model based on skip-GANomaly, which brings a new method for industrial anomaly detection. First, by constructing a network architecture based on skip-GANomaly and introducing the structure of skip connection, this application can better capture the multi-scale distribution of normal data distribution in high-dimensional image space. Secondly, the use of adversarial training scheme in the model architecture can provide excellent reconstruction effects in high-dimensional image space and low-dimensional latent vector space encoding. Finally, this application uses evaluation indicators to compare with traditional network models to verify the effectiveness, accuracy and adjustability of the constructed anomaly detection model. The model provided in this application can accurately identify anomalies of industrial equipment, thereby intelligently assisting industrial production. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplifications do not constitute limitations on the embodiments. Unless otherwise stated, the pictures in the drawings do not constitute proportional limitations.

[0063] Figure 1 It is a flowchart of an industrial equipment anomaly detection method based on skip-GANomaly provided in one embodiment of the present application.

[0064] Figure 2It is a structural block diagram of an industrial equipment anomaly detection system based on skip-GANomaly provided in one embodiment of the present application.

[0065] Figure 3 This is a network structure diagram of a generator based on a skip-GANomaly model provided in an embodiment of the present application.

[0066] Figure 4 This is a network structure diagram of a discriminator in a skip-GANomaly model provided in an embodiment of the present application.

[0067] Figure 5 This is a schematic diagram of the result of the total loss of the generator during training provided by an embodiment of the present application changing with the increase of training rounds.

[0068] Figure 6 It is a schematic diagram showing the result of the discriminator training loss changing with the increase of training rounds provided in one embodiment of the present application.

[0069] FIG. 7A to FIG. 7B A schematic diagram of the evaluation index results of a grid category image using a skip-GANomaly-based industrial equipment anomaly detection method provided in an embodiment of the present application. DETAILED DESCRIPTION

[0070] As can be seen from the background technology, traditional anomaly detection systems cannot adapt well to the needs, and deep learning-based system anomaly detection has become a research hotspot. Deep learning algorithms use neural networks to automatically learn features, and can extract and learn features of more complex data than traditional anomaly detection methods. With the advantages of strong learning ability and high adaptability, it has become increasingly popular in the field of anomaly detection and has been applied to various tasks. In addition, existing industrial anomaly detection has problems such as insufficient detection accuracy and low maintenance efficiency.

[0071] In view of the shortcomings of the prior art, the embodiment of the present application provides an industrial equipment anomaly detection method and system based on skip-GANomaly, which includes the following steps: first, obtain a data set, the data set selected by the embodiment of the present application is an MVTec AD data set for unsupervised anomaly detection, and preprocess the data set, such as random cropping, to obtain a preprocessed data set; and divide the preprocessed data set into a training set and a test set; secondly, construct a deep learning model based on skip-GANomaly; then, input the training set into the deep learning model for model training, and optimize it using the Adam optimizer. After a preset number of training rounds, adjust the model parameters according to the results, such as the learning rate, batch size, and number of iterations, to achieve the optimal model and obtain a trained deep learning model; finally, input the test set into the trained deep learning model for testing, obtain the detection results, and evaluate the detection results. The purpose of this application is to provide an industrial equipment anomaly detection method and system based on skip-GANomaly, which can accurately identify the anomalies of industrial equipment, thereby intelligently assisting industrial production.

[0072] The following will describe the various embodiments of the present application in detail with reference to the accompanying drawings. However, it will be appreciated by those skilled in the art that in the various embodiments of the present application, many technical details are provided in order to enable the reader to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical solution claimed in the present application can be implemented.

[0073] See also Figure 1 , the embodiment of the present application provides an industrial equipment anomaly detection method based on skip-GANomaly, the method comprising the following steps:

[0074] Step S1, obtaining a data set, preprocessing the data set, and dividing the preprocessed data set into a training set and a test set.

[0075] Step S2: construct a deep learning model based on skip-GANomaly.

[0076] Step S3: input the training set into the deep learning model for model training, adjust the model parameters, and obtain a trained deep learning model.

[0077] Step S4: Input the test set into the trained deep learning model for testing to obtain the detection result.

[0078] The skip-GANomaly-based industrial equipment anomaly detection method provided in this application is improved based on the skip-GANomaly network model. The network model introduces the structure of skip connection, which can better capture the multi-scale distribution of normal data distribution in high-dimensional image space. Secondly, the use of adversarial training scheme in the model architecture can provide excellent reconstruction effects in high-dimensional image space and low-dimensional latent vector space encoding. Based on this, the model provided in this application can accurately identify anomalies of industrial equipment, thereby intelligently assisting industrial production.

[0079] In some embodiments, the data set is preprocessed in step S1, including: randomly cropping the images in the data set; and performing data format processing on the cropped images to obtain a preprocessed data set.

[0080] Specifically, the data set is obtained by obtaining pictures of key components during the operation of the device. The preprocessing process of the data set in step S1 mainly includes random cropping and data format processing. First, the image data is read from the memory cache and decoded into an image format, and then the image size is standardized to 256×256; secondly, each picture in the data set is randomly cropped into a set number of num sub-images, and the parameter num refers to the number of sub-images that each picture is cropped into, so as to further enrich the training data; finally, all pictures are converted to RGB format, and the number of channels is standardized to facilitate deep learning model training.

[0081] In some embodiments, the skip-GANomaly-based deep learning model in step S2 is an improved deep learning model with a skip-connected encoder-decoder network structure; the skip-GANomaly-based deep learning model learns data features through a generator and reconstructs an input image to obtain a generated image, compares and scores the generated image with the real image through a discriminator, and calculates an anomaly score based on the loss to obtain an anomaly detection result.

[0082] Specifically, in step S2, the skip-GANomaly-based deep learning model extracts features from the image through the generator encoder part, and then generates a new image through the generator decoder part. The discriminator is used to compare the similarity between the real image and the generated image. After training using only normal samples, the generator can only generate normal images. If there are anomalies in the real image, there will be a large reconstruction error between the real image and the generated image. After comparison with the discriminator, a higher anomaly score is obtained.

[0083] In some embodiments, step S2 constructs a skip-GANomaly-based deep learning model, including:

[0084] Step S201, define the generator network structure; the generator network structure consists of two parts: an encoder unit and a decoder unit.

[0085] Step S202: Connect the encoder unit and the decoder unit in the generator network structure to obtain a skip connection network structure, and complete the forward propagation function to determine the flow path of the data in the network.

[0086] Step S203: define the discriminator network structure.

[0087] Step S204: Complete the forward propagation function of the discriminator to determine the flow path of the data in the network.

[0088] See also Figure 3 , the generator G encoder unit G defined in step S201 E It includes a two-dimensional convolution and four identical convolution layers for extracting data features; the input end of each convolution layer is provided with a LeakyReLU activation function, and the output end is provided with batch normalization; the encoder unit G of the generator G E The input is x and the output is z, and the relationship is as follows:

[0089] z=G E (x)

[0090] The calculation formula of the LeakyReLU activation function is as follows:

[0091] f(x)=max(0,x)+α·min(0,x),0<α<<1

[0092] Where x is the input value and f(x) is the output of the LeakyReLU activation function. The LeakyReLU function introduces a very small slope α when the input is negative to prevent the gradient from disappearing. When the input is non-negative, the output is the same as the input.

[0093] In some embodiments, the generator G decoder unit (G D ) comprises four deconvolution layers, three of which have the same structure, with a ReLU activation function at the input end and a batch normalization at the output end; the remaining deconvolution layer has a Tanh activation function only at the output end; the decoder unit G of the generator G D The input is z(encoder(G E ), the output is The relationship is as follows:

[0094]

[0095] The calculation formula of the ReLU activation function is as follows:

[0096] f(x)=max(0,x)

[0097] Where x is the input value and f(x) is the output of the ReLU activation function. The ReLU function outputs 0 when the input is negative and the output is the same as the input when the input is non-negative.

[0098] The calculation formula of the Tanh activation function is as follows:

[0099]

[0100] Where x is the input value and Tanh(x) is the output of the Tanh activation function; the Tanh function converts its input to the (-1,1) interval.

[0101] See also Figure 4 The discriminator network structure defined in step S203, the discriminator D, includes a two-dimensional convolution, four identical convolution layers, a LeakyReLU activation function at its input, and batch normalization at its output. The last layer of the discriminator is composed of a two-dimensional convolution and a sigmoid activation function; the calculation formula of the sigmoid activation function is as follows:

[0102]

[0103] Here, x is the input value, sigmoid(x) is the output of the sigmoid function; the sigmoid activation function smoothly converts the input to between (0,1).

[0104] In some embodiments, in step S3, the training set is input into a skip-GANomaly-based deep learning model for model training. After a preset number of training rounds, the model parameters are adjusted according to the results to achieve model optimization and obtain a trained deep learning model.

[0105] See also Figure 5 , the generator G is optimized by the total loss function obtained by adding three loss functions according to certain weights. The three loss functions are adversarial loss function, context loss function and potential loss function.

[0106] The calculation formula of the adversarial loss function is as follows:

[0107]

[0108] Among them, x represents the real image input, represents the fake image generated by the generator, and D represents the discriminator, indicating that the generator is updated based on the inside of the discriminator. The change of the loss function of the discriminator can be found in Figure 6 .

[0109] The calculation formula of the context loss function is as follows:

[0110]

[0111] Among them, x represents the real image input, Represents the fake graph generated by the generator, which represents the pair x and Perform L1 normalization.

[0112] The calculation formula of the potential loss function is as follows:

[0113]

[0114] Among them, x represents the real image input, represents the fake graph generated by the generator, and f represents a function, which is equivalent to the last convolutional layer of the discriminator, for x and Perform feature extraction.

[0115] The calculation formula of the generator's total loss function is as follows:

[0116]

[0117] Among them, λ adv , con and λ lat They represent the weights of the three loss functions, which are selected based on experience and training results.

[0118] In some embodiments, in step S4, the test set is input into the trained deep learning model for testing to obtain a test result, and the test result is evaluated, including:

[0119] Step S401: Input the test set into the trained deep learning model for testing, obtain the anomaly score of each sample and then classify it.

[0120] Step S402: Evaluate the detection result.

[0121] The anomaly score calculation process is as follows:

[0122] First, for each input test image Calculate its anomaly score

[0123]

[0124] in, is a reconstruction score based on the contextual loss function, which is used to measure the contextual similarity between the input image and the generated image. represents the latent representation score based on the latent loss function, which is used to measure the difference between the input image and the generated image. λ is a weighting parameter that controls the relative importance of the score function.

[0125] Secondly, the anomaly scores of all input test images are The vector A is composed of

[0126]

[0127] Finally, the feature of vector A is scaled to between (0,1) to obtain the final anomaly score of each test image. The calculation formula is as follows:

[0128]

[0129] Step S402, evaluating the detection results, including:

[0130] S4021. Draw a ROC curve and a PRC curve based on the detection result and the true result.

[0131] The vertical axis of the coordinate system of the ROC curve is TPR (true positive rate), with a maximum value of 1, and the horizontal axis is FPR (false positive rate), with a maximum value of 1. The calculation formulas for TPR and FPR are as follows:

[0132]

[0133] Among them, TP is the number of positive samples correctly identified, FN is the number of positive samples misclassified as negative samples, P is the number of positive samples, FP is the number of negative samples misclassified as positive samples, TN is the number of negative samples correctly identified, and N is the number of negative samples. The larger the area under the ROC curve, i.e., AUC, the higher the accuracy of the model.

[0134] The horizontal axis of the PRC curve coordinate system is the recall rate (Recall), the maximum value is 1, and the vertical axis is the precision rate (Precision), the maximum value is 1. The calculation formulas for the recall rate and precision rate are as follows:

[0135]

[0136] The meaning of each symbol is the same as described above. The larger the area under the PRC curve, i.e., the AP, the better the classification performance of the model.

[0137] S4022. Based on the detection results and the true results, AUC, best_accuracy, best_threshold and best_F1_score are calculated; based on the drawn ROC curve, PRC curve and the calculated AUC, best_accuracy, best_threshold, best_F1_score, the model is evaluated.

[0138] Best_accuracy refers to the maximum value of the accuracy obtained in the test round. Accuracy refers to the ratio of the number of correct predictions in positive samples and negative samples to the total number. The calculation formula is as follows:

[0139]

[0140] Best_F1_score refers to the maximum value of the F1_score obtained in the test round. F1_score is the harmonic mean of recall and precision. Its calculation formula is as follows:

[0141]

[0142] Best_threshold refers to the optimal threshold value, which is calculated here using the method provided by sklearn, that is, after calculating the F1_score corresponding to all possible thresholds, the threshold corresponding to the maximum F1_score is selected as the optimal threshold.

[0143] See also Figure 2 The embodiment of the present application also provides an industrial equipment anomaly detection system based on skip-GANomaly, including: a data set acquisition module 101, a model construction module 102, a model training module 103, a model testing module 104 and a detection evaluation module 105 connected in sequence; the data set acquisition module 101 is used to acquire a data set, preprocess the data set, and divide the preprocessed data set into a training set and a test set; the model construction module 102 is used to build a deep learning model based on skip-GANomaly; the model training module 103 is used to input the training set into a deep learning model for model training, adjust the model parameters, and obtain a trained deep learning model; the model testing module 104 is used to input the test set into the trained deep learning model for testing to obtain a detection result; the prediction evaluation module 105 is used to evaluate the detection result.

[0144] In some embodiments, the model building module 102 includes a generator structure definition unit and a discriminator structure definition unit; the generator structure definition unit is used to define the generator network structure respectively, which consists of an encoder and a decoder, and connects the encoder part and the decoder part in the generator network structure to obtain a jump connection network structure, and completes the forward propagation function to determine the flow path of data in the network; the discriminator structure definition unit is used to define the discriminator network structure, and completes the forward propagation function of the discriminator to determine the flow path of data in the network; the input of the discriminator network structure is the true graph and the generator generated graph, and the final detection result is obtained.

[0145] The following is a detailed introduction to the skip-GANomaly-based industrial equipment anomaly detection method provided by the present application through a specific embodiment. This embodiment uses a skip-GANomaly model to benchmark the anomaly detection method, focusing on the analysis and prediction of the data set (MVTec AD) for industrial inspection.

[0146] See also Figure 1 ,This method first preprocesses the data set, which includes random cropping and data format processing; then, the preprocessed data is input into the network training model; the optimal model is selected from the training model to predict the test set, and finally anomaly detection is completed.

[0147] The specific process is as follows:

[0148] Step 1, obtain a data set. The data set selected in the embodiment of the present application is MVTec AD, which is a data set for unsupervised anomaly detection that can simulate real-world industrial inspection scenarios. This data set provides the possibility of evaluating unsupervised anomaly detection methods on various textures and object classes with different types of anomalies. It contains more than 5,000 high-resolution images, divided into 15 different object and texture categories. Each category contains a set of defect-free training images and a set of test images with various defects and images without defects. As shown in Table 1 below.

[0149] Table 1 MVTec AD dataset files

[0150]

[0151]

[0152] Step 2: Dataset preprocessing: According to the relevant principles and methods of deep learning, the data set selected in step 1 is preprocessed.

[0153] The specific process of this step is as follows:

[0154] First, read the image data from the memory cache and decode it into image format, and then normalize the image size to 256×256; secondly, randomly crop each image in the dataset into a set number of num sub-images, where the parameter num refers to the number of sub-images that each image is cropped into, to further enrich the training data; finally, convert all images to RGB format and normalize the number of channels to facilitate deep learning model training.

[0155] Step 3: Figure 3 and Figure 4 As shown, build the generator and discriminator networks to complete the construction of the deep learning model. The specific process of this step is as follows:

[0156] Step 3-1, define the generator network structure. The generator consists of two parts: the encoder and the decoder. First, define that the first layer of the encoder consists of a 4×4 convolutional layer, and the second to fifth layers are four encoding submodules (Encoder layer 1, Encoder layer 2, Encoder layer 3, Encoder layer 4). Each encoding submodule includes a LeakyReLU activation function, a 4×4 convolutional layer, and batch normalization connected in sequence, which realizes the extraction of the bottleneck feature z, that is, z = G E (x), x represents the input of the encoder.

[0157] The decoder includes four decoding submodules (Decoder layer 4, Decoder layer 3, Decoder layer 2, Decoder layer 1), wherein Decoder layers 2 to 4 are composed of sequentially connected ReLU activation functions, 4×4 deconvolution layers and batch normalization; Decoder layer 1 is composed of sequentially connected 4×4 deconvolution layers and Tanh activation functions; wherein the bottleneck feature z is sequentially increased through upsampling operations until it is consistent with the resolution of the input image, i.e. Indicates the generated image.

[0158] See also Figure 3 , the output of the Encoder layer 3 is jump-connected with the output of the submodule with the same resolution in the Decoder layer 3 in the encoder as the input of the next decoding submodule Decoder layer 3 in the decoder.

[0159] Step 3-2, define the discriminator network structure. The discriminator includes five decoding submodules (discriminator layer 1, discriminator layer 2, discriminator layer 3, discriminator layer 4, discriminator layer 5), where discriminator layers 1 to 4 are composed of LeakyReLU activation functions, 4×4 convolutional layers and batch normalization connected in sequence; discriminator layer 5, that is, the classification layer, is composed of 3×3 convolutional layers and sigmoid activation functions connected in sequence.

[0160] Step 4: Input the training set data into the skip-GANomaly-based deep learning model for model training. After the preset number of training rounds, adjust the model parameters according to the results to obtain a trained deep learning model.

[0161] The specific steps of this process are as follows:

[0162] In this embodiment, the optimizer selected for model training is Adam, and the batch size (batch_size) of data input is set to 132. Too small batch_size will increase the training time of the model, while too large batch_size will slow down the convergence speed of the model; the training rounds of the model are set to 100 rounds, and the learning rate is set to 0.0002. Figure 5 The generator (G) is trained and optimized by adding three loss functions according to certain weights to obtain the total loss function. The three loss functions are adversarial loss function, context loss function and potential loss function.

[0163] The calculation formula of the adversarial loss function is as follows:

[0164]

[0165] Among them, x represents the real image input, represents the fake image generated by the generator, and D represents the discriminator, indicating that the generator is updated based on the inside of the discriminator. The change of the loss function of the discriminator can be found in Figure 6 .

[0166] The calculation formula of the context loss function is as follows:

[0167]

[0168] Among them, x represents the real image input, Represents the fake graph generated by the generator, which represents the pair x and Perform L1 normalization.

[0169] The calculation formula of the potential loss function is as follows:

[0170]

[0171] Among them, x represents the real image input, represents the fake image generated by the generator, f(x) represents a function, which is equivalent to the last convolutional layer of the discriminator, and performs feature extraction on and.

[0172] The calculation formula of the generator's total loss function is as follows:

[0173]

[0174] Among them, λ adv , con and λ lat They represent the weights of the three loss functions, which are selected based on experience and training results. In this example, λ is selected adv =1,λ con =40,λ lat =1.

[0175] Step 5: Input the test set into the trained deep learning model for testing, obtain the test results, and evaluate the test results.

[0176] The specific steps of this process are as follows:

[0177] The test set is input into the trained model obtained in step 4. The model outputs the anomaly score for each input image and selects an appropriate threshold to determine the image anomaly. The evaluation module then compares the detection results with the correct results and calculates the model evaluation index. After the preset rounds of training, the optimal model is obtained and tested in the test set of each category of items to obtain the following results, as shown in Table 2.

[0178] Table 2 Test categories and test AUC results

[0179]

[0180] It can be seen from the detection results in Table 2 that the model proposed in the embodiment of the present application has a good prediction effect under the experimental conditions and can well complete the industrial anomaly detection task.

[0181] FIG. 7A to FIG. 7B Schematic diagram of the evaluation index results of the industrial equipment anomaly detection method based on skip-GANomaly for grid category images provided in this application. Fig. 7A and Figure 7B It can be seen that the ROC curve of this category is close to the upper left corner, the area below the graph is large, the AUC is high, and the PRC curve is close to the upper right corner, indicating that the model performance is good in accuracy and comprehensiveness, and can better realize anomaly recognition.

[0182] Based on the above technical scheme, the embodiment of the present application provides an industrial equipment anomaly detection method and system based on skip-GANomaly, and the method includes the following steps: first, obtaining a data set, preprocessing the data set, and dividing the preprocessed data set into a training set and a test set; secondly, building a deep learning model based on skip-GANomaly; next, inputting the training set into the deep learning model for model training, adjusting the model parameters, and obtaining a trained deep learning model; finally, inputting the test set into the trained deep learning model for testing, obtaining detection results, and evaluating the detection results.

[0183] The industrial deep learning anomaly detection method provided in this application adopts an industrial equipment anomaly detection method based on skip-GANomaly, which is improved based on the skip-GANomaly network model. It introduces the structure of skip connection, which can better capture the multi-scale distribution of normal data distribution in high-dimensional image space. Secondly, the use of adversarial training scheme in the model architecture can provide excellent reconstruction effects in high-dimensional image space and low-dimensional latent vector space encoding. Based on this, the model provided in this application can accurately identify anomalies of industrial equipment, thereby intelligently assisting industrial production.

[0184] Those skilled in the art can understand that the above-mentioned embodiments are specific examples for implementing the present application, and in practical applications, various changes can be made to them in form and detail without departing from the spirit and scope of the present application. Any person skilled in the art can make their own changes and modifications without departing from the spirit and scope of the present application, so the scope of protection of the present application shall be based on the scope defined in the claims.

Claims

1. A method for detecting anomalies of industrial equipment based on skip-GANomaly, characterized in that: The following steps are involved: Obtaining a data set, preprocessing the data set, and dividing the preprocessed data set into a training set and a test set; Build a deep learning model based on skip-GANomaly; Inputting the training set into the deep learning model for model training, adjusting model parameters, and obtaining a trained deep learning model; The test set is input into the trained deep learning model for testing, a test result is obtained, and the test result is evaluated.

2. The method for detecting anomalies of industrial equipment based on skip-GANomaly according to claim 1, characterized in that: The data set is preprocessed, including: The images in the data set are randomly cropped and data format processed to obtain a preprocessed data set.

3. The method for detecting anomalies of industrial equipment based on skip-GANomaly according to claim 1, characterized in that: The skip-GANomaly-based deep learning model is an improved deep learning model with a skip-connected encoder-decoder network structure; The skip-GANomaly-based deep learning model learns data features through a generator and reconstructs an input image to obtain a generated image, compares and scores the generated image with the real image through a discriminator, and calculates an anomaly score based on the loss to obtain an anomaly detection result.

4. The method for detecting anomalies of industrial equipment based on skip-GANomaly according to claim 1, characterized in that: Build a skip-GANomaly based deep learning model, including: Defining a generator network structure; the generator network structure includes an encoder unit and a decoder unit; Connecting the encoder unit and the decoder unit in the generator network structure to obtain a skip connection network structure, and completing a forward propagation function to determine a flow path of data in the network; Define the discriminator network structure; Complete the forward propagation function of the discriminator to determine the flow path of data in the network.

5. The method for detecting anomalies of industrial equipment based on skip-GANomaly according to claim 4, characterized in that: The encoder unit includes a two-dimensional convolution and four identical convolution layers for extracting data features; the input end of each convolution layer is provided with a LeakyReLU activation function, and the output end is provided with batch normalization; the relationship between the input and output of the encoder unit is as follows: z=G E (x) Among them, G E is the encoder unit, x is the encoder unit G E The input of z is the encoder unit G E Output: The calculation formula of the LeakyReLU activation function is as follows: f(x)=max(0,x)+α·min(0,x),0<α<<1 Where x is the input value, and f(x) is the output of the LeakyReLU activation function. The LeakyReLU function introduces a very small slope α when the input is negative to prevent the gradient from disappearing. When the input is non-negative, the output is the same as the input. The decoder unit includes four deconvolution layers, three of which have the same structure, with a ReLU activation function at the input end and a batch normalization at the output end; the remaining deconvolution layer is only provided with a Tanh activation function at the output end; the relationship between the input and output of the decoder unit is as follows: Among them, G D is the decoder unit, z is the decoder unit G D Input, is the decoder unit G D Output: The calculation formula of the ReLU activation function is as follows: f(x)=max(0,x) Where x is the input value and f(x) is the output of the ReLU activation function. The ReLU function outputs 0 when the input is negative and the output is the same as the input when the input is non-negative. The calculation formula of the Tanh activation function is as follows: Where x is the input value and Tanh(x) is the output of the Tanh activation function; the Tanh function converts its input to the (-1,1) interval; The discriminator includes a two-dimensional convolution and four identical convolution layers, with a LeakyReLU activation function at its input and a batch normalization at its output; the last layer of the discriminator is composed of a two-dimensional convolution and a sigmoid activation function; The calculation formula of the sigmoid activation function is as follows: Among them, x is the input value of the discriminator, sigmoid(x) is the output of the sigmoid function; the sigmoid activation function smoothly converts the input to between (0,1).

6. The method for detecting anomalies of industrial equipment based on skip-GANomaly according to claim 4, characterized in that: The generator is optimized by a total loss function obtained by adding three loss functions according to certain weights, wherein the three loss functions are an adversarial loss function, a contextual loss function, and a potential loss function; The calculation formula of the adversarial loss function is as follows: Among them, x represents the real image input, represents the fake image generated by the generator, and D represents the discriminator; The calculation formula of the context loss function is as follows: Among them, x represents the real image input, Represents the fake image generated by the generator; The calculation formula of the potential loss function is as follows: Among them, x represents the real image input, represents the fake graph generated by the generator, and f represents a function, which is equivalent to the last convolutional layer of the discriminator, for x and Perform feature extraction; The calculation formula of the total loss function of the generator is as follows: Among them, λ adv , con and λ lat Represent the weights of the three loss functions respectively.

7. The method for detecting anomalies of industrial equipment based on skip-GANomaly according to claim 1, characterized in that: The test set is input into the trained deep learning model for testing to obtain a test result, including: Input the test set into the trained deep learning model for testing, calculate the anomaly score of each sample, perform classification prediction, and evaluate the test results; The calculation process of the anomaly score includes: First, for each input test image Calculate its anomaly score in, is a reconstruction score based on the contextual loss function, which is used to measure the contextual similarity between the input image and the generated image; represents the potential representation score based on the potential loss function, which is used to measure the difference between the input image and the generated image; λ is a weighting parameter that controls the relative importance of the score function; The anomaly scores of all input test images Composition vector A: Finally, the feature of vector A is scaled to between (0,1) to obtain the final anomaly score of each corresponding test image. The calculation formula is as follows: Among them, min(A) represents the minimum value in vector A, and max(A) represents the maximum value in vector A.

8. The method for detecting anomalies of industrial equipment based on skip-GANomaly according to claim 1, characterized in that: The test results are evaluated, including: Based on the test results and the true results, a ROC curve and a PRC curve are drawn; Based on the detection results and the true results, AUC, best_accuracy, best_threshold and best_F1_score are calculated; based on the drawn ROC curve, PRC curve and the calculated AUC, best_accuracy, best_threshold, best_F1_score, the model is evaluated; The vertical axis of the coordinate system of the ROC curve is the true positive rate, with a maximum value of 1, and the horizontal axis is the false positive rate, with a maximum value of 1. The calculation formulas for TPR and FPR are as follows: Among them, TP is the number of positive samples correctly identified, FN is the number of positive samples misclassified as negative samples, P is the number of positive samples, FP is the number of negative samples misclassified as positive samples, TN is the number of negative samples correctly identified, N is the number of negative samples, TPR is the true positive rate, and FPR is the false positive rate; the larger the area under the ROC curve, i.e., AUC, the higher the accuracy of the model; The horizontal axis of the PRC curve coordinate system is the recall rate, the maximum value is 1, and the vertical axis is the precision rate, the maximum value is 1. The calculation formulas for the recall rate and the precision rate are as follows: Recall represents the recall rate, and Precision represents the precision rate. The larger the area under the PRC curve, i.e., the larger the AP, the better the classification performance of the model. AUC is the area under the ROC curve; Best_accuracy refers to the maximum value of the accuracy obtained in the test round. Accuracy refers to the ratio of the number of correct predictions in positive samples and negative samples to the total number. The calculation formula is as follows: Best_F1_score refers to the maximum value of the F1_score obtained in the test round. F1_score is the harmonic mean of recall and precision. Its calculation formula is as follows: best_threshold refers to the optimal threshold.

9. An industrial equipment anomaly detection system based on skip-GANomaly, characterized in that: include: A data set acquisition module, a model building module, a model training module, a model testing module, and a detection and evaluation module connected in sequence; The data set acquisition module is used to acquire a data set, preprocess the data set, and divide the preprocessed data set into a training set and a test set; The model building module is used to build a deep learning model based on skip-GANomaly; The model training module is used to input the training set into the deep learning model for model training, adjust model parameters, and obtain a trained deep learning model; The model testing module is used to input the test set into the trained deep learning model for testing to obtain a test result; The detection evaluation module is used to evaluate the detection result.

10. The industrial equipment anomaly detection system based on skip-GANomaly according to claim 9, characterized in that: The model building module includes a generator structure definition unit and a discriminator structure definition unit; The generator structure definition unit is used to define an encoder network structure and a decoder network structure respectively, connect the encoder network structure and the decoder network structure to obtain a skip connection network structure, and complete a forward propagation function to determine a flow path of data in the network; The discriminator structure definition unit is used to define the discriminator network structure and complete the forward propagation function of the discriminator to determine the flow path of data in the network; The input of the discriminator network structure is the true image and the image generated by the generator, and the final detection result is obtained.

Images