Semi-supervised anomaly detection model training method

Abstract

The invention relates to a semi-supervised anomaly detection model training method, belongs to the technical field of detection model training, and solves the problems of difficulty in data labeling, sample imbalance and high over-fitting risk in the prior art. Obtaining an abnormal product image, marking an abnormal region in the abnormal product image to obtain an abnormal sample, obtaining a normal product image, marking the normal product image to obtain a normal sample, and constructing a training set and a verification set based on the normal sample and the abnormal sample; each of the training set and the verification machine comprises an abnormal sample and a normal sample at the same time; constructing a semi-supervised anomaly detection model, and performing initialization operation on the semi-supervised anomaly detection model; and inputting the training set into the initialized semi-supervised anomaly detection model for model training, updating model parameters, evaluating model performance through the verification set, and stopping training until the model performance meets a predetermined standard to obtain a trained semi-supervised anomaly detection model.

Description

Technical Field

[0001] The present invention relates to the technical field of detection model training, and in particular to a semi-supervised anomaly detection model training method. Background Art

[0002] With the rapid development of industrial automation technology, traditional manual detection methods can no longer meet the high standards of modern production. As a key link to ensure product quality and production safety, industrial anomaly detection uses automation technology to identify abnormal conditions of products or equipment during the production process. Automatic defect detection technology based on deep learning identifies anomalies by analyzing product images, which has significant advantages in improving detection efficiency and accuracy.

[0003] However, it is very costly to obtain a large amount of labeled data in industrial scenarios. Not only does it require professionals to spend a lot of time and energy to label abnormal areas and categories in images, but the accuracy and consistency of the labeling are also difficult to guarantee. A small number of labeled samples are difficult to support the model to fully learn complex abnormal features, and a large number of labeling faces cost bottlenecks. Secondly, the problem of sample imbalance in model training is serious. In actual production, the number of normal product samples is often far greater than that of abnormal samples. This unbalanced data distribution can easily lead to the model's tendency to learn normal sample features during training, and its ability to identify abnormal samples is weak. Furthermore, the risk of overfitting during training is high. Due to the complexity and diversity of industrial image data, if the model structure design is unreasonable or the training parameters are improperly selected, the model is prone to overfitting on the training set, resulting in poor performance and poor generalization ability when actually detecting new samples. Summary of the invention

[0004] In view of the above analysis, an embodiment of the present invention aims to provide a semi-supervised anomaly detection model training method to solve the problems of difficult data labeling, sample imbalance and high risk of overfitting in existing training.

[0005] A semi-supervised anomaly detection model training method provided by an embodiment of the present invention includes:

[0006] Acquire an abnormal product image and annotate an abnormal area in the abnormal product image to obtain an abnormal sample, wherein the abnormal sample includes the abnormal product image and a label corresponding to the abnormal product image, and the label corresponding to the abnormal product image includes the abnormal area and classification information of the abnormal product image;

[0007] Acquire a normal product image and annotate the normal product image to obtain a normal sample, wherein the normal sample includes a normal product image and a label corresponding to the normal product image, and the label corresponding to the normal product image includes classification information;

[0008] Constructing a training set and a verification set based on normal samples and abnormal samples; the training set and the verification set both include abnormal samples and normal samples;

[0009] Construct a semi-supervised anomaly detection model and initialize the semi-supervised anomaly detection model; input the training set into the initialized semi-supervised anomaly detection model for model training, update the model parameters, evaluate the model performance through the validation set, and stop training until the model performance meets the predetermined standards to obtain a trained semi-supervised anomaly detection model.

[0010] As a further improvement of the present application, the step of inputting the training set into the initialized semi-supervised anomaly detection model for model training includes multiple epochs, in each epoch, the training set is divided into multiple batches, each batch includes random normal samples and abnormal samples, and the multiple batches are respectively input into the initialized semi-supervised anomaly detection model for model training;

[0011] After completing the processing of all batches in an epoch, the performance of the model is evaluated through the validation set. If the model performance does not meet the predetermined standard, the training of the next epoch will continue;

[0012] If the model performance meets the predetermined standards, the training is stopped and the model parameters are saved.

[0013] As a further improvement of the present application, the semi-supervised anomaly detection model includes an input module, an Encoder-Decoder module and a VAE-GAN encoder;

[0014] Input each batch into the initialized semi-supervised anomaly detection model for model training, including:

[0015] The normal samples and abnormal samples in the batch are input to the input module of the semi-supervised anomaly detection model. The input module outputs the normal samples to the Encoder-Decoder module and outputs the abnormal samples to the VAE-GAN encoder.

[0016] The Encoder-Decoder module extracts features from the image data of normal samples, generates image reconstruction data based on the extracted features, and outputs the extracted features to the pseudo labeler;

[0017] The VAE-GAN encoder extracts features from the image data of abnormal samples and generates potential abnormal information and abnormal reconstruction data;

[0018] The pseudo-labeler generates pseudo-labels based on the features extracted by the Encoder-Decoder module;

[0019] A loss function is constructed based on image reconstruction data, potential abnormal information, and abnormal reconstruction data, and the model parameters are updated through back propagation. After each batch training, the learning rate is adjusted according to the training progress.

[0020] As a further improvement of the present application, the semi-supervised anomaly detection model further includes a memory module and an anomaly detection module;

[0021] A memory module, used to store historical abnormal images and their corresponding abnormal detection results;

[0022] The anomaly detection module obtains the anomaly detection result of the product to be detected based on the image data of the product to be detected, the image reconstruction data, potential anomaly information, anomaly reconstruction data, pseudo labels, historical anomaly image data stored in the memory module and the corresponding anomaly detection results, and transmits the anomaly detection result to the memory module.

[0023] As a further improvement of the present application, the Encoder-Decoder module includes an Encoder unit and a Decoder unit;

[0024] The encoder unit includes a first convolutional layer, a first pooling layer, a second convolutional layer, and a second pooling layer connected end to end in sequence;

[0025] The first convolution layer extracts the features of the image data, and the first pooling layer performs a pooling operation on the features extracted by the first convolution layer. The second convolution layer performs a convolution operation on the output of the first pooling layer to extract the features of the image data. The second pooling layer performs dimensionality reduction processing on the output of the second convolution layer to obtain the final features of the image data. The output of the second pooling layer is used as the output of the Encoder unit and connected to the input of the Decoder unit and the pseudo labeler.

[0026] The decoder unit includes a transposed convolution layer, an upsampling layer, and a decoder convolution layer connected end to end;

[0027] The transposed convolution layer receives the output of the second pooling layer and performs a transposed convolution operation on the final features of the image data; the upsampling layer upsamples the output of the transposed convolution layer to increase the feature space dimension; the decoder convolution layer performs a convolution operation on the output of the upsampling layer to generate image reconstruction data. The output of the decoder convolution layer is used as the output of the decoder unit and connected to the input of the anomaly detection module.

[0028] As a further improvement of the present application, the VAE-GAN encoder includes an encoder and a decoder connected end to end, the encoder is used to generate potential abnormality information based on the image data; the decoder is used to generate reconstructed image data based on the potential abnormality information;

[0029] The encoder includes a first convolution layer, a first activation function layer, a first pooling layer, a second convolution layer, a second activation function layer, a second pooling layer, a Flatten layer, a first fully connected layer, a third activation function layer, and a second fully connected layer, which are connected end to end in sequence;

[0030] Among them, the first convolution layer extracts the features of the image data, the first activation function layer performs nonlinear processing on the output of the first convolution layer; the first pooling layer performs a pooling operation on the output of the first encoder activation function layer to reduce the feature dimension after nonlinear processing; the second convolution layer performs a convolution operation on the output of the first pooling layer to extract the features of the image data; the second activation function layer performs nonlinear processing on the output of the second convolution layer; the second pooling layer performs a pooling operation on the output of the second activation function layer; the Flatten layer converts the output of the second pooling layer into a one-dimensional feature vector; the first fully connected layer performs feature compression on the one-dimensional feature vector; the third activation function layer performs nonlinear processing on the output of the first fully connected layer; the second fully connected layer performs feature mapping on the output of the third activation function layer to generate potential abnormal information of the image data;

[0031] The decoder comprises a first transposed convolution layer, a first activation function layer, an upsampling layer, a second transposed convolution layer, a second activation function layer, and a decoder convolution layer which are sequentially connected end to end;

[0032] Among them, the first transposed convolution layer performs a transposed convolution operation on the potential abnormal information received from the encoder; the first activation function layer performs nonlinear processing on the output of the first transposed convolution layer; the upsampling layer upsamples the output of the first activation function layer; the second transposed convolution layer performs a transposed convolution operation on the output of the upsampling layer; the second activation function layer performs nonlinear processing on the output of the second transposed convolution layer; the decoder convolution layer performs a convolution operation on the output of the second activation function layer to obtain reconstructed image data.

[0033] As a further improvement of the present application, the pseudo labeler includes an input layer, a fully connected layer, an activation function layer, and an output layer connected end to end in sequence;

[0034] The input layer receives the final features of the image data output by the Encoder unit and transmits them to the fully connected layer;

[0035] The fully connected layer performs spatial mapping on the final features of the image data to generate the number of feature categories;

[0036] An activation function layer generates a distribution probability of each feature category based on the number of feature categories through a Softmax function;

[0037] The output layer generates anomaly labels based on the distribution probability of each feature category and transmits them to the anomaly detection module.

[0038] As a further improvement of the present application, the loss function is shown in calculation formula (1);

[0039] L total =ω 1 .L recon +ω 2 .L anomaly +ω 3 .L latent +ω 4 .L pseudo (1);

[0040] Among them, L total is the total loss function; L recon is the loss function of the Encoder-Decoder module; L anomaly is the loss function of the VAE-GAN encoder; L latent is the regularization loss of potential abnormal information; L pseudo is the loss function of the pseudo labeler.

[0041] As a further improvement of the present application, the loss function of the Encoder-Decoder module is shown in formula (2);

[0042]

[0043] Where n is the number of samples, y i is the image data of the i-th sample, is the reconstructed data of the i-th sample;

[0044] The loss function of the VAE-GAN encoder is calculated as shown in formula (3);

[0045]

[0046] Where n is the number of samples, a i is the image data of the i-th sample, Reconstruct the data for the anomaly of the i-th sample;

[0047] The regularization loss of potential abnormal information is shown in formula (4);

[0048] L latent =D KL (q(z|xi)||p(z)) (4)

[0049] Among them, D KL is the KL divergence, q(z|xi) is the potential abnormal information, and p(z) is the prior distribution of the abnormal information.

[0050] As a further improvement of the present application, the pseudo labeler loss function is shown in formula (5);

[0051]

[0052] Where n is the number of samples, y ic is the encoding of the label of the i-th sample, is the pseudo label encoding of the i-th sample.

[0053] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:

[0054] 1. The present invention constructs a training set and a validation set based on normal samples and abnormal samples, which contain both. The model can be fully exposed to different types of samples during the training process, avoiding learning bias caused by large differences in the number of samples. In each epoch, the training set is divided into multiple batches, and each batch randomly contains normal samples and abnormal samples, which helps the model better capture the feature differences between normal and abnormal samples, improves the ability to identify abnormal samples, and effectively overcomes the negative impact of sample imbalance.

[0055] 2. The present invention uses multi-module collaborative training, including Encoder-Decoder module and VAE-GAN encoder, to learn sample features from multiple perspectives, avoid the model's over-reliance on a certain type of sample features or local features, and enhance the model's ability to learn and integrate different features. During the training process, the model performance is evaluated through the validation set, the training strategy is adjusted according to the model performance, and the learning rate is adjusted according to the training progress after each batch training. The model training process is further optimized, the risk of overfitting is effectively reduced, and the generalization ability of the model is improved, so that it can accurately identify various new abnormal situations in actual detection.

[0056] 3. A pseudo-labeler is used to generate pseudo-labels based on a small amount of labeled data. When calculating the pseudo-label deviation, the relationship between the pseudo-label and other features is fully considered, so that the model can more accurately utilize the information in the unlabeled data, effectively improving the accuracy of semi-supervised anomaly detection, reducing the demand for large-scale labeled data, reducing the cost of data labeling, and improving the practicality and scalability of the detection method.

[0057] In the present invention, the above-mentioned technical solutions can also be combined with each other to achieve more preferred combination solutions. Other features and advantages of the present invention will be described in the subsequent description, and some advantages can become obvious from the description, or can be understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained through the contents particularly pointed out in the description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] The accompanying drawings are only used for the purpose of illustrating specific embodiments and are not to be considered as limiting the present invention. In the entire drawings, the same reference symbols represent the same components;

[0059] Figure 1 A flowchart of a semi-supervised anomaly detection model training method provided by one embodiment of the present invention;

[0060] Figure 2 A structural diagram of a semi-supervised anomaly detection model is provided for one embodiment of the present invention. DETAILED DESCRIPTION

[0061] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not used to limit the scope of the present invention.

[0062] A specific embodiment of the present invention discloses a semi-supervised anomaly detection model training method, such as Figure 1 As shown, a semi-supervised anomaly detection model training method includes:

[0063] Step 101, obtain an abnormal product image and mark the abnormal area in the abnormal product image to obtain an abnormal sample, wherein the abnormal sample includes the abnormal product image and its corresponding label, and the label corresponding to the abnormal product image includes the abnormal area and classification information of the abnormal product image.

[0064] The product images are acquired through image acquisition equipment, and the product images with abnormalities are identified through manual screening or other means, and the abnormal areas in the abnormal product images are labeled to obtain abnormal samples. The abnormal areas in the abnormal product images can be labeled by the image labeling tool Labelme. The abnormal samples include abnormal product images and their corresponding labels, and the labels include abnormal areas and classification information.

[0065] Specifically, when marking, a marking box is used to frame the abnormal position in the image, and the category of the abnormality is marked. For example, the coordinates of the marking box can be expressed in pixel coordinates, and the range of the marking box is determined by the coordinates of the pixel points in the upper left corner and the lower right corner; the categories include shape deformation, surface scratches and pits, color deviation, and component missing.

[0066] Step 102, obtaining a normal product image and labeling the normal product image to obtain a normal sample, wherein the normal sample includes a normal product image and a label corresponding to the normal product image, and the label corresponding to the normal product image includes classification information.

[0067] For normal samples, since there are no anomalies in the image, only the category information needs to be labeled. The category information is normal, which is different from the anomaly category labeled in the abnormal sample; after the labeling is completed, the normal product image and its corresponding label are regarded as normal samples.

[0068] Step 103, constructing a training set and a verification set based on normal samples and abnormal samples; the training set and the verification set both include abnormal samples and normal samples.

[0069] The normal samples and abnormal samples obtained are used to construct the training set in a certain proportion. During the division process, it is necessary to ensure that both the training set and the validation set contain normal samples and abnormal samples, and the distribution is as even as possible. Random sampling can be used to randomly select a certain number of normal samples and abnormal samples from all samples to form the training set, and the remaining samples to form the validation set.

[0070] Step 104, construct a semi-supervised anomaly detection model, initialize the semi-supervised anomaly detection model; input the training set into the initialized semi-supervised anomaly detection model for model training, update the model parameters, evaluate the model performance through the validation set, until the model performance meets the predetermined standard, stop training, and obtain a trained semi-supervised anomaly detection model. During training, the training set is input into the initialized model for training, and the training process includes multiple epochs. In each epoch, the training set is divided into multiple batches, each batch includes random normal samples and abnormal samples. Multiple batches are input into the model in sequence for training.

[0071] The initialization operation refers to the initial setting of the semi-supervised anomaly detection model before training begins, including setting initial values ​​for the model's weights and biases to avoid initial bias, setting the initial learning rate, setting the regularization term to prevent overfitting, determining the number of neurons in the model, and setting model hyperparameters, including the number of epochs, batch size, etc.

[0072] Specifically, the training set is input into the initialized semi-supervised anomaly detection model for model training, which includes multiple epochs. In each epoch, the training set is divided into multiple batches, each batch includes random normal samples and abnormal samples, and the multiple batches are input into the initialized semi-supervised anomaly detection model for model training.

[0073] When training a model, hyperparameters include the number of epochs, batch size, and learning rate. The specific number of epochs can be determined through experience and experiments, and can be adjusted based on the performance of the model on the validation set. If the model converges slowly during training, the number of epochs can be appropriately increased; if there are signs of overfitting, training may need to be terminated early. The batch size is the number of samples in each batch, which needs to be adjusted according to the actual training objectives and model performance requirements. The number of samples in each batch affects the training efficiency and stability of the model. A batch with a smaller number of samples can make the model closer to the global optimal solution at each update, but it may cause the training process to fluctuate more and converge more slowly; a batch with a larger number of samples can speed up the convergence speed, but may fall into the local optimal solution and have higher memory requirements. The learning rate determines the step size of the model at each parameter update. A larger learning rate can speed up the model training, but may cause the model to miss the optimal solution; a smaller learning rate can make the model more accurately approach the optimal solution, but the training process will be very slow. During the training process, a learning rate decay strategy is usually adopted to gradually reduce the learning rate as the training progresses.

[0074] At the beginning of each epoch, the training set is divided into multiple batches according to the set batch size. The samples in each batch are input into the initialized semi-supervised anomaly detection model in turn. After completing the processing of all batches in an epoch, the performance of the model is evaluated through the validation set. If the model performance does not meet the predetermined standard, the training of the next epoch is continued. If the model performance meets the predetermined standard, the training is stopped and the model parameters are saved. The calculated performance indicators are compared with the predetermined standards. If the model performance does not meet the predetermined standard, the training of the next epoch is continued. If the model performance meets the predetermined standard, the training is stopped and the parameters of the current model are saved. The model corresponding to these parameters is the trained semi-supervised anomaly detection model, which can be used for subsequent anomaly detection tasks.

[0075] Furthermore, the semi-supervised anomaly detection model includes an input module, an Encoder-Decoder module, and a VAE-GAN encoder and a pseudo labeler;

[0076] Input each batch into the initialized semi-supervised anomaly detection model for model training, including:

[0077] The normal samples and abnormal samples in the batch are input to the input module of the semi-supervised anomaly detection model. The input module outputs the normal samples to the Encoder-Decoder module and outputs the abnormal samples to the VAE-GAN encoder.

[0078] The Encoder-Decoder module extracts features from the image data of normal samples, generates image reconstruction data based on the extracted features, and outputs the extracted features to the pseudo labeler;

[0079] The VAE-GAN encoder extracts features from the image data of abnormal samples and generates potential abnormal information and abnormal reconstruction data;

[0080] The pseudo-labeler generates pseudo-labels based on the features extracted by the Encoder-Decoder module;

[0081] The loss function is constructed based on image reconstruction data, potential abnormal information, and abnormal reconstruction data, and the model parameters are updated through back propagation; after each batch training, the learning rate is adjusted according to the training progress. The loss function is shown in formula (1);

[0082] L total =ω 1 .L recon +ω 2 .L anomaly +ω 3 .L latent +ω 4 .L pseudo (1);

[0083] Among them, L total is the total loss function; L recon is the loss function of the Encoder-Decoder module; L anomaly is the loss function of the VAE-GAN encoder; L latent is the regularization loss of potential abnormal information; L pseudo is the loss function of the pseudo labeler.

[0084] The loss function of the Encoder-Decoder module is shown in formula (2);

[0085]

[0086] Where n is the number of samples, y i is the image data of the i-th sample, is the reconstructed data of the i-th sample;

[0087] The loss function of the VAE-GAN encoder is calculated as shown in formula (3);

[0088]

[0089] Where n is the number of samples, a i is the image data of the i-th sample, Reconstruct the data for the anomaly of the i-th sample;

[0090] The regularization loss of potential abnormal information is shown in formula (4);

[0091] L latent =D KL (q(z|xi)||p(z)) (4)

[0092] Among them, D KL is the KL divergence, q(z|xi) is the potential abnormal information, and p(z) is the prior distribution of the abnormal information.

[0093] The pseudo labeler loss function is shown in formula (5);

[0094]

[0095] Where n is the number of samples, y ic is the encoding of the label of the i-th sample, is the pseudo label encoding of the i-th sample.

[0096] More specifically, the input module is the input end of the semi-supervised anomaly detection model, and receives the image data of the product to be detected. The image data of the product to be detected refers to a two-dimensional or three-dimensional image including information such as the appearance and structure of the product to be detected, which can be a grayscale image, a color image or other forms of image data. The input module sends the received image data to the Encoder-Decoder module, the VAE-GAN encoder, the anomaly detection module and the memory module respectively.

[0097] The Encoder-Decoder module extracts features from the image data of the product to be inspected and sends them to the pseudo labeler; it also generates image reconstruction data based on the extracted features and sends the image reconstruction data to the anomaly detection module. The extracted features include features that can represent image information, such as edges, textures, and shapes. Generating image reconstruction data means restoring the extracted features to an image. Through the combination of the transposed convolution layer, the upsampling layer, and the convolution layer, the low-dimensional features are gradually restored to a spatial size similar to the original image. The difference between the original image and the reconstructed image can be judged to detect anomalies.

[0098] Specifically, the Encoder-Decoder module includes an Encoder unit and a Decoder unit;

[0099] The encoder unit includes a first convolutional layer, a first pooling layer, a second convolutional layer, and a second pooling layer connected end to end in sequence;

[0100] The first convolution layer extracts the features of the image data, and the first pooling layer performs a pooling operation on the features extracted by the first convolution layer. The second convolution layer performs a convolution operation on the output of the first pooling layer to extract the features of the image data. The second pooling layer performs dimensionality reduction processing on the output of the second convolution layer to obtain the final features of the image data. The output of the second pooling layer is used as the output of the Encoder unit and connected to the input of the Decoder unit and the pseudo labeler.

[0101] The decoder unit includes a transposed convolution layer, an upsampling layer, and a decoder convolution layer connected end to end;

[0102] The transposed convolution layer receives the output of the second pooling layer and performs a transposed convolution operation on the final features of the image data; the upsampling layer upsamples the output of the transposed convolution layer to increase the feature space dimension; the decoder convolution layer performs a convolution operation on the output of the upsampling layer to generate image reconstruction data. The output of the decoder convolution layer is used as the output of the decoder unit and connected to the input of the anomaly detection module.

[0103] The VAE-GAN encoder is used to generate potential abnormality information based on the image data of the product to be detected, generate abnormal reconstruction data based on the potential abnormality information, and send the potential abnormality information and the abnormal reconstruction data to the abnormality detection module.

[0104] Specifically, the VAE-GAN encoder includes an encoder and a decoder connected end to end. The encoder is used to generate potential anomaly information based on image data; the decoder is used to generate reconstructed image data based on the potential anomaly information. Potential anomaly information is a potential representation obtained by encoding image data, which can contain abnormal features existing in the image. The potential anomaly information needs to be restored to observable image information by the decoder. The abnormal reconstructed data is an image obtained by decoding the potential anomaly information through the decoder part of the VAE-GAN encoder. The decoded image will show obvious differences in the abnormal area compared with the original image, and this difference can be used as a basis for anomaly detection.

[0105] Based on the potential anomaly information, anomaly reconstruction data is generated, which provides another angle for detecting anomalies. The potential anomaly information and anomaly reconstruction data are sent to the anomaly detection module.

[0106] The encoder includes a first convolution layer, a first activation function layer, a first pooling layer, a second convolution layer, a second activation function layer, a second pooling layer, a Flatten layer, a first fully connected layer, a third activation function layer, and a second fully connected layer, which are connected end to end in sequence; wherein the first convolution layer extracts features of image data, and the first activation function layer performs nonlinear processing on the output of the first convolution layer; the first pooling layer performs a pooling operation on the output of the first encoder activation function layer to reduce the feature dimension after nonlinear processing; the second convolution layer performs a convolution operation on the output of the first pooling layer to extract features of image data; the second activation function layer performs nonlinear processing on the output of the second convolution layer; the second pooling layer performs a pooling operation on the output of the second activation function layer; the Flatten layer converts the output of the second pooling layer into a one-dimensional feature vector; the first fully connected layer performs feature compression on the one-dimensional feature vector; the third activation function layer performs nonlinear processing on the output of the first fully connected layer; the second fully connected layer performs feature mapping on the output of the third activation function layer to generate potential abnormal information of the image data.

[0107] The decoder includes a first transposed convolution layer, a first activation function layer, an upsampling layer, a second transposed convolution layer, a second activation function layer, and a decoder convolution layer which are connected end to end in sequence;

[0108] Among them, the first transposed convolution layer performs a transposed convolution operation on the potential abnormal information received from the encoder; the first activation function layer performs nonlinear processing on the output of the first transposed convolution layer; the upsampling layer upsamples the output of the first activation function layer; the second transposed convolution layer performs a transposed convolution operation on the output of the upsampling layer; the second activation function layer performs nonlinear processing on the output of the second transposed convolution layer; the decoder convolution layer performs a convolution operation on the output of the second activation function layer to obtain reconstructed image data.

[0109] The pseudo labeler is used to generate anomaly labels based on the features extracted by the Encoder-Decoder module and send them to the anomaly detection module. The pseudo label is a predicted label obtained by processing the input layer, the fully connected layer, the activation function layer, and the output layer. It can provide additional supervision information when the labeled data is limited. For product images, the pseudo labeler can generate anomaly labels based on the distribution probability of each feature category.

[0110] Specifically, the pseudo labeler includes an input layer, a fully connected layer, an activation function layer, and an output layer which are sequentially connected end to end;

[0111] The input layer receives the final features of the image data output by the Encoder unit and transmits them to the fully connected layer;

[0112] The fully connected layer performs spatial mapping on the final features of the image data to generate the number of feature categories;

[0113] An activation function layer generates a distribution probability of each feature category based on the number of feature categories through a Softmax function;

[0114] The output layer generates anomaly labels based on the distribution probability of each feature category and transmits them to the anomaly detection module.

[0115] Furthermore, the semi-supervised anomaly detection model also includes a memory module and an anomaly detection module;

[0116] A memory module, used to store historical abnormal images and their corresponding abnormal detection results;

[0117] The anomaly detection module obtains the anomaly detection result of the product to be detected based on the image data of the product to be detected, the image reconstruction data, potential anomaly information, anomaly reconstruction data, pseudo labels, historical anomaly image data stored in the memory module and the corresponding anomaly detection results, and transmits the anomaly detection result to the memory module.

[0118] When retrieving the historical abnormal image with the highest similarity to the comprehensive deviation feature in the memory module, the historical abnormal image with the highest similarity to the comprehensive deviation feature can be retrieved in the memory module by using the Euclidean distance or the hash value as the similarity metric. The abnormal classification unit maps the abnormal label to a specific abnormal category according to the abnormal label output by the pseudo labeler.

[0119] The memory module is used to store historical abnormal image data and its corresponding abnormal detection results. The historical abnormal image data is image data that has been detected before and is determined to have abnormalities, and its corresponding abnormal detection results include abnormal areas and abnormal types.

[0120] The above-mentioned embodiments of the present invention have the following beneficial effects: the present invention constructs a training set and a validation set containing both normal samples and abnormal samples, so that the model can fully contact different types of samples during the training process, avoiding learning bias caused by excessive differences in the number of samples. In each epoch, the training set is divided into multiple batches, and each batch randomly contains normal samples and abnormal samples, which helps the model to better capture the feature differences between normal and abnormal samples, improve the ability to identify abnormal samples, and effectively overcome the negative impact of sample imbalance; the present invention learns sample features from multiple angles through multi-module collaborative training, including Encoder-Decoder modules and VAE-GAN encoders, avoids the model from over-relying on a certain type of sample features or local features, and enhances the model's learning and integration capabilities for different features. During the training process, the model performance is evaluated through the validation set, the training strategy is adjusted according to the model performance, and the learning rate is adjusted according to the training progress after each batch training is completed, so as to further optimize the model training process, effectively reduce the risk of overfitting, and improve the generalization ability of the model, so that it can accurately identify various new abnormal situations in actual detection. The pseudo-labeler generates pseudo-labels based on a small amount of labeled data. When calculating the pseudo-label deviation, the relationship between the pseudo-label and other features is fully considered, so that the model can more accurately utilize the information in the unlabeled data, effectively improving the accuracy of semi-supervised anomaly detection, reducing the demand for large-scale labeled data, reducing the cost of data labeling, and improving the practicality and scalability of the detection method.

[0121] Those skilled in the art will appreciate that all or part of the processes of the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, wherein the computer-readable storage medium is a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0122] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. A semi-supervised anomaly detection model training method, characterized in that: The method comprises: Acquire an abnormal product image and annotate an abnormal area in the abnormal product image to obtain an abnormal sample, wherein the abnormal sample includes the abnormal product image and a label corresponding to the abnormal product image, and the label corresponding to the abnormal product image includes the abnormal area and classification information of the abnormal product image; Acquire a normal product image and annotate the normal product image to obtain a normal sample, wherein the normal sample includes a normal product image and a label corresponding to the normal product image, and the label corresponding to the normal product image includes classification information; Constructing a training set and a verification set based on normal samples and abnormal samples; the training set and the verification set both include abnormal samples and normal samples; Construct a semi-supervised anomaly detection model and initialize the semi-supervised anomaly detection model; input the training set into the initialized semi-supervised anomaly detection model for model training, update the model parameters, evaluate the model performance through the validation set, and stop training until the model performance meets the predetermined standards to obtain a trained semi-supervised anomaly detection model.

2. The method according to claim 1, characterized in that The step of inputting the training set into the initialized semi-supervised anomaly detection model for model training includes multiple epochs, in each epoch, dividing the training set into multiple batches, each batch including random normal samples and abnormal samples, and inputting the multiple batches into the initialized semi-supervised anomaly detection model for model training; After completing the processing of all batches in an epoch, the performance of the model is evaluated through the validation set. If the model performance does not meet the predetermined standard, the training of the next epoch will continue; If the model performance meets the predetermined standards, the training is stopped and the model parameters are saved.

3. The method according to claim 2, characterized in that The semi-supervised anomaly detection model includes an input module, an Encoder-Decoder module and a VAE-GAN encoder; Input each batch into the initialized semi-supervised anomaly detection model for model training, including: The normal samples and abnormal samples in the batch are input to the input module of the semi-supervised anomaly detection model. The input module outputs the normal samples to the Encoder-Decoder module and outputs the abnormal samples to the VAE-GAN encoder. The Encoder-Decoder module extracts features from the image data of normal samples, generates image reconstruction data based on the extracted features, and outputs the extracted features to the pseudo labeler; The VAE-GAN encoder extracts features from the image data of abnormal samples and generates potential abnormal information and abnormal reconstruction data; The pseudo-labeler generates pseudo-labels based on the features extracted by the Encoder-Decoder module; A loss function is constructed based on image reconstruction data, potential abnormal information, and abnormal reconstruction data, and the model parameters are updated through back propagation. After each batch training, the learning rate is adjusted according to the training progress.

4. The method according to claim 1, characterized in that: The semi-supervised anomaly detection model also includes a memory module and an anomaly detection module; A memory module, used to store historical abnormal images and their corresponding abnormal detection results; The anomaly detection module obtains the anomaly detection result of the product to be detected based on the image data of the product to be detected, the image reconstruction data, potential anomaly information, anomaly reconstruction data, pseudo labels, historical anomaly image data stored in the memory module and the corresponding anomaly detection results, and transmits the anomaly detection result to the memory module.

5. The method according to claim 4, characterized in that The Encoder-Decoder module includes an Encoder unit and a Decoder unit; The encoder unit includes a first convolutional layer, a first pooling layer, a second convolutional layer, and a second pooling layer connected end to end in sequence; The first convolution layer extracts the features of the image data, and the first pooling layer performs a pooling operation on the features extracted by the first convolution layer. The second convolution layer performs a convolution operation on the output of the first pooling layer to extract the features of the image data. The second pooling layer performs dimensionality reduction processing on the output of the second convolution layer to obtain the final features of the image data. The output of the second pooling layer is used as the output of the Encoder unit and connected to the input of the Decoder unit and the pseudo labeler. The decoder unit includes a transposed convolution layer, an upsampling layer, and a decoder convolution layer connected end to end; The transposed convolution layer receives the output of the second pooling layer and performs a transposed convolution operation on the final features of the image data; the upsampling layer upsamples the output of the transposed convolution layer to increase the feature space dimension; The decoder convolution layer performs a convolution operation on the output of the upsampling layer to generate image reconstruction data. The output of the decoder convolution layer is used as the output of the decoder unit and connected to the input of the anomaly detection module.

6. The method according to claim 1 according to claim 3, characterized in that The VAE-GAN encoder includes an encoder and a decoder connected end to end. The encoder is used to generate potential abnormal information based on image data; A decoder for generating reconstructed image data based on the potential anomaly information; The encoder includes a first convolution layer, a first activation function layer, a first pooling layer, a second convolution layer, a second activation function layer, a second pooling layer, a Flatten layer, a first fully connected layer, a third activation function layer, and a second fully connected layer, which are connected end to end in sequence; Among them, the first convolution layer extracts the features of the image data, the first activation function layer performs nonlinear processing on the output of the first convolution layer; the first pooling layer performs a pooling operation on the output of the first encoder activation function layer to reduce the feature dimension after nonlinear processing; the second convolution layer performs a convolution operation on the output of the first pooling layer to extract the features of the image data; the second activation function layer performs nonlinear processing on the output of the second convolution layer; the second pooling layer performs a pooling operation on the output of the second activation function layer; the Flatten layer converts the output of the second pooling layer into a one-dimensional feature vector; the first fully connected layer performs feature compression on the one-dimensional feature vector; the third activation function layer performs nonlinear processing on the output of the first fully connected layer; the second fully connected layer performs feature mapping on the output of the third activation function layer to generate potential abnormal information of the image data; The decoder comprises a first transposed convolution layer, a first activation function layer, an upsampling layer, a second transposed convolution layer, a second activation function layer, and a decoder convolution layer which are sequentially connected end to end; Among them, the first transposed convolution layer performs a transposed convolution operation on the potential abnormal information received from the encoder; the first activation function layer performs nonlinear processing on the output of the first transposed convolution layer; the upsampling layer upsamples the output of the first activation function layer; the second transposed convolution layer performs a transposed convolution operation on the output of the upsampling layer; the second activation function layer performs nonlinear processing on the output of the second transposed convolution layer; the decoder convolution layer performs a convolution operation on the output of the second activation function layer to obtain reconstructed image data.

7. The method according to claim 6, characterized in that The pseudo labeler comprises an input layer, a fully connected layer, an activation function layer, and an output layer connected end to end in sequence; The input layer receives the final features of the image data output by the Encoder unit and transmits them to the fully connected layer; The fully connected layer performs spatial mapping on the final features of the image data to generate the number of feature categories; An activation function layer generates a distribution probability of each feature category based on the number of feature categories through a Softmax function; The output layer generates anomaly labels based on the distribution probability of each feature category and transmits them to the anomaly detection module.

8. The method according to claim 3, characterized in that The loss function is shown in formula (1); L total =ω 1. L recon +oh 2. L anomaly +oh 3. L latent +oh 4. L pseudo (1); Among them, L total is the total loss function; L recon is the loss function of the Encoder-Decoder module; L anomaly is the loss function of the VAE-GAN encoder; L latent is the regularization loss of potential abnormal information; L pseudo is the loss function of the pseudo labeler.

9. The method according to claim 8, characterized in that The loss function of the Encoder-Decoder module is shown in formula (2); Where n is the number of samples, y i is the image data of the i-th sample, is the reconstructed data of the i-th sample; The loss function of the VAE-GAN encoder is calculated as shown in formula (3); Where n is the number of samples, a i is the image data of the i-th sample, a i Reconstruct the data for the anomaly of the i-th sample; The regularization loss of potential abnormal information is shown in formula (4); L latent =D KL (q(z|xi)||p(z)) (4) Among them, D KL is the KL divergence, q(z|xi) is the potential abnormal information, and p(z) is the prior distribution of the abnormal information.

10. The method according to claim 9, characterized in that The pseudo labeler loss function is shown in formula (5); Where n is the number of samples, y ic is the encoding of the label of the i-th sample, is the pseudo label encoding of the i-th sample.

Images