Adversarial training-based pseudo-anomalous sample surface defect detection method and device

Abstract

The present application is directed to the problem of "sample imbalance" of surface defects of industrial products in practical application, and the problem that efficient positioning and classification of surface defects is still a challenge due to the similarity between non-significant defects and different defects, and proposes a surface defect detection method and equipment with adversarial training of pseudo-abnormal samples, which can adapt to the detection data set without additional abnormal sample learning, thereby achieving effective detection of surface defects of industrial products. In the method, a mixed object detector, a feature encoder and decoder and a classifier are used. The object detector can screen a round of abnormal areas before formal detection to achieve high efficiency of the entire model. The feature encoder and decoder have adversarial training of pseudo-abnormal samples, which can enable the model to complete normal and abnormal learning of the data set without real abnormal samples, and reduce the influence of "sample imbalance" in practical application on the detection result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision and defect detection, and particularly relates to a pseudo-anomaly sample surface defect detection method and device with adversarial training. BACKGROUND

[0002] Object surface defect detection is an indispensable part of industrial automation production, and researching object surface defect detection technology has strong practical significance. The general process of object surface defect detection method based on traditional machine vision includes image processing, feature extraction and target classification. In actual work, after obtaining image data, image enhancement and image segmentation processing techniques are usually used to separate the defect area in advance, and then the defect area features are manually extracted according to the defect target feature rules designed by human, and finally some classification algorithms are applied to classify the defect features.

[0003] However, traditional machine vision defect detection technology has encountered many problems and challenges in practical application, especially in the era when traditional image features used to distinguish defects and non-defects are manually designed based on experience. The features of traditional image feature extraction operators are usually low-level. In the case of complex scene changes such as illumination changes, perspective distortion, occlusion, object deformation, etc., the extracted features are usually not robust enough to handle them, so many algorithms are not applicable in actual environment. While the rapid development of deep learning has made it more and more widely used in the field of defect detection.

[0004] The object surface defect detection method based on deep learning is an end-to-end detection scheme that extracts defect features autonomously by convolutional neural network. It can more accurately describe and understand defect information than manually designed rule extraction features, and the detection is more accurate. In recent years, deep learning has been proven to be very successful in object detection and classification, face detection, pattern recognition, fault diagnosis, target tracking and other various image-based applications. It has been proven to be robust to background, lighting, color, shape, size and intensity when detecting patterns in images. When detecting complex surface defects in industrial environments, the effect of using deep learning-based methods is particularly ideal. Common deep learning methods can be divided into supervised learning and unsupervised learning.

[0005] But when used for surface defect detection of industrial products, the identification of unbalanced samples is a difficulty based on deep learning methods. In deep learning, when the model is trained, it is usually necessary to balance the number of samples of various classes in the sample set. However, this ideal situation rarely occurs in reality. In more cases, the data of normal samples in the data set usually accounts for the majority, and the number of defect samples accounts for only a small part of the total samples. This phenomenon is called "unbalanced sample" phenomenon. The unbalanced sample identification problem mainly exists in supervised learning tasks. The occurrence of this problem will cause the algorithm to pay more attention to the class with more data, and underestimate the class with less data, thereby affecting the generalization and prediction ability of the model in the test data. Unsupervised learning can largely solve this problem.

[0006] However, due to the similarity between non-significant defects and different defects, efficient positioning and classification of surface defects is still a challenge. In addition, the shape and characteristics of defects of different industrial products are complex and varied, and the imaging angle and lighting condition when taking pictures of the product are also different, which makes it difficult for the defect detection model to adapt to the defect detection needs of different types of products, and the performance is obviously reduced. Therefore, designing a defect detection model with domain adaptation ability is a major research direction in the future. SUMMARY

[0007] In view of the deficiencies of the prior art, the present application provides a pseudo-abnormal sample surface defect detection method and device with adversarial training. The purpose is to solve the problem of few abnormal samples in surface defect detection and complete more efficient abnormal position positioning, and because specific abnormal samples are not used in training, this model has a certain domain adaptation ability.

[0008] According to a first aspect of the present application, a pseudo-abnormal sample surface defect detection method with adversarial training comprises the following steps:

[0009] S1: constructing an object detector, the object detector comprising a feature extraction network and an RPN;

[0010] S2: feeding a normal object into the object detector to obtain n target sub-samples;

[0011] S3: constructing a feature encoder-decoder, the feature encoder-decoder being composed of a CAE, the CAE comprising an encoder, a first decoder and a second decoder;

[0012] S4: feeding any target sub-sample in the n target sub-samples obtained by the object detector into the encoder, feeding data unrelated to the surface defect detection dataset into the encoder as a pseudo-abnormal sample, performing normal sample training through the first decoder to reconstruct a normal object, and performing pseudo-abnormal sample adversarial training through the second decoder to decode a pseudo-abnormal object;

[0013] S5: continue to repeat S4 for the remaining n-1 target sub-samples until one object ends; meanwhile, update the network parameters according to each loss to complete the training of the encoder;

[0014] S6: calculate the absolute difference between each target sub-sample and its reconstruction, and the absolute difference between the pseudo abnormal sample and its decoding result;

[0015] S7: build a classifier and train the classifier;

[0016] S8: in the test phase, remove the second decoder, and input the test sample into the object detector, CAE and classifier which have been trained to obtain the surface defect detection result.

[0017] Further, in step S1, the feature extraction network adopts Resnet-50, which is pre-trained on ImageNet and then trained again using the industrial product surface defects collected to improve the precision; the RPN is used to obtain the proposal frame, and each proposal frame is cropped to obtain each suspected target region as a target sub-sample. In the training stage, the proposal frame is randomly obtained.

[0018] Further, in step S3, each encoder is composed of three convolutional layers, each followed by a max-pooling layer with a filter size of 2x2 and a step of 2; the convolutional layer is composed of a filter of 3x3; each convolutional layer is followed by ReLU as the activation function; the first two convolutional layers are composed of 32 filters, and the third convolutional layer is composed of 16 filters, and the latent representation is composed of 16 activation maps with a size of 8x8;

[0019] Each decoder starts from the up-sampling layer to increase the spatial support of the activation map by 2 times; the up-sampling operation is based on the nearest neighbor interpolation; after up-sampling, a convolutional layer with 16 3x3 filters is applied; the first up-sampling layer and the convolutional layer are followed by another two up-sampling layers and convolutional layers, and the last convolutional layer is composed of a single convolutional filter.

[0020] Further, in step S4, for the first decoder d that reconstructs the normal object, the MSE loss function is used as the loss function L d :

[0021]

[0022] where h and w are the height and width of the input normal sample respectively and h=w=64, x ij is the grayscale value of the i-th row and j-th column normal sample, is x ij the value obtained after the decoder, θ e is the parameter of the encoder, and θd parameters of the first decoder;

[0023] For the second decoder branch d' decoding pseudo-exceptional objects, the MSE loss function is used as the loss function L d′ :

[0024]

[0025] where h, w are the height and width of the input pseudo-exceptional sample respectively and h = w = 64, x ij is the grayscale value of the pseudo-exceptional sample in the ith row and jth column with h rows and w columns, is x ij the value obtained after decoding

[0026] the value obtained after decoding e is the parameter of the encoder, and θ d′ is the parameter of the second decoder.

[0027] Further, all target sub-samples of a normal object obtained by the object detector complete steps S3-S4, while the network parameters are updated in real time;

[0028] The update rule of the network parameters is as follows:

[0029]

[0030]

[0031]

[0032] where η is the learning rate, and λ is the weight of the backpropagation gradient. During the training process, the first decoder adopts the gradient descent method, and the second decoder adopts the gradient ascent method.

[0033] Further, step S6 specifically includes:

[0034] After training the encoder until convergence, the parameters θ e , a d are frozen for subsequent training of the classifier.

[0035] After each target sub-sample is reconstructed by the first decoder, the absolute difference between the reconstructed image and the original target sub-sample is calculated for subsequent training of the classifier:

[0036] The absolute difference between the result of the pseudo-exceptional sample after passing through the second decoder and the original pseudo-exceptional sample is calculated for subsequent training of the classifier.

[0037] Further, step S7 specifically includes:

[0038] A binary classifier is constructed, and a binary cross-entropy loss is used to train the classifier; the binary cross-entropy function is as follows:

[0039]

[0040] Wherein is the predicted probability of the absolute difference, y is the true label, y = 1 when it is the absolute difference of the pseudo abnormal sample; y = 0 when it is the absolute difference of each target sub-sample; Adam is used for optimization.

[0041] Further, step S8 specifically comprises:

[0042] Remove the second decoder in the feature encoder-decoder;

[0043] After obtaining n sub-samples through the object detector for the test sample, the n sub-samples are sequentially sent into the feature encoder-decoder after removing the second decoder, to obtain the absolute difference, and then sent into the classifier to obtain the prediction result of each sub-sample;

[0044] After completing the sample detection of all objects, the abnormal scores of each sub-sample obtained by the classifier are sorted, a threshold is set, and if the threshold is exceeded, it is judged as abnormal, and the position information stored in the detection picture is displayed to obtain the abnormal detection area.

[0045] Preferably, the threshold is 0.5.

[0046] According to another aspect of the present application, an electronic device comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the pseudo abnormal sample surface defect detection method.

[0047] The technical solution provided by the present application has the following beneficial effects:

[0048] (1) The present method proposes a pseudo abnormal sample surface defect detection method with adversarial training, which can adapt to the detection data set without the need for additional abnormal sample learning, so as to achieve effective detection of industrial product surface defects, and has certain field self-adaptation ability.

[0049] (2) Compared with other surface defect detection methods, the present method further reduces the area where the defect is located by adding an object detector, which can effectively reduce the interference of the background, so that the model can more accurately locate the defect position. BRIEF DESCRIPTION OF DRAWINGS

[0050] The present application will be further described below in conjunction with the drawings and examples, wherein:

[0051] Figure 1A framework diagram of a pseudo-abnormal sample surface defect detection method with adversarial training is proposed in the present application.

[0052] Figure 2 An RPN structure diagram used in the embodiments of the present application is provided.

[0053] Figure 3 Proposal boxes obtained by an object detector in the AITEX dataset in the embodiments of the present application are provided.

[0054] Figure 4 A CAE structure diagram used in the embodiments of the present application is provided.

[0055] Figure 5 Subsample original images of normal samples and abnormal samples obtained by an object detector, pseudo-abnormal sample original images, reconstructed images and difference images generated by the three in the embodiments of the present application are provided.

[0056] Figure 6 The final detection result of the AITEX dataset in the embodiments of the present application is provided.

[0057] Figure 7 A schematic diagram of an electronic device in the embodiments of the present application is provided. DETAILED DESCRIPTION

[0058] In order to have a clearer understanding of the technical features, objects and effects of the present application, the specific embodiments of the present application will be described in detail with reference to the drawings.

[0059] The present application aims at the problem of "sample imbalance" in the actual application of industrial product surface defects. Due to the similarity between non-significant defects and different defects, efficient positioning and classification of surface defects is still a challenging problem. A pseudo-abnormal sample surface defect detection method with adversarial training is proposed, which can adapt to the detection dataset without the need for additional abnormal sample learning, so as to achieve effective detection of industrial product surface defects. The method includes a mixed object detector, a feature encoder and decoder, and a classifier. The object detector can screen a round of abnormal areas before formal detection to achieve the efficient detection capability of the entire model. The feature encoder and decoder have adversarial training of pseudo-abnormal samples, which can enable the model to complete the learning of normal and abnormal datasets without real abnormal samples, and reduce the influence of "sample imbalance" in actual application on the detection result.

[0060] REFERENCE Figure 1 The present application provides a pseudo-abnormal sample surface defect detection method with adversarial training, which mainly includes the following steps:

[0061] S1: An object detector is constructed, which includes a feature extraction network and an RPN.

[0062] S2: sending the normal object into the object detector to obtain n target sub-samples;

[0063] S3: constructing a feature encoder-decoder, the feature encoder-decoder is composed of a CAE (Convolutional Auto-Encoder), the CAE includes an encoder, a first decoder and a second decoder;

[0064] S4: sending any target sub-sample in the n target sub-samples obtained by the object detector into the encoder, sending data irrelevant to the surface defect detection dataset as a pseudo abnormal sample into the encoder, training the normal sample through the first decoder to reconstruct the normal object, and training the pseudo abnormal sample through the second decoder to decode the pseudo abnormal object;

[0065] S5: continuing to repeat S4 for the remaining n-1 target sub-samples until one object ends; meanwhile, updating the network parameters according to each loss to complete the training of the encoder;

[0066] S6: calculating the absolute difference between each target sub-sample and its reconstruction, and calculating the absolute difference between the pseudo abnormal sample and its decoding result;

[0067] S7: constructing a classifier and training the classifier;

[0068] S8: in the test phase, removing the second decoder, sending the test sample into the object detector, the CAE and the classifier which have been trained to obtain the surface defect detection result.

[0069] Next, the above steps are described in detail:

[0070] In the embodiment, in step S1, the object detector is constructed, and the steps are specifically as follows:

[0071] An object detector composed of a feature extraction network and an RPN (Region Proposal Network) is constructed, the structure of the RPN is as shown in Figure 2 There is a sliding window on the feature map to generate an initial proposal box for each position, and the region score layer judges whether the initial proposal box is the required foreground box, and the region proposal layer obtains the position of the initial proposal box, and finally obtains the proposal box;

[0072] The object detector is mainly used to extract the target suspected area, reduce the detection range and improve the detection accuracy.

[0073] In the embodiment, in step S2, the normal object is sent into the object detector to obtain n target sub-samples, and the steps are specifically as follows:

[0074] The feature extraction network adopts Resnet-50, which is pre-trained on ImageNet and then trained again on the collected industrial product surface defects to improve the precision.

[0075] The RPN is used to obtain the proposal frame, and each proposal frame is cropped to obtain a suspected target region, each region being a target sub-sample. Since only normal samples are used in the training stage, a random proposal frame is obtained, and the RPN result is used in the test stage. Specifically, the proposal frame obtained by the object detector in the AITEX data set (fabric defect data set) is as shown in Figure 3 .

[0076] In this embodiment, in step S3, a feature encoder-decoder is constructed, and the steps are specifically as follows:

[0077] The feature encoder-decoder is composed of a CAE (Convolutional Auto-Encoder), which includes an encoder and two decoders. The first decoder is only used for normal sample training, and the second decoder is only used for pseudo abnormal sample adversarial training. The structure of the encoder and the decoder is as shown in Figure 4 .

[0078] Encoder part: Each encoder is composed of three convolutional layers, each followed by a max-pooling layer with a filter size of 2x2 and a stride of 2. The convolutional layer is composed of 3x3 filters. Each convolutional layer is followed by ReLU as the activation function. The first two convolutional layers are composed of 32 filters, and the third layer is composed of 16 filters. The latent representation is composed of 16 activation maps with a size of 8x8.

[0079] Decoder part: Each decoder starts with an up-sampling layer to increase the spatial support of the activation map by a factor of 2. The up-sampling operation is based on nearest-neighbor interpolation. After up-sampling, a convolutional layer with 16 3x3 filters is applied. The first up-sampling and convolutional layer is followed by another two up-sampling and convolutional layers. The last convolutional layer is composed of a single convolutional filter.

[0080] In this embodiment, the steps of step S4 are specifically as follows:

[0081] S401: For the first decoder d that reconstructs the normal object, a MSE loss function is used as the loss function L d :

[0082]

[0083] where h and w are the height and width of the input normal sample respectively, h = w = 64, x ij is the grayscale value of the i-th row and j-th column of the normal sample, For x ij The value obtained after decoding, θ e θ is a parameter of the encoder. d These are the parameters for the first decoder;

[0084] S402: For the second decoder branch d' that decodes the pseudo-exception object, the MSE loss function is used as the loss function L. d′ :

[0085]

[0086] Where h and w are the height and width of the input pseudo-anomaly sample, respectively, and h = w = 64, x ij This represents the grayscale value of the pseudo-anomaly sample in the i-th row and j-th column, with h as the row and w as the column. For x ij The value obtained after decoding, θ e θ is a parameter of the encoder. d′ These are the parameters for the second decoder.

[0087] In this embodiment, in step S5, the network parameters are updated to complete the encoder training. The specific steps are as follows:

[0088] The S3-S4 steps are performed on all target sub-samples obtained by the object detector from a normal sample, while the network parameters are updated in real time.

[0089] The update rules for the above network parameters are as follows:

[0090]

[0091]

[0092]

[0093] η is the learning rate, and λ is the weight of the backpropagation gradient. During training, the first decoder uses gradient descent, and the second decoder uses gradient ascent. To ensure convergence, λ must be less than 1; otherwise, the gradient ascent step size will be larger than the gradient descent step size.

[0094] In this embodiment, in step S6, the absolute difference between each target sub-sample and its reconstructed form is calculated, and the absolute difference between the pseudo-anomaly sample and its decoding result is calculated. The specific steps are as follows:

[0095] After training the encoder until convergence, freeze the parameters θ. e θ d , used for training the classifier;

[0096] Calculate the absolute difference between each target sub-sample and the original target sub-sample after obtaining the reconstructed image through the first decoder, for subsequent classifier training:

[0097] Calculate the absolute difference between the result of the pseudo abnormal sample after passing through the second decoder and the original pseudo abnormal sample, for subsequent classifier training.

[0098] Specifically, the sub-sample original image of the normal sample and the abnormal sample obtained by the object detector and the pseudo abnormal sample original image, the reconstructed images and the difference images generated by the three are as shown in Figure 5 .

[0099] In this embodiment, in step S7, the classifier is constructed and trained, and the steps are specifically:

[0100] A binary classifier is constructed, and the binary cross entropy loss is used to train the classifier, and the binary cross entropy function L is as follows:

[0101]

[0102] Wherein is the predicted probability of the absolute difference in the foregoing step, y is the true label, when it is the absolute difference of the pseudo abnormal sample, y=1; when it is the absolute difference of each sub-sample, y=0. Adam is used for optimization.

[0103] In this embodiment, in step S8, the test sample is sent into the object detector, CAE and classifier which have been trained to obtain the defect detection result, and the steps are specifically:

[0104] Remove the second decoder in the feature encoder-decoder;

[0105] After obtaining n sub-samples through the object detector, the n sub-samples are sequentially sent into the feature encoder-decoder without the second decoder to obtain the absolute difference, and then sent into the classifier to obtain the prediction result of each sub-sample;

[0106] After completing the sample detection of an object, sort the abnormal scores of each sub-sample according to the classifier, set the threshold value to 0.5, and judge as abnormal if it exceeds the threshold value, and display the abnormal detection area in the picture by storing the position information to obtain the final detection result on the AITEX dataset (fabric defect dataset), as shown in Figure 6 .

[0107] In order to verify the outstanding effect of the detection method of the present application, the index AUC of the present application method is compared with the following eight methods, and the eight methods are respectively: KDAD (Knowledge Distillation for Anomaly Detection, knowledge distillation anomaly detection), DeepSAD (Deep Semi-Supervised Anomaly Detection, deep semi-supervised anomaly detection), DevNet (Deviation Network, deviation network), SAOE (Synthetic Anomalies with Outlier Exposure, synthetic anomalies with outlier exposure), FLOS (Focal Loss-Driven Classifier, focal loss-driven classifier), MLEP (Margin Learning Embedded Prediction, margin learning embedded prediction), DRA (Disentangled Representations of Abnormalities, disentangled representations of abnormalities), and LBFE (Local Binary Matrix Feature Extraction, local binary matrix feature extraction). The comparison results are shown in Table 1:

[0108] Table 1 Comparison of index AUC of the present application with other existing methods

[0109]

[0110] As can be seen from Table 1, the evaluation index of the pseudo-anomaly sample surface defect detection method with adversarial training provided by the present application is better than that of other existing methods.

[0111] The following will make specific description on the related details in the method:

[0112] (1) The object detector is specifically described as follows:

[0113] For the object detector, the feature map generated by using ResNet-50 as a feature extraction network is sent to RPN to generate a proposal box.

[0114] For RPN, the input is the feature map generated by the feature extraction network, and initial proposal boxes with different scales and aspect ratios are generated for each point (anchor point) on the map. The loU (Intersection over Union) of the initial proposal box and the label box is compared, and if the loU is higher than a certain threshold, the initial proposal box is labeled as a foreground box, otherwise it belongs to the background box; for the foreground box, the 4 position offsets (x, y, w, h) of the real label box are also calculated; compare the labeled initial proposal box (with foreground and background class labels and position offset labels) with the label box to learn how to extract the foreground box. After learning how to extract the foreground box, the output probability value of the region score layer is used to determine whether it is a foreground box; if the position offset value of the region proposal layer is integrated into the coordinates of the initial proposal box to obtain the coordinates of the actual box; in this way, the foreground box is obtained, but they have different scales and lengths, so a ROl pooling (Region of interest pooling) layer is needed to obtain a unified size to obtain the final proposal box.

[0115] (2) The feature encoder-decoder is described as follows:

[0116] For the feature encoder-decoder, CAE is used, and two decoder branches are equipped. At the same time, a skip connection is also equipped. Each convolutional layer in the encoder skips the connection with the corresponding convolutional layer in each standard or adversarial decoder, which belongs to the same CAE and follows the U-Net architecture. A skip connection is also added between the last convolutional layer of the encoder and the third convolution of the corresponding binary classifier. Since the first part of the binary classifier is consistent with the encoder in architecture, the corresponding activation map can be summarized, and both form an 8x8x16 component tensor.

[0117] (3) The classifier is described as follows:

[0118] The absolute difference between the input of the encoder and the output of the decoder is taken as the input of the classifier. The classifier will output an anomaly score for each input according to its own learning result, and set a threshold value. When the anomaly score is higher than the set threshold value, it is identified as an anomaly. When the anomaly scores of all sub-samples are returned to the original object map, the abnormal position can be obtained.

[0119] In order to better implement the above-mentioned method of the present application, as Figure 7As shown, the present application also provides a physical structure diagram of an electronic device, which can include a processor 610, a communications interface 620, a memory 630 and a communications bus 640, wherein the processor 610, the communications interface 620 and the memory 630 complete mutual communication through the communications bus 640. The processor 610 can call the logical instructions in the memory 630 to execute the steps of the above-mentioned surface defect detection method with adversarial training of pseudo-abnormal samples, specifically including: S1: constructing an object detector, the object detector including a feature extraction network and an RPN; S2: sending a normal object into the object detector to obtain n target sub-samples; S3: constructing a feature encoder-decoder, the feature encoder-decoder being composed of a CAE, the CAE including an encoder, a first decoder and a second decoder; S4: sending any target sub-sample in the n target sub-samples obtained by the object detector into the encoder, sending data irrelevant to the surface defect detection dataset into the encoder as a pseudo-abnormal sample, performing normal sample training through the first decoder to reconstruct a normal object, performing adversarial training of the pseudo-abnormal sample through the second decoder to decode a pseudo-abnormal object; S5: continuing to repeat S4 for the remaining n-1 target sub-samples until an object ends; meanwhile, updating the network parameters according to each loss to complete the training of the encoder; S6: calculating the absolute difference between each target sub-sample and its reconstruction, and calculating the absolute difference between the pseudo-abnormal sample and its decoding result; S7: constructing a classifier and training the classifier; S8: in the test phase, removing the second decoder, sending a test sample into the object detector, the CAE and the classifier which have been trained to obtain a surface defect detection result.

[0120] In addition, the logical instructions in the above-mentioned memory 630 can be implemented in the form of a software functional unit and sold or used as an independent product, which can be stored in a computer-readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk and various program code storage media.

[0121] In yet another aspect, the embodiments of the present application also provide a storage medium having stored thereon a computer program, which, when executed by a processor, implements the steps of the above-mentioned surface defect detection method with adversarial training of pseudo abnormal samples, and specifically includes: S1: constructing an object detector, the object detector including a feature extraction network and an RPN; S2: feeding a normal object into the object detector to obtain n target sub-samples; S3: constructing a feature encoder-decoder, the feature encoder-decoder being composed of a CAE, the CAE including an encoder, a first decoder and a second decoder; S4: feeding any target sub-sample of the n target sub-samples obtained by the object detector into the encoder, feeding data irrelevant to the surface defect detection dataset into the encoder as a pseudo abnormal sample, performing normal sample training through the first decoder to reconstruct a normal object, performing adversarial training of the pseudo abnormal sample through the second decoder to decode a pseudo abnormal object; S5: continuing to repeat S4 for the remaining n-1 target sub-samples until an object is completed; meanwhile, updating network parameters according to each loss to complete training of the encoder; S6: calculating the absolute difference between each target sub-sample and its reconstruction, and calculating the absolute difference between the pseudo abnormal sample and its decoding result; S7: constructing a classifier and training the classifier; and S8: in a test phase, removing the second decoder, feeding a test sample into the object detector, the CAE and the classifier which have been trained to obtain a surface defect detection result.

[0122] It should be noted that, in this document, the terms "comprising", "containing", or any other similar term are intended to encompass non-exclusive inclusion, such that a process, method, article or system that comprises a list of elements does not necessarily include only those elements in the list, but can include other elements not expressly listed or inherent to such process, method, article or system. Without more limitations, an element defined by the phrase "comprising a" does not exclude the presence of additional identical elements in the process, method, article or system that includes the element.

[0123] The above-mentioned embodiment numbers of the present application are only for description, and do not represent the advantages or disadvantages of the embodiments. In the unit claims in which several devices are listed, several of these devices can be embodied by the same hardware item. The use of the words first, second, and third does not indicate any order, and these words can be interpreted as identifiers.

[0124] The above is only the preferred embodiments of the present application, and does not limit the patent scope of the present application, and any equivalent structure or equivalent process transformation using the contents of the present application specification and drawings, or direct or indirect application in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for surface defect detection of pseudo-anomalous samples with adversarial training, characterized in that, The method comprises the following steps: S1: constructing an object detector, the object detector comprising a feature extraction network and an RPN; S2: feeding a normal object into the object detector to obtain n target sub-samples; S3: constructing a feature encoder-decoder, the feature encoder-decoder comprising a CAE, the CAE comprising an encoder, a first decoder and a second decoder; S4: feeding any target sub-sample obtained by the object detector into the encoder, feeding data unrelated to the surface defect detection dataset into the encoder as a pseudo abnormal sample, performing normal sample training through the first decoder to reconstruct the normal object, and performing adversarial training of the pseudo abnormal sample through the second decoder to decode the pseudo abnormal object; S5: continuing to repeat S4 for the remaining n-1 target sub-samples until an object is completed; meanwhile, updating the network parameters according to the losses to complete the training of the encoder; S6: calculating the absolute difference between each target sub-sample and its reconstructed result, and calculating the absolute difference between the pseudo abnormal sample and its decoded result; S7: constructing a classifier and training the classifier; S8: in the testing stage, removing the second decoder, feeding a test sample into the object detector, the CAE and the classifier which have been trained to obtain a surface defect detection result; In step S4, for the first decoder reconstructing the normal object d , the MSE loss function is used as the loss function : wherein, h , w are the height and width of the input normal sample, respectively, and h = w = 64, is the number of the normal sample in the row h is the number of the normal sample in the column w is the gray scale value of the normal sample in the row i is the gray scale value of the normal sample in the column j is the gray scale value of the normal sample in the row is the gray scale value of the normal sample in the column is the value obtained after the decoder, is the parameter of the encoder, is the parameter of the first decoder; For a second decoder branch decoding pseudo-exceptional objects d' , using an MSE loss function as loss function : wherein, h , w are height and width of the input pseudo abnormal samples, respectively, and h = w = 64, is the number of the pseudo abnormal samples in the h th row w th column, i th row j th column of the grayscale image, is the value of the pseudo abnormal sample, is the value obtained after decoding by the decoder, is the parameter of the encoder, is the parameter of the second decoder. Step S8 specifically comprises: removing the second decoder in the feature encoder-decoder; feeding the n sub-samples obtained after the test sample passes through the object detector into the feature encoder-decoder after the second decoder is removed in sequence to obtain the absolute difference, and then feeding the n sub-samples into the classifier to obtain the prediction result of each sub-sample; after the detection of all samples of an object is completed, sorting the abnormal scores of each sub-sample obtained by the classifier, setting a threshold, and determining that the object is abnormal if the abnormal score exceeds the threshold, and displaying the location information of the abnormal detection area in the detection picture.

2. The pseudo-anomalous specimen surface defect detection method of claim 1, wherein, In step S1, the feature extraction network adopts Resnet-50, which is pre-trained on ImageNet and then trained again using the surface defects of industrial products that can be collected to improve the precision; the RPN is used to obtain a proposal frame, each proposal frame is cropped to obtain a suspected target region, and each region is taken as a target sub-sample; in the training stage, the proposal frame is randomly obtained.

3. The pseudo-anomalous specimen surface defect detection method of claim 1, wherein, In step S3, each encoder consists of three convolutional layers, each followed by a max-pooling layer with a filter size of and a stride of 2; the convolutional layers consist of filters of ; each convolutional layer is followed by a ReLU as activation function; the first two convolutional layers consist of 32 filters, while the third convolutional layer consists of 16 filters, the latent representation consisting of 16 activation maps of size ; Each decoder starts from an up-sampling layer to increase the spatial support of the activation map by 2 times; The up-sampling operation is based on nearest neighbor interpolation; After upsampling, a 16-bit array was applied. The filter consists of convolutional layers; after the first upsampling layer and convolutional layer, there are two more upsampling layers and convolutional layers, and the last convolutional layer is composed of a single convolutional filter.

4. The pseudo-anomalous specimen surface defect detection method of claim 1, wherein, Step S5 specifically comprises: updating the network parameters in real time while completing steps S3-S4 for all target sub-samples of a normal object obtained by the object detector; The updating rule of the network parameters is as follows: is a learning rate, is a weight of the reverse gradient, and during the training process, the first decoder adopts a gradient descent manner and the second decoder adopts a gradient ascent manner.

5. The pseudo-anomalous specimen surface defect detection method of claim 1, wherein, Step S6 specifically comprises: After training the encoder until convergence, the parameters are frozen , for the training of the subsequent classifier; calculating the absolute difference between each target sub-sample and its reconstructed result after passing through the first decoder, which is used for subsequent classifier training: calculating the absolute difference between the result of the pseudo abnormal sample after passing through the second decoder and the original pseudo abnormal sample, which is used for subsequent classifier training.

6. The pseudo-anomalous specimen surface defect detection method of claim 1, wherein, Step S7 specifically comprises: constructing a binary classifier and using binary cross-entropy loss to train the classifier; the binary cross-entropy function is as follows: wherein is the predicted probability of absolute difference, is the true label, when the absolute difference is a pseudo-anomalous sample, ; when the absolute difference is each target subsample, ; is optimized using Adam.

7. The pseudo-anomalous specimen surface defect detection method of claim 1, wherein, The threshold is 0.

5.

8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the steps of the pseudo-anomalous sample surface defect detection method as claimed in any one of claims 1-7 when executing the program.

Images