Image defect detection and model training method and device, equipment, medium and product

Abstract

The invention discloses an image defect detection and model training method and device, equipment, a medium and a product, and the training method comprises the steps: inputting an image sample into an image defect detection model, and obtaining a feature result outputted by a feature mapping layer and a feature result outputted by a student network; performing feature constraint on the feature result output by the feature mapping layer to obtain a first loss item, and performing feature constraint on the feature result output by the feature mapping layer and the feature result output by the student network to obtain a second loss item; training an image defect detection model based on the first loss item and the second loss item; the image samples comprise a first image sample comprising a defect-free image, and a second image sample obtained by performing brightness adjustment and / or angle deflection processing on the first image sample. And constraining the feature distribution of the defect-free images with different brightness and / or angle deflection in the same feature space, so that the image change caused by the brightness and / or angle deflection change is ignored during image defect detection.

Description

TECHNICAL FIELD

[0001] The present application relates to image defect detection technology, and in particular to an image defect detection and model training method, device, equipment, medium and product. BACKGROUND

[0002] Appearance defect detection is a key step for product quality control. Industrial products need to be inspected before leaving the factory. With the development of deep learning algorithms, image defect detection methods based on deep learning algorithms gradually replace manual appearance defect detection due to their high efficiency. The image defect detection method based on deep learning is divided into supervised image defect detection method and unsupervised image defect detection method.

[0003] In related technologies, the supervised image defect detection method relies on manual labeling of defect image samples for model training. Therefore, the supervised image defect detection method needs to collect a large number of defect image samples for model training, and if the detection image has a defect that has not been labeled, it is easy to miss the report risk. The unsupervised image defect detection method usually includes image defect detection method based on memory bank search, image defect detection method based on image reconstruction and image defect detection method based on inverse distillation. Among them, the image defect detection method based on memory bank search cannot meet the requirements of industrial inspection due to its low computational efficiency. Although the image defect detection method based on image reconstruction can determine that the detection image has abnormal features through image reconstruction failure, the image defect detection model has certain generalization ability after long-term model training, and the image with defects can still be reconstructed successfully, resulting in image defect detection failure. In the image defect detection method based on inverse distillation, the image defect detection model includes a teacher network and a student network. If the detection image has a defect, the feature results output by the teacher network and the student network are not similar, and the area with defects in the image can be determined based on the dissimilar results.

[0004] However, although the image defect detection method based on inverse distillation can quickly detect defects in the image, in a complex industrial scene, such as appearance detection of metal products, if the image defect detection method based on inverse distillation is used for detection, the brightness and rotation angle of the detection image will directly affect the overall feature distribution of the detection image. If the illumination or shooting angle of the product changes greatly, it may cause a large deviation between the feature results output by the teacher network and the student network, and thus false positives occur. SUMMARY

[0005] Therefore, the embodiments of the present application provide an image defect detection and model training method, device, equipment, medium and product, aiming to improve the image defect detection accuracy in industrial scenes.

[0006] The technical solution of this application embodiment is implemented as follows:

[0007] In a first aspect, embodiments of this application provide a training method for an image defect detection model, wherein the image defect detection model is a backdistillation model comprising a sequentially connected teacher network, a feature mapping layer, a single-class bottleneck embedding layer, and a student network, and the training method includes:

[0008] The image sample is input into the image defect detection model to obtain the feature results output by the feature mapping layer and the feature results output by the student network.

[0009] The feature results output by the feature mapping layer are subjected to feature constraints to obtain a first loss term, and the feature results output by the feature mapping layer and the feature results output by the student network are subjected to feature constraints to obtain a second loss term.

[0010] The image defect detection model is trained based on the first loss term and the second loss term;

[0011] The image samples include: a first image sample comprising a defect-free image, and a second image sample obtained by adjusting the brightness and / or rotating the angle of the first image sample.

[0012] In the above scheme, the step of inputting image samples into the image defect detection model to obtain the feature results output by the feature mapping layer and the feature results output by the student network includes:

[0013] The image sample is input into the pre-trained teacher network, and after being encoded by the teacher network and mapped by the feature mapping layer, the feature result output by the feature mapping layer is obtained.

[0014] The feature results output by the feature mapping layer are fused by the single-class bottleneck embedding layer and then decoded by the student network to obtain the feature results output by the student network.

[0015] In the above scheme, the step of inputting image samples into the image defect detection model to obtain the feature results output by the feature mapping layer and the feature results output by the student network includes:

[0016] The first image sample is input into the image defect detection model to obtain a first feature result output by the feature mapping layer and a second feature result output by the student network; and...

[0017] The second image sample is input into the image defect detection model to obtain the third feature result output by the feature mapping layer.

[0018] In the above scheme, the step of performing feature constraints on the feature results output by the feature mapping layer to obtain a first loss term includes:

[0019] The first feature result and the third feature result are subjected to feature constraints to obtain the first loss term;

[0020] The second loss term is obtained by subjecting feature constraints to the feature results output by the feature mapping layer and the feature results output by the student network, including:

[0021] The first feature result and the second feature result are subjected to feature constraints to obtain the second loss term.

[0022] In the above scheme, the step of performing feature constraints on the first feature result and the third feature result to obtain the first loss term includes:

[0023] Calculate the cosine similarity between the first feature result and the third feature result to obtain the first loss term;

[0024] The step of performing feature constraints on the first feature result and the second feature result to obtain the second loss term includes:

[0025] The cosine similarity between the first feature result and the third feature result is calculated to obtain the second loss term.

[0026] In the above scheme, training the image defect detection model based on the first loss term and the second loss term includes:

[0027] Based on the first loss term and the second loss term, update the weights of the feature mapping layer, the single-class bottleneck embedding layer, and the student network.

[0028] Secondly, embodiments of this application provide an image defect detection method, the method comprising:

[0029] The image to be detected is input into the image defect detection model to obtain an anomaly map;

[0030] Based on the anomaly map, it is determined whether the image to be detected has defects;

[0031] The image defect detection model is trained based on the method described in the first aspect.

[0032] In the above scheme, the step of inputting the image to be detected into the image defect detection model to obtain an anomaly map includes:

[0033] The image to be detected is input into the image defect detection model to obtain the fourth feature result output by the feature mapping layer of the image defect detection model and the fifth feature result output by the student network of the image defect detection model.

[0034] Calculate the distance matrix between each layer of the fourth feature result and the fifth feature result;

[0035] Interpolate the distance matrices of each layer, and add the interpolated distance matrices of the same size to generate an anomaly score matrix.

[0036] The anomaly score matrix is ​​denoised to obtain an anomaly map.

[0037] In the above scheme, calculating the distance matrix of each layer between the fourth feature result and the fifth feature result includes:

[0038] Calculate the cosine similarity between the fourth feature result and the fifth feature result of each layer to obtain the distance matrix of each layer.

[0039] Thirdly, embodiments of this application provide a training apparatus for an image defect detection model, wherein the image defect detection model is a reverse distillation model comprising a sequentially connected teacher network, a feature mapping layer, a single-class bottleneck embedding layer, and a student network, and the training apparatus includes:

[0040] The first processing module is used to input image samples into the image defect detection model and obtain the feature results output by the feature mapping layer and the feature results output by the student network.

[0041] The second processing module is used to perform feature constraints on the feature results output by the feature mapping layer to obtain a first loss term, and to perform feature constraints on the feature results output by the feature mapping layer and the feature results output by the student network to obtain a second loss term.

[0042] The training module is used to train the image defect detection model based on the first loss term and the second loss term;

[0043] The image samples include: a first image sample comprising a defect-free image, and a second image sample obtained by adjusting the brightness and / or rotating the angle of the first image sample.

[0044] Fourthly, embodiments of this application provide an image defect detection device, the image defect detection device comprising:

[0045] The detection module is used to input the image to be detected into the image defect detection model to obtain an anomaly map;

[0046] The judgment module is used to determine whether the image to be detected has defects based on the anomaly map;

[0047] The image defect detection model is trained based on the training device described in the third aspect.

[0048] Fifthly, embodiments of this application provide a training device for an image defect detection model, wherein the image defect detection model is a reverse distillation model comprising a sequentially connected teacher network, a feature mapping layer, a single-class bottleneck embedding layer, and a student network, and the training device comprises: a processor and a memory for storing a computer program capable of running on the processor, wherein the processor, when running the computer program, performs the steps of the method described in the first aspect.

[0049] In a sixth aspect, embodiments of this application provide an image defect detection device, the image defect detection device comprising: a processor and a memory for storing a computer program capable of running on the processor, wherein the processor, when running the computer program, performs the steps of the method as described in the second aspect.

[0050] In a seventh aspect, embodiments of this application provide a storage medium storing a computer program, which, when executed by a processor, implements the steps of the methods described in the first and / or second aspects.

[0051] Eighthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the steps of the methods described in the first and / or second aspects.

[0052] The image defect detection model training method provided in this application embodiment is a reverse distillation model comprising a sequentially connected teacher network, a feature mapping layer, a single-class bottleneck embedding layer, and a student network. The training method includes: inputting image samples into the image defect detection model to obtain feature results output by the feature mapping layer and feature results output by the student network; performing feature constraints on the feature results output by the feature mapping layer to obtain a first loss term; and performing feature constraints on the feature results output by the feature mapping layer and the feature results output by the student network to obtain a second loss term; and training the image defect detection model based on the first loss term and the second loss term; wherein the image samples include: a first image sample comprising a defect-free image, and a second image sample obtained after brightness adjustment and / or angle deflection processing of the first image sample. Thus, in this embodiment of the application, a feature mapping layer is added to the image defect detection model to constrain the feature distribution of defect-free images with different brightness and / or different angles of deflection within the same feature space for model training. This allows the image defect detection model to ignore image changes caused by brightness and / or angle of deflection when used for image defect detection, thereby reducing the false alarm rate and improving the model's generalization ability and image defect detection accuracy in industrial scenarios. Attached Figure Description

[0053] Figure 1 This is a schematic diagram of the structure of the image defect detection model in an embodiment of this application;

[0054] Figure 2 This is a schematic diagram of the structure of an image defect detection model in one application example of this application;

[0055] Figure 3 This is a schematic diagram of the feature mapping layer structure in one application example of this application;

[0056] Figure 4 This is a schematic diagram of the structure of a multi-scale fusion layer in one application example of this application;

[0057] Figure 5 This is a schematic diagram of the structure of a convolutional block in a single-class embedding layer in an application example of this application;

[0058] Figure 6 This is a schematic diagram of the structure of an identity block in a single-class embedding layer in an application example of this application;

[0059] Figure 7 This is a schematic diagram of the structure of a student network in one application example of this application;

[0060] Figure 8 This is a schematic diagram of the deconvolution computation structure of a student network in an application example of this application.

[0061] Figure 9This is a schematic flowchart of the training method for the image defect detection model in an embodiment of this application;

[0062] Figure 10 This is a flowchart illustrating the training method of an image defect detection model in an application example of this application.

[0063] Figure 11 This is a schematic flowchart of the image defect detection method according to an embodiment of this application;

[0064] Figure 12 This is a flowchart illustrating an image defect detection method in one application example of this application;

[0065] Figure 13 This is a schematic diagram of the structure of the training device for the image defect detection model in an embodiment of this application;

[0066] Figure 14 This is a schematic diagram of the image defect detection device according to an embodiment of this application;

[0067] Figure 15 This is a schematic diagram of the structure of the training device for the image defect detection model according to an embodiment of this application;

[0068] Figure 16 This is a schematic diagram of the structure of the image defect detection device according to an embodiment of this application. Detailed Implementation

[0069] The present application will now be described in further detail with reference to the accompanying drawings and embodiments.

[0070] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.

[0071] This application provides a training method for an image defect detection model, aiming to improve the accuracy of image defect detection in industrial scenarios.

[0072] Before describing in detail the training method of the image defect detection model of the present application embodiments, the image defect detection model of the present application embodiments will be described first.

[0073] The image defect detection model in this application embodiment is as follows: Figure 1 As shown, this is a reverse distillation model, which includes a sequentially connected teacher network 1, a feature mapping layer 2, a single-class bottleneck embedding layer 3, and a student network 4.

[0074] Here, in the image defect detection model of this application embodiment, the teacher network 1, feature mapping layer 2, single-class bottleneck embedding layer 3 and student network 4 are connected sequentially. That is, after the image is input into the image defect detection model, the output of the teacher network 1 is used as the input of the feature mapping layer 2, the output of the feature mapping layer 2 is used as the input of the single-class bottleneck embedding layer 3, and the output of the single-class bottleneck embedding layer 3 is used as the input of the student network 4.

[0075] It's important to note that backdistillation methods based on one-class embeddings are commonly used in anomaly detection. In related technologies, backdistillation models typically consist of a sequentially connected teacher network, a one-class bottleneck embedding layer, and a student network. The teacher network, also known as the teacher encoder, encodes the image input to the backdistillation model. Each convolutional layer of the teacher network extracts multi-scale feature representations from the input image. The one-class bottleneck embedding (OCBE) layer compresses the multi-scale feature representations output by the teacher network into low-dimensional one-class feature embeddings, processing only one-class information and removing redundant information from the feature representations. The student network, also known as the student decoder, unlike knowledge distillation methods, does not directly input the image into the backdistillation model. Instead, it takes the one-class feature embeddings output by the one-class bottleneck embedding layer as input and uses deconvolution to decode and reconstruct the multi-scale feature representations output by the teacher network. After unsupervised learning on image samples, including defect-free images, the backdistillation model can be used for image defect detection. Specifically, for the defect-free parts of the input image, the student network of the backdistillation model can successfully decode and reconstruct the multi-scale feature representation output by the teacher network; for the defective parts of the input image, the teacher network can capture and extract the features corresponding to the defects, while the student network cannot decode and reconstruct the features corresponding to the defects. Therefore, based on the similarity between the feature results output by the teacher network and the student network, it can be determined whether there are defects in the input image.

[0076] It should be noted that while the backdistillation method can be used for surface defect detection in industrial products, in complex industrial scenarios, such as the inspection of metal products, changes in the light intensity on the metal product surface will alter the brightness of the captured image, affecting the overall feature distribution. This may cause the student network to fail to accurately decode and reconstruct the feature distribution changes caused by brightness, leading to low similarity between the feature outputs of the teacher and student networks and false positives indicating defects. Although the Convolutional Neural Networks (CNNs) in the backdistillation model possess some rotation invariance, excessively large deflection angles of the metal product in the captured image will also directly affect the decoding and reconstruction of the student network, resulting in false positives indicating defects. In other words, when the backdistillation method is applied to surface defect detection in industrial scenarios, the problem of false positives due to excessively large changes in the brightness and deflection angle of the captured image still exists.

[0077] It should be noted that, compared with the reverse distillation model for image defect detection in related technologies, the image defect detection model in this application embodiment adds a feature mapping layer (Project Layer). The feature results output by the teacher network are not directly input into the single-class bottleneck embedding layer, but are mapped by the feature mapping layer before being input into the single-class bottleneck embedding layer.

[0078] Here, the feature mapping layer, also known as the pooling layer, downsampling layer, or computation layer, is used to pool the feature results output by the teacher network, so that the feature distribution is constrained to the same feature space, reducing the number of weights and improving the image defect detection model's tolerance to distortion of the input image.

[0079] In one application example of this application, the image defect detection model is as follows: Figure 2 As shown in the figure. The teacher network includes convolutional layers conv2, conv3, and conv4 (not shown in the figure). After an image is input into the teacher network, each convolutional layer extracts features from the image, obtaining multi-scale feature results [f1, f2, f3]. Feature f1 is the output of convolutional layer conv2, feature f2 is the output of convolutional layer conv3, and feature f3 is the output of convolutional layer conv4. Figure 3As shown, the feature mapping layer consists of four convolutional layers (conv), a normalization layer (instanceNorm), and an activation layer (LeakyReLU). The feature results [f1, f2, f3] are input into the feature mapping layer and processed to obtain the feature results [tf1, tf2, tf3]. Here, feature tf1 corresponds to feature f1, feature tf2 corresponds to feature f2, and feature tf3 corresponds to feature f3. The single-class bottleneck embedding layer includes a multi-scale fusion layer and a single-class embedding layer. The multi-scale fusion layer is as follows: Figure 4 As shown, the embedding layer is composed of a convolutional layer (conv), a normalization layer (batchNorm), and an activation layer (ReLU). The feature results [tf1, tf2, tf3] are input into the single-class bottleneck embedding layer. Feature tf1 is combined twice to obtain feature mf1, and feature tf2 is combined once to obtain feature mf2. Features mf1, mf2, and tf3 are then stacked along the channel dimension, and the resulting feature matrix is ​​combined in this way to obtain the multi-scale fusion feature mcf. The single-class embedding layer consists of a convolutional block (convBlock) and two identity blocks (identityBlock). The structure of the convolutional block (convBlock) is as follows: Figure 5 As shown, the structure of the identity block is as follows: Figure 6 As shown, the multi-scale fusion feature mcf is calculated sequentially through the convolutional block convBlock and the identity block identityBlock of the single-class embedding layer to obtain the single-class feature embedding res-feature; as Figure 7 As shown, the student network decoder structure is paired with the teacher network encoder structure. However, unlike the teacher network encoder, the computation of each block in the student network decoder begins with deconvolution. Each block includes a deconvolution computation structure and multiple identity blocks, such as identityBlocks. Figure 8 As shown, after the single-class feature embedding res-feature is decoded by the student network decoder, the feature results [df3, df2, df1] are obtained. Among them, feature df3 is calculated by combining a deconvolution calculation structure and five identity blocks, feature df2 is calculated by combining a deconvolution calculation structure and three identity blocks, and feature df1 is calculated by combining a deconvolution calculation structure and one identity block.

[0080] Based on the aforementioned image defect detection model, the training method for the image defect detection model provided in this application embodiment is as follows: Figure 9 As shown, it includes:

[0081] Step 901: Input the image sample into the image defect detection model to obtain the feature results output by the feature mapping layer and the feature results output by the student network.

[0082] Here, the image sample includes: a first image sample including a defect-free image, and a second image sample obtained by performing brightness adjustment and / or angle deflection processing on the first image sample.

[0083] Here, the first image sample includes only defect-free images.

[0084] Here, the images included in the second image sample obtained after processing the first image sample can be understood as defect-free images after enhancement processing.

[0085] Specifically, the second image sample includes: one or more sets of defect-free images after different brightness adjustment processing of the defect-free images in the first image sample; and / or, one or more sets of defect-free images after different angle deflection processing of the defect-free images in the first image sample; and / or, one or more sets of defect-free images after different brightness adjustment and different angle deflection processing of the defect-free images in the first image sample.

[0086] Here, in this embodiment of the application, the first image sample can be processed based on a set brightness adjustment amount and / or a set angle deflection amount to obtain the second image sample, or the first image sample can be processed based on a random brightness adjustment amount and / or a random angle deflection amount to obtain the second image sample.

[0087] It is understandable that, compared to supervised learning based on image samples including defective images, the image defect detection model in this application uses defect-free images for unsupervised learning, which are easier to obtain and collect, thus reducing the difficulty and cost of model training.

[0088] In some embodiments, in order to improve the training effect of the image defect detection model, after acquiring a certain number of defect-free images, the acquired defect-free images are resized and normalized to obtain a first image sample. After the acquired defect-free images are resized and / or angle-shifted, they are resized and normalized to obtain a second image sample. That is, the images in the first image sample and the second image sample have the same resolution.

[0089] In one application example of this application, the first image sample includes an image of an apparent defect-free metal product.

[0090] For example, inputting image samples into an image defect detection model to obtain feature results output by the feature mapping layer and feature results output by the student network includes: inputting image samples into a pre-trained teacher network, which encodes the image samples and maps them through the feature mapping layer to obtain feature results output by the feature mapping layer; and then fusing the feature results output by the feature mapping layer through a single-class bottleneck embedding layer and decoding them through the student network to obtain feature results output by the student network.

[0091] It should be noted that the teacher network of the image defect detection model in this application embodiment is pre-trained, and the weights of the teacher network are frozen after pre-training. The training method in this application embodiment does not update the weights of the teacher network.

[0092] In some embodiments, the teacher network is a WideResNet50 wide residual network model pre-trained on the large-scale visual object recognition database ImageNet.

[0093] In some embodiments, the teacher network is a pre-trained WideResNet101 wide residual network model.

[0094] In some embodiments, the teacher network is a pre-trained encoder based on a neural network model, transformer.

[0095] For example, inputting an image sample into an image defect detection model to obtain feature results output by the feature mapping layer and feature results output by the student network includes: inputting a first image sample into the image defect detection model to obtain a first feature result output by the feature mapping layer and a second feature result output by the student network; and inputting a second image sample into the image defect detection model to obtain a third feature result output by the feature mapping layer.

[0096] It is understandable that the first feature result is the feature result output after the first image sample is processed sequentially by the teacher network and the feature mapping layer; the second feature result is the feature result output after the first feature result is processed sequentially by the single-class bottleneck embedding layer and the student network; and the third feature result is the feature result output after the second image sample is processed sequentially by the teacher network and the feature mapping layer.

[0097] It should be noted that the feature results output after the third feature result is processed by the single-class bottleneck embedding layer, as well as the feature results output after being processed sequentially by the single-class bottleneck embedding layer and the student network, are not required in the training method of this application embodiment.

[0098] Step 902: Apply feature constraints to the feature results output by the feature mapping layer to obtain the first loss term; and apply feature constraints to the feature results output by the feature mapping layer and the feature results output by the student network to obtain the second loss term.

[0099] For example, feature constraints are applied to the feature results output by the feature mapping layer to obtain a first loss term, including: applying feature constraints to the first feature result and the third feature result to obtain the first loss term.

[0100] For example, feature constraints are applied to the feature results output by the feature mapping layer and the feature results output by the student network to obtain a second loss term, including: applying feature constraints to the first feature result and the second feature result to obtain a second loss term.

[0101] Here, the feature mapping layer is used to constrain the features corresponding to different brightness and / or different angle deflections obtained after processing the first image sample and the second image sample into the same feature space, and calculate the corresponding first loss term. It is easy to understand that the image defect detection model is trained based on the first loss term, so that the trained image defect detection model ignores the image changes caused by changes in brightness and / or angle deflection as much as possible, focuses the anomaly detection on the image changes caused by defects, and reduces the false alarm rate.

[0102] Here, the image defect detection model is trained based on the second loss term, so that when the trained image defect detection model detects defect-free images, the feature results output by the feature mapping layer and the feature results output by the student network are highly similar; when detecting defective images, the feature results output by the feature mapping layer and the feature results output by the student network are lowly similar.

[0103] It is understood that the embodiments of this application add a feature mapping layer to the image defect detection model, constraining the feature distribution of defect-free images with different brightness and / or different angles of deflection within the same feature space for model training. This allows the trained image defect detection model to ignore image changes caused by brightness and / or angle deflection when performing image defect detection, thereby reducing the false alarm rate and improving the model's generalization ability and image defect detection accuracy in industrial scenarios.

[0104] Here, based on Figure 3 The image defect detection model shown is as follows: Figure 10As shown, after the first image sample is input into the image defect detection model, the feature mapping layer outputs the first feature result [tf1_1, tf2_1, tf3_1], and the student network outputs the second feature result [df1_1, df2_1, df3_1]; after the second image sample is input into the image defect detection model, the feature mapping layer outputs the third feature result [tf1_2, tf2_2, tf3_2]; the first feature result [tf1_1, tf2_1, tf3_1] is processed. The first loss term [lrid-l1, lrid-l2, lrid-l3] is obtained by subjecting the corresponding layer feature constraints on the first feature result [tf1_1, tf2_1, tf3_1] and the second feature result [df1_1, df2_1, df3_1]].

[0105] For example, feature constraints are applied to the first feature result and the third feature result to obtain a first loss term, including: calculating the cosine similarity between the first feature result and the third feature result to obtain the first loss term.

[0106] For example, feature constraints are applied to the first feature result and the second feature result to obtain the second loss term, including: calculating the cosine similarity between the first feature result and the third feature result to obtain the second loss term.

[0107] Here, cosine similarity loss is a loss function used to measure the similarity between two vectors, calculated based on cosine similarity. Figure 10 The first loss term, lrid_loss, can be obtained based on the following formula:

[0108] lrid_loss=mean(1-cos(tf1_1,tf1_2))+mean(1-cos(tf2_1,tf2_2))+mean(1-cos(tf3_1,tf3_2))

[0109] Here, based on cosine similarity calculation, Figure 10 The second loss term, st_loss, can be obtained based on the following formula:

[0110] st_loss=mean(1-cos(tf1_1,df1_1))+mean(1-cos(tf2_1,df2_1))+mean(1-cos(tf3_1,df3_1))

[0111] It should be noted that the aforementioned method for calculating the loss term by cosine similarity between the calculated feature results is only a preferred example. This application does not specifically limit the method for calculating the loss term. In some embodiments, the first loss term and the second loss term can also be calculated by other loss functions such as Euclidean distance.

[0112] Step 903: Train the image defect detection model based on the first loss term and the second loss term.

[0113] For example, training an image defect detection model based on a first loss term and a second loss term includes updating the weights of the feature mapping layer, the single-class bottleneck embedding layer, and the student network based on the first and second loss terms. Here, the overall loss term of the image defect detection model is the sum of the first and second loss terms. During model training, the overall loss term is minimized by continuously optimizing and updating the weights of the feature mapping layer, the single-class bottleneck embedding layer, and the student network, thus obtaining a well-trained image defect detection model.

[0114] In some embodiments, the weights of the feature map layer, the single-class bottleneck embedding layer, and the student network are updated based on the Adaptive Moment Estimation (Adam) algorithm.

[0115] Here, the weights of the teacher network are not updated based on the first and second loss terms.

[0116] This application also provides an image defect detection method, such as... Figure 11 As shown, the method includes:

[0117] Step 1101: Input the image to be detected into the image defect detection model to obtain an anomaly map.

[0118] The image defect detection model was trained using the aforementioned training method.

[0119] Step 1102: Based on the anomaly map, determine whether the image to be detected has defects.

[0120] For example, inputting the image to be detected into the image defect detection model to obtain an anomaly map includes: inputting the image to be detected into the image defect detection model to obtain the fourth feature result output by the feature mapping layer of the image defect detection model and the fifth feature result output by the student network of the image defect detection model; and calculating the anomaly map based on the fourth feature result and the fifth feature result.

[0121] Here, an anomaly map is a tool used to visualize and quantify anomalous regions in an image, highlighting areas in an image that contain defects.

[0122] It is understandable that if the image to be detected has defects, the features corresponding to the defects can be captured and extracted by the teacher network, and after being mapped and processed by the feature mapping layer, the corresponding fourth feature result is output. Since the training method of this application embodiment is based on the image defect detection model to train on the defect-free image, the features corresponding to the defects in the fourth feature result cannot be successfully decoded and reconstructed by the student network. Therefore, an anomaly map is calculated based on the fourth feature result and the fifth feature result. Through the anomaly map, the defect area in the detected image can be determined.

[0123] It is understood that, since the image defect detection model in this application embodiment adds a feature mapping layer and is jointly trained based on a first image sample including a defect-free image and a second image sample obtained after brightness adjustment and / or angle deflection processing of the first image sample, when performing image defect detection, the features corresponding to different brightness and / or different deflection angles can be successfully decoded and reconstructed by the student network. The calculated anomaly map will ignore the image changes caused by changes in brightness and / or deflection angle, and only highlight the areas with defects.

[0124] For example, based on the fourth feature result and the fifth feature result, an anomaly map is calculated: the distance matrix between each layer between the fourth feature result and the fifth feature result is calculated; the distance matrix of each layer is interpolated, and the interpolated distance matrices of the same size are added together to generate an anomaly score matrix; the anomaly score matrix is ​​denoised to obtain the anomaly map.

[0125] Here, based on Figure 3 The image defect detection model shown is as follows: Figure 12 As shown, after the image to be detected is input into the image defect detection model, the feature results [f1_3, f2_3, f3_3] are extracted by the teacher network encoder. Here, feature f1_3 is calculated based on convolutional layer conv2, feature f2_3 is calculated based on convolutional layer conv3, and feature f3_3 is calculated based on convolutional layer conv4. After the feature results [f1_3, f2_3, f3_3] are mapped by the feature mapping layer, the fourth feature result [tf1_3, tf2_3, tf3_3] is output. After the fourth feature result [tf1_3, tf2_3, tf3_3] is fused by the single-class bottleneck embedding layer, it is decoded by the student network decoder to output the fifth feature result [df1_3, df2_3, df3_3].

[0126] Further, based on the output fourth feature results [tf1_3, tf2_3, tf3_3] and fifth feature results [df1_3, df2_3, df3_3], the distance matrix of each layer is calculated. In some embodiments, calculating the distance matrix of each layer between the fourth feature results and the fifth feature results includes: calculating the cosine similarity between the fourth feature results and the fifth feature results of each layer to obtain the distance matrix of each layer.

[0127] Specifically, the distance matrix dm between each layer is as follows:

[0128] dm1 = 1 - cos(tf1_3, df1_3)

[0129] dm2 = 1 - cos(tf2_3, df2_3)

[0130] dm3 = 1 - cos(tf3_3, df3_3)

[0131] Furthermore, interpolation is used to unify the distance matrices dm of different layers to the same size, and all the obtained distance matrices dm are summed to obtain the anomaly score matrix score, which is as follows:

[0132] score=interpolate(dm3)+interpolate(dm2)+dm1

[0133] Furthermore, to eliminate the influence of noise, the calculated anomaly score matrix is ​​denoised as follows: Gaussian smoothing can be used to denoise the anomaly score matrix. This yields the anomaly map.

[0134] To implement the method of this application embodiment, this application embodiment also provides a training device for an image defect detection model. This training device corresponds to the above-described training method, and each step in the above-described training method embodiment is also fully applicable to this device embodiment.

[0135] like Figure 13As shown in the illustration, this application provides a training device for an image defect detection model. The image defect detection model is a reverse distillation model comprising a teacher network, a feature mapping layer, a single-class bottleneck embedding layer, and a student network connected in sequence. The training device includes a first processing module 1301, a second processing module 1302, and a training module 1303. The first processing module 1301 is used to input image samples into the image defect detection model to obtain feature results output by the feature mapping layer and feature results output by the student network. The second processing module 1302 is used to perform feature constraints on the feature results output by the feature mapping layer to obtain a first loss term, and to perform feature constraints on the feature results output by the feature mapping layer and the feature results output by the student network to obtain a second loss term. The training module 1303 is used to train the image defect detection model based on the first loss term and the second loss term. The image samples include a first image sample comprising a defect-free image, and a second image sample obtained after brightness adjustment and / or angle deflection processing of the first image sample.

[0136] In some embodiments, the first processing module 1301 is specifically used to: input image samples into the pre-trained teacher network, and after encoding by the teacher network and mapping by the feature mapping layer, obtain the feature result output by the feature mapping layer; after the feature result output by the feature mapping layer is fused by the single-class bottleneck embedding layer and decoded by the student network, obtain the feature result output by the student network.

[0137] In some embodiments, the first processing module 1301 is specifically used to: input a first image sample into an image defect detection model to obtain a first feature result output by the feature mapping layer and a second feature result output by the student network; and input a second image sample into the image defect detection model to obtain a third feature result output by the feature mapping layer.

[0138] In some embodiments, the second processing module 1302 is specifically used to: perform feature constraints on the first feature result and the third feature result to obtain a first loss term.

[0139] In some embodiments, the second processing module 1302 is specifically used to: perform feature constraints on the first feature result and the third feature result to obtain a first loss term.

[0140] In some embodiments, the second processing module 1302 is specifically used to: calculate the cosine similarity between the first feature result and the third feature result to obtain a first loss term.

[0141] In some embodiments, the second processing module 1302 is specifically used to: calculate the cosine similarity between the first feature result and the third feature result to obtain a second loss term.

[0142] In some embodiments, the training module 1303 is specifically used to: update the weights of the feature mapping layer, the single-class bottleneck embedding layer, and the student network based on the first loss term and the second loss term.

[0143] It should be noted that the training device provided in the above embodiments is only illustrated by the division of the above program modules. In practical applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device can be divided into different program modules to complete all or part of the processing described above. In addition, the training device and training method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0144] To implement the method of the embodiments of this application, the embodiments of this application also provide an image defect detection device. This training device corresponds to the above-described image defect detection method, and each step in the above-described image defect detection method embodiments is also fully applicable to the embodiments of this device.

[0145] like Figure 14 As shown in the illustration, this application also provides an image defect detection device, which includes a detection module 1401 and a judgment module 1402. The detection module 1401 is used to input the image to be detected into an image defect detection model to obtain an anomaly map. The judgment module is used to determine whether the image to be detected has defects based on the anomaly map. The image defect detection model is trained using the aforementioned image defect detection model training device.

[0146] In some embodiments, the detection module 1401 is specifically used to: input the image to be detected into the image defect detection model to obtain the fourth feature result output by the feature mapping layer of the image defect detection model and the fifth feature result output by the student network of the image defect detection model; calculate the distance matrix between each layer between the fourth feature result and the fifth feature result; perform interpolation processing on the distance matrix of each layer, and add the interpolated distance matrices of the same size to generate an anomaly score matrix; and perform noise reduction processing on the anomaly score matrix to obtain an anomaly map.

[0147] In some embodiments, the detection module 1401 is specifically used to: calculate the cosine similarity between the fourth feature result and the fifth feature result of each layer to obtain the distance matrix of each layer.

[0148] It should be noted that the image defect detection device provided in the above embodiments is only illustrated by the division of the above program modules during training. In practical applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device can be divided into different program modules to complete all or part of the processing described above. In addition, the image defect detection device and the image defect detection method embodiments provided in the above embodiments belong to the same concept, and their specific implementation process can be found in the method embodiments, which will not be repeated here.

[0149] Based on the hardware implementation of the above program modules, and in order to implement the training method of the embodiments of this application, the embodiments of this application also provide a training device for an image defect detection model, such as... Figure 15 As shown, the training device 1500 includes at least one processor 1501, a memory 1502, a user interface 1503, and at least one network interface 1504. The various components in the server 1500 are coupled together via a bus system 1505. It can be understood that the bus system 1505 is used to implement communication between these components. In addition to a data bus, the bus system 1505 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in… Figure 15 The general labeled all buses as Bus System 1505.

[0150] The user interface 1503 may include a monitor, keyboard, mouse, trackball, click wheel, buttons, touchpad, or touch screen.

[0151] The memory 1502 in this embodiment is used to store various types of data to support the operation of the training device 1500. Examples of such data include any computer program used to operate on the training device 1500.

[0152] The training method for virtual machines disclosed in this application can be applied to or implemented by processor 1501. Processor 1501 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the training method can be completed by integrated logic circuits in the hardware of processor 1501 or by instructions in software form. The processor 1501 can be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 1501 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of this application can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software modules can be located in a storage medium, specifically memory 1502. Processor 1501 reads information from memory 1502 and, in conjunction with its hardware, completes the steps of the training method for virtual machines provided in the embodiments of this application.

[0153] In an exemplary embodiment, the training device 1500 may be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), FPGAs, general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to perform the aforementioned training method.

[0154] It is understood that memory 1502 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), EEPROM, ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), Sync Link Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM). The memory 1501 described in this application embodiment is intended to include, but is not limited to, these and any other suitable types of memory.

[0155] Based on the hardware implementation of the above program modules, and in order to implement the image defect detection method of this application embodiment, this application embodiment also provides an image defect detection device, such as... Figure 16 As shown, the image defect detection device 1600 includes at least one processor 1601, a memory 1602, a user interface 1603, and at least one network interface 1604. The various components in the image defect detection device 1600 are coupled together via a bus system 1605. It can be understood that the bus system 1605 is used to implement communication between these components. In addition to a data bus, the bus system 1605 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in… Figure 16 The general labeled all buses as Bus System 1605.

[0156] The user interface 1603 may include a monitor, keyboard, mouse, trackball, click wheel, buttons, touchpad, or touch screen.

[0157] The memory 1602 in this embodiment is used to store various types of data to support the operation of the image defect detection device 1600. Examples of such data include any computer program used to operate on the image defect detection device 1600.

[0158] The image defect detection method for an image defect detection device disclosed in this application can be applied to or implemented by the processor 1601. The processor 1601 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the image defect detection method can be completed by the integrated logic circuitry in the hardware of the processor 1601 or by instructions in software form. The processor 1601 can be a general-purpose processor, a DSP, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 1601 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the embodiments of this application can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software modules can be located in a storage medium, specifically memory 1602. The processor 1601 reads information from memory 1602 and, in conjunction with its hardware, completes the steps of the image defect detection method for an image defect detection device provided in the embodiments of this application.

[0159] In an exemplary embodiment, the image defect detection device 1600 may be implemented by one or more ASICs, DSPs, PLDs, CPLDs, FPGAs, general-purpose processors, controllers, MCUs, microprocessors, or other electronic components to perform the aforementioned image defect detection method.

[0160] It is understood that memory 1602 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be ROM, PROM, EPROM, EEPROM, FRAM, Flash Memory, magnetic surface memory, optical disk, or read-only optical disk; magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be RAM, which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as SRAM, SSRAM, DRAM, SDRAM, DDRSDRAM, ESDRAM, SLDRAM, and DRRAM. The memory 1602 described in the embodiments of this application is intended to include, but is not limited to, these and any other suitable types of memory.

[0161] In an exemplary embodiment, this application also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, such as a memory 1502 including a computer program that can be executed by the processor 1501 of the training device 1500 to complete the steps described in the training method of the image defect detection model of this application embodiment; and a memory 1602 including a computer program that can be executed by the processor 1601 of the image defect detection device 1600 to complete the steps described in the image defect detection method of this application embodiment. The computer-readable storage medium may be a ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disc, or CD-ROM, etc.

[0162] In some embodiments, the image defect detection device 1600 is used to detect apparent defects in metal products.

[0163] In an exemplary embodiment, this application also provides a computer program product, including a computer program that can be executed by a processor 1501 of a training device 1500 and by a processor 1601 of an image defect detection device 1600, to perform the steps described in the method of this application embodiment.

[0164] It should be noted that terms such as "first" and "second" are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0165] Furthermore, the technical solutions described in the embodiments of this application can be combined arbitrarily without conflict.

[0166] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A training method for an image defect detection model, characterized in that, The image defect detection model is a dedistillation model comprising a sequentially connected teacher network, a feature mapping layer, a single-class bottleneck embedding layer, and a student network. The training method includes: The image sample is input into the image defect detection model to obtain the feature results output by the feature mapping layer and the feature results output by the student network. The feature results output by the feature mapping layer are subjected to feature constraints to obtain a first loss term, and the feature results output by the feature mapping layer and the feature results output by the student network are subjected to feature constraints to obtain a second loss term. The image defect detection model is trained based on the first loss term and the second loss term; The image samples include: a first image sample comprising a defect-free image, and a second image sample obtained by adjusting the brightness and / or rotating the angle of the first image sample.

2. The training method according to claim 1, characterized in that, The step of inputting image samples into the image defect detection model to obtain the feature results output by the feature mapping layer and the feature results output by the student network includes: The image sample is input into the pre-trained teacher network, and after being encoded by the teacher network and mapped by the feature mapping layer, the feature result output by the feature mapping layer is obtained. The feature results output by the feature mapping layer are fused by the single-class bottleneck embedding layer and then decoded by the student network to obtain the feature results output by the student network.

3. The training method according to claim 1, characterized in that, The step of inputting image samples into the image defect detection model to obtain the feature results output by the feature mapping layer and the feature results output by the student network includes: The first image sample is input into the image defect detection model to obtain a first feature result output by the feature mapping layer and a second feature result output by the student network; and... The second image sample is input into the image defect detection model to obtain the third feature result output by the feature mapping layer.

4. The training method according to claim 3, characterized in that, The step of performing feature constraints on the feature results output by the feature mapping layer to obtain a first loss term includes: The first feature result and the third feature result are subjected to feature constraints to obtain the first loss term; The second loss term is obtained by subjecting feature constraints to the feature results output by the feature mapping layer and the feature results output by the student network, including: The first feature result and the second feature result are subjected to feature constraints to obtain the second loss term.

5. The training method according to claim 4, characterized in that, The step of performing feature constraints on the first feature result and the third feature result to obtain the first loss term includes: Calculate the cosine similarity between the first feature result and the third feature result to obtain the first loss term; The step of performing feature constraints on the first feature result and the second feature result to obtain the second loss term includes: The cosine similarity between the first feature result and the third feature result is calculated to obtain the second loss term.

6. The training method according to claim 4 or 5, characterized in that, The step of training the image defect detection model based on the first loss term and the second loss term includes: Based on the first loss term and the second loss term, update the weights of the feature mapping layer, the single-class bottleneck embedding layer, and the student network.

7. An image defect detection method, characterized in that, The method includes: The image to be detected is input into the image defect detection model to obtain an anomaly map; Based on the anomaly map, it is determined whether the image to be detected has defects; The image defect detection model is trained based on the method described in any one of claims 1 to 6.

8. The image defect detection method according to claim 7, characterized in that, The step of inputting the image to be detected into the image defect detection model to obtain an anomaly map includes: The image to be detected is input into the image defect detection model to obtain the fourth feature result output by the feature mapping layer of the image defect detection model and the fifth feature result output by the student network of the image defect detection model. Calculate the distance matrix between each layer of the fourth feature result and the fifth feature result; Interpolate the distance matrices of each layer, and add the interpolated distance matrices of the same size to generate an anomaly score matrix. The anomaly score matrix is ​​denoised to obtain an anomaly map.

9. The image defect detection method according to claim 8, characterized in that, The calculation of the distance matrix between each layer of the fourth feature result and the fifth feature result includes: Calculate the cosine similarity between the fourth feature result and the fifth feature result of each layer to obtain the distance matrix of each layer.

10. A training device for an image defect detection model, characterized in that, The image defect detection model is a dedistillation model comprising a sequentially connected teacher network, a feature mapping layer, a single-class bottleneck embedding layer, and a student network. The training device includes: The first processing module is used to input image samples into the image defect detection model and obtain the feature results output by the feature mapping layer and the feature results output by the student network. The second processing module is used to perform feature constraints on the feature results output by the feature mapping layer to obtain a first loss term, and to perform feature constraints on the feature results output by the feature mapping layer and the feature results output by the student network to obtain a second loss term. The training module is used to train the image defect detection model based on the first loss term and the second loss term; The image samples include: a first image sample comprising a defect-free image, and a second image sample obtained by adjusting the brightness and / or rotating the angle of the first image sample.

11. An image defect detection device, characterized in that, The image missing item detection device includes: The detection module is used to input the image to be detected into the image defect detection model to obtain an anomaly map; The judgment module is used to determine whether the image to be detected has defects based on the anomaly map; The image defect detection model is trained based on the training device described in claim 10.

12. A training device for an image defect detection model, characterized in that, The image defect detection model is a reverse distillation model comprising a sequentially connected teacher network, a feature mapping layer, a single-class bottleneck embedding layer, and a student network. The training device comprises a processor and a memory for storing a computer program that can run on the processor, wherein the processor, when running the computer program, performs the steps of the method according to any one of claims 1 to 6.

13. An image defect detection device, characterized in that, The image defect detection device includes: a processor and a memory for storing a computer program capable of running on the processor, wherein the processor, when running the computer program, performs the steps of the method according to any one of claims 7 to 9.

14. A storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 9.

15. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 9.

Images