Defect detection method, electronic device, and storage medium

Abstract

The application provides a flaw detection method, an electronic device and a storage medium. The method comprises the following steps: processing a no-flaw sample training image based on a flaw detection model to obtain a first reconstructed image, and calculating a first reconstruction error by using a first preset error function; training a discriminator based on the no-flaw sample training image and the first reconstructed image, and calculating an adversarial learning error by using a second preset error function; calculating a sample error according to the first reconstruction error and the adversarial learning error, and determining an error threshold; obtaining a test sample image and calculating a second reconstruction error; and determining a detection result of the test sample image according to the second reconstruction error and the error threshold. The application can detect subtle flaws in an image, thereby improving the accuracy of flaw detection on products.

Description

Technical Field

[0001] This application relates to the field of image detection technology, and in particular to a defect detection method, electronic device, and storage medium. Background Technology

[0002] In the field of industrial inspection, to improve the quality of industrial products, defect detection is usually performed on the products before packaging. However, due to limitations in the network architecture and loss function design of existing defect detection methods using autoencoders, the output image reconstructed from the input image is often blurry and cannot accurately match the original image. Figure 1 The image may appear unclear, but significant errors can occur even at the edges of objects within the image. The accumulation of these errors makes it difficult to detect subtle imperfections. Furthermore, the output image reconstructed by the autoencoder will have significant errors at object edges compared to the original image. Directly calculating the error between the original and reconstructed images can only detect obvious flaws, and is ineffective at detecting smaller and more subtle imperfections. Summary of the Invention

[0003] In view of the above, it is necessary to provide a defect detection method, apparatus and related equipment that can detect minute defects in images, thereby improving the accuracy of defect detection.

[0004] A first aspect of this application provides a defect detection method, the method comprising: acquiring a flawless sample training image; inputting the flawless sample training image into an autoencoder of a defect detection model, and calculating a first latent feature of the flawless sample training image through the encoding layer of the autoencoder; inputting the first latent feature into the decoding layer of the autoencoder, and calculating a first reconstructed image of the flawless sample training image, and calculating a first reconstruction error between the flawless sample training image and the first reconstructed image using a first preset error function; inputting the flawless sample training image and the first reconstructed image into a discriminator of the defect detection model, training the discriminator, determining that the autoencoder training is complete when the discriminator training is complete, and calculating the training completion using a second preset error function. The discriminator and the trained autoencoder are compared to the adversarial learning error; a sample error is calculated based on the first reconstruction error and the adversarial learning error; it is determined whether the sample error meets a first preset condition; if the first preset condition is met, an error threshold is determined based on the sample error; a test sample image is acquired, the test sample image is input into the encoding layer of the trained autoencoder, and a second latent feature of the test sample image is calculated; the second latent feature is input into the decoding layer of the trained autoencoder, and a second reconstructed image of the test sample image is calculated; a second reconstruction error between the test sample image and the second reconstructed image is calculated using a first preset error function; and the detection result of the test sample image is determined based on the second reconstruction error and the error threshold.

[0005] In one optional implementation, the step of inputting the flawless sample training image into the autoencoder of the flaw detection model and calculating the first latent feature of the flawless sample training image through the encoding layer of the autoencoder includes: vectorizing the flawless sample training image to obtain the feature vector of the flawless sample training image; and using the encoding layer of the autoencoder to perform operations on the feature vector of the flawless sample training image to obtain the first latent feature.

[0006] In one optional implementation, the step of using the encoding layer of the autoencoder to operate on the feature vector of the flawless sample training image to obtain the first latent feature includes: extracting the hidden layer corresponding to the encoding layer of the autoencoder; obtaining the weight matrix and bias value of the hidden layer; multiplying the first feature vector with the weight matrix to obtain the operation result; and adding the operation result with the bias value to obtain the first latent feature.

[0007] In one optional implementation, the step of inputting the first latent feature into the decoding layer of the autoencoder and calculating the first reconstructed image of the flawless sample training image includes: performing operations on the first latent feature using the decoding layer of the autoencoder; and performing restoration processing on the vector obtained after the operation to obtain the first reconstructed image.

[0008] In one optional implementation, the step of inputting the flawless training image and the first reconstructed image into the discriminator of the flaw detection model to train the discriminator, and determining that the autoencoder training is complete when the discriminator training is complete, includes: labeling the flawless training image as real and the first reconstructed image as fake, training the discriminator to distinguish between real and fake images; when the discriminator cannot distinguish between real and fake images, determining that the discriminator training is complete, and determining that the autoencoder training is complete.

[0009] In one optional implementation, calculating the sample error based on the first reconstruction error and the adversarial learning error includes: performing a weighted sum operation on the first reconstruction error and the adversarial learning error to obtain the sample error.

[0010] In one optional implementation, determining the error threshold based on the sample error includes: sorting the sample errors in ascending order to obtain a total list of sample errors and a serial number for each sample error; and selecting a sample error from the total list of sample errors as the error threshold according to a second preset condition.

[0011] In one optional implementation, determining the detection result of the test sample image based on the second reconstruction error and the error threshold includes: determining that the test sample image has defects when the second reconstruction error is greater than or equal to the error threshold; or determining that the test sample image has no defects when the second reconstruction error is less than the error threshold.

[0012] A second aspect of this application provides an electronic device, the electronic device comprising:

[0013] A memory that stores at least one instruction; and a processor that executes the instructions stored in the memory to implement the defect detection method.

[0014] A third aspect of this application provides a computer-readable storage medium storing at least one instruction that, when executed by a processor in an electronic device, implements the defect detection method described above.

[0015] This application trains an autoencoder so that the trained autoencoder can detect subtle defects in images, thereby improving the accuracy of defect detection for products. Attached Figure Description

[0016] Figure 1 This is a flowchart of a defect detection method provided in an embodiment of this application.

[0017] Figure 2 This is a schematic diagram of an electronic device structure provided in an embodiment of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0019] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein in the specification of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.

[0020] See Figure 1 The flowchart shown is a defect detection method provided in the application embodiment, which specifically includes the following:

[0021] 101, Obtain flawless sample training images.

[0022] In at least one embodiment of this application, the flawless samples can be selected from different types of flawless products (such as computer cases, mobile phone cases, etc.), and images of the flawless products can be obtained as training images for the flawless samples. For example, images of computer cases without dents, scratches, paint peeling, etc., can be used as training images for flawless samples. This application does not specifically limit the number of training images obtained.

[0023] 102. The flawless sample training image is input into the autoencoder of the flaw detection model, and the first latent feature of the flawless sample training image is calculated through the encoding layer of the autoencoder.

[0024] It's important to note that an autoencoder is a type of neural network that uses backpropagation to make the output equal to the input. Typically, an autoencoder consists of an encoder and a decoder. The encoder compresses the input into a latent space representation, and the decoder reconstructs this latent space representation into the output.

[0025] In at least one embodiment of this application, the encoder in the autoencoder (AE) includes multiple hidden layers, and the encoding layer is at least one corresponding hidden layer. The number of the multiple hidden layers can be arbitrarily set according to the application scenario.

[0026] In at least one embodiment of this application, calculating the first latent feature of the flawless sample training image through the encoding layer of the autoencoder includes:

[0027] The flawless sample training image is vectorized to obtain the feature vector of the flawless sample training image;

[0028] Extract the hidden layer corresponding to the coding layer;

[0029] The first latent feature is obtained by performing operations on the feature vector using the hidden layer.

[0030] Specifically, the step of using the hidden layer to operate on the feature vector to obtain the first latent feature includes:

[0031] Obtain the weight matrix and bias values ​​of the hidden layer;

[0032] The feature vector is multiplied by the weight matrix to obtain the result.

[0033] The first latent feature is obtained by adding the calculation result to the bias value.

[0034] 103. Input the first latent feature into the decoding layer of the autoencoder, calculate the first reconstructed image of the flawless sample training image, and use the first preset error function to calculate the first reconstruction error between the flawless sample training image and the first reconstructed image.

[0035] In at least one embodiment of this application, the step of inputting the first latent feature into the decoding layer of the autoencoder in the defect detection model and calculating the first reconstructed image of the defect-free sample training image includes:

[0036] The first latent feature is processed by the decoding layer in the autoencoder, and the resulting vector is restored to obtain the first reconstructed image.

[0037] In at least one embodiment of this application, the decoding layer in the autoencoder can also map low-dimensional data into high-dimensional data, increase the amount of data, and restore the original appearance of the compressed data, that is, restore the vector obtained by the first latent feature operation.

[0038] In at least one embodiment of this application, calculating the first reconstruction error between the flawless sample training image and the first reconstructed image using a first preset error function includes: the first preset error function is: This error function is existing technology, and the meaning of each symbol will not be repeated here.

[0039] 104. Input the flawless sample training image and the first reconstructed image into the discriminator of the flaw detection model to train the discriminator. When the discriminator training is completed, determine that the autoencoder training is completed. Calculate the adversarial learning error between the trained discriminator and the trained autoencoder using a second preset error function.

[0040] In at least one embodiment of this application, the flawless sample training image and the first reconstructed image are input into the discriminator of the first flaw detection model, and training the discriminator includes:

[0041] The flawless training sample image is labeled as real and the first reconstructed image is labeled as fake. The discriminator is trained so that it can distinguish between real and fake images.

[0042] When flawless training sample images are input into the autoencoder, the autoencoder learns the features of the flawless training sample images and generates a corresponding first reconstructed image. The discriminator is responsible for distinguishing whether the input data is a flawless training sample image or a first reconstructed image and feeding this information back to the autoencoder. The two networks are trained alternately, and their capabilities improve synchronously until the first reconstructed image generated by the autoencoder is indistinguishable from the real image and reaches a certain balance with the discriminator's capabilities.

[0043] When the discriminator can no longer distinguish between real and fake images, that is, when the capabilities of the autoencoder and the discriminator reach a certain balance, it is determined that the discriminator training is complete, and the autoencoder training is complete. The adversarial learning error is then calculated using a second preset error function. The second preset error function is: This error function is existing technology, and the meaning of each symbol will not be repeated here.

[0044] 105. Calculate the sample error based on the first reconstruction error and the adversarial learning error, and determine whether the sample error meets the first preset condition.

[0045] In at least one embodiment of this application, calculating the sample error based on the first reconstruction error and the adversarial learning error includes:

[0046] The sample error is obtained by weighting and summing the adversarial learning error and the first reconstruction error.

[0047] For example, if the adversarial learning error is 0.5 and the first reconstruction error is 0.04, calculate the weighted sum of 0.5 and 0.04. When the adversarial learning error accounts for 10% of the sample error and the first reconstruction error accounts for 90% of the sample error, the calculated sample error is: 0.5*10%+0.04*90%=0.086.

[0048] In at least one embodiment of this application, when it is determined that the discriminator training is complete, the autoencoder training is also determined to be complete, that is, the sample error is determined to meet a first preset condition, and all sample errors are collected. If the sample error does not meet the first preset condition, the discriminator training continues.

[0049] 106. Determine the error threshold based on the sample error.

[0050] In at least one embodiment of this application, determining the error threshold based on the sample error includes:

[0051] Collect all the sample errors, sort the sample errors in ascending order, and obtain a total list of the sample errors and the serial number of each sample error;

[0052] Select a sample error from the total list of sample errors according to the second preset condition as the error threshold.

[0053] Specifically, selecting a sample error from the total list of sample errors as the error threshold according to the second preset condition includes:

[0054] Calculate the number of sample errors and multiply the number by the configuration value to obtain the target value;

[0055] The sample error whose sample number is equal to the target value is selected from the total list and used as the error threshold.

[0056] 107. Obtain a test sample image, input the test sample image into the encoding layer of the trained autoencoder, calculate the second latent feature of the test sample image, input the second latent feature into the decoding layer of the trained autoencoder, calculate the second reconstructed image of the test sample image, and use a first preset error function to calculate the second reconstruction error between the test sample image and the second reconstructed image.

[0057] 108. Determine the detection result of the test sample image based on the second reconstruction error and the error threshold.

[0058] In at least one embodiment of this application, the detection result includes whether the test sample image has defects or whether the test sample image has no defects.

[0059] In at least one embodiment of this application, determining the detection result of the image to be detected based on the second reconstruction error and the error threshold includes:

[0060] When the second reconstruction error is greater than or equal to the error threshold, it is determined that the test sample image has defects; or

[0061] When the second reconstruction error is less than the error threshold, the test sample image is determined to be flawless.

[0062] By comparing the second reconstruction error with the error threshold, since the comparison is numerical, subtle defects in the test sample image can be detected, thereby improving the accuracy of defect detection.

[0063] like Figure 2 The diagram shown is a schematic representation of an electronic device structure provided in an embodiment of this application. Specifically, it includes the following:

[0064] In one embodiment of this application, the electronic device 2 includes, but is not limited to, a memory 201, a processor 202, and a computer program stored in the memory 201 and executable on the processor 202.

[0065] Those skilled in the art will understand that the schematic diagram is merely an example of the electronic device 2 and does not constitute a limitation on the electronic device 2. It may include more or fewer components than shown in the diagram, or combine certain components, or different components. For example, the electronic device 2 may also include input / output devices, network access devices, buses, etc.

[0066] The processor 202 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processor 202 is the computing core and control center of the electronic device 2, connecting various parts of the electronic device 2 through various interfaces and lines, and obtaining the operating system of the electronic device 2, as well as various installed applications and program code.

[0067] The memory 201 can be used to store the computer programs and / or modules. The processor 202 implements various functions of the electronic device 2 by running or retrieving the computer programs and / or modules stored in the memory 201 and by calling the data stored in the memory 201. The memory 201 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 201 may include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other non-volatile solid-state storage device.

[0068] The memory 201 can be the external memory and / or internal memory of the electronic device 2. Furthermore, the memory 201 can be a physical memory, such as a memory module, a TF card (Trans-flash Card), etc.

[0069] If the modules / units integrated in the electronic device 2 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when the computer program is acquired by a processor, it can implement the steps of the various method embodiments described above.

[0070] The computer program includes computer program code, which may be in the form of source code, object code, accessible file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, and read-only memory (ROM).

[0071] Combination Figure 1 The memory 201 in the electronic device 2 stores multiple instructions to implement a defect detection method, and the processor 202 can acquire the multiple instructions to implement:

[0072] Obtain training images of flawless samples;

[0073] The flawless sample training image is input into the autoencoder of the flaw detection model, and the first latent feature of the flawless sample training image is calculated through the encoding layer of the autoencoder.

[0074] The first latent feature is input into the decoding layer of the autoencoder, and the first reconstructed image of the flawless sample training image is calculated. The first reconstruction error between the flawless sample training image and the first reconstructed image is calculated using a first preset error function.

[0075] The flawless sample training image and the first reconstructed image are input into the discriminator of the flaw detection model to train the discriminator. When the discriminator training is completed, the autoencoder training is determined to be completed. The adversarial learning error between the trained discriminator and the trained autoencoder is calculated using a second preset error function.

[0076] The sample error is calculated based on the first reconstruction error and the adversarial learning error;

[0077] Determine whether the sample error meets the first preset condition;

[0078] If the first preset condition is met, an error threshold is determined based on the sample error;

[0079] A test sample image is acquired, and the test sample image is input into the encoding layer of the trained autoencoder. The second latent feature of the test sample image is calculated, and the second latent feature is input into the decoding layer of the trained autoencoder. The second reconstructed image of the test sample image is calculated, and the second reconstruction error between the test sample image and the second reconstructed image is calculated using a first preset error function.

[0080] Specifically, the processor 202's implementation method for the above instructions can be found in [reference needed]. Figure 1 The descriptions of the relevant steps in the corresponding embodiments are not repeated here.

[0081] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0082] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0083] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0084] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of this application is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within this application. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0085] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. In this application, the terms "first," "second," etc., are used to indicate names and do not indicate any particular order.

[0086] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application and are not intended to limit it. Although this application has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of this application without departing from the spirit and scope of the technical solutions of this application.

Claims

1. A defect detection method, characterized in that, The defect detection method includes: Obtain training images of flawless samples; The flawless sample training image is input into the autoencoder of the flaw detection model, and the first latent feature of the flawless sample training image is calculated through the encoding layer of the autoencoder. The first latent feature is input into the decoding layer of the autoencoder, and the first reconstructed image of the flawless sample training image is calculated. The first reconstruction error between the flawless sample training image and the first reconstructed image is calculated using a first preset error function. The flawless sample training image and the first reconstructed image are input into the discriminator of the flaw detection model to train the discriminator. When the discriminator training is completed, the autoencoder training is determined to be completed. The adversarial learning error between the trained discriminator and the trained autoencoder is calculated using a second preset error function. The sample error is calculated based on the first reconstruction error and the adversarial learning error; Determine whether the sample error meets the first preset condition; If the first preset condition is met, an error threshold is determined based on the sample error; A test sample image is acquired, and the test sample image is input into the encoding layer of the trained autoencoder. The second latent feature of the test sample image is calculated, and the second latent feature is input into the decoding layer of the trained autoencoder. The second reconstructed image of the test sample image is calculated, and the second reconstruction error between the test sample image and the second reconstructed image is calculated using a first preset error function. The detection result of the test sample image is determined based on the second reconstruction error and the error threshold.

2. The defect detection method according to claim 1, characterized in that, The step of inputting the flawless sample training image into the autoencoder of the flaw detection model, and calculating the first latent feature of the flawless sample training image through the encoding layer of the autoencoder, includes: The flawless sample training image is vectorized to obtain the feature vector of the flawless sample training image; The first latent feature is obtained by using the encoding layer of the autoencoder to operate on the feature vector of the flawless sample training image.

3. The defect detection method according to claim 2, characterized in that, The step of using the encoding layer of the autoencoder to process the feature vector of the flawless sample training image to obtain the first latent feature includes: Extract the hidden layer corresponding to the coding layer of the autoencoder; Obtain the weight matrix and bias values ​​of the hidden layer; The feature vector is multiplied by the weight matrix to obtain the result. The first latent feature is obtained by adding the calculation result to the bias value.

4. The defect detection method according to claim 1, characterized in that, The step of inputting the first latent feature into the decoding layer of the autoencoder and calculating the first reconstructed image of the flawless sample training image includes: The first latent feature is processed using the decoding layer of the autoencoder; The vector obtained after the operation is restored to obtain the first reconstructed image.

5. The defect detection method according to claim 1, characterized in that, The step of inputting the flawless sample training image and the first reconstructed image into the discriminator of the flaw detection model to train the discriminator, and determining that the autoencoder training is complete when the discriminator training is complete, includes: The flawless training sample image is labeled as real, and the first reconstructed image is labeled as fake. The discriminator is trained to distinguish between real and fake images. When the discriminator cannot distinguish between real and fake images, it is determined that the discriminator training is complete, and the autoencoder training is complete.

6. The defect detection method according to claim 1, characterized in that, The step of calculating the sample error based on the first reconstruction error and the adversarial learning error includes: The first reconstruction error and the adversarial learning error are weighted and summed to obtain the sample error.

7. The defect detection method according to claim 1, characterized in that, The step of determining the error threshold based on the sample error includes: The sample errors are sorted in ascending order to obtain a total list of sample errors and a serial number for each sample error; Select a sample error from the total list of sample errors according to the second preset condition as the error threshold.

8. The defect detection method according to claim 1, characterized in that, The step of determining the detection result of the test sample image based on the second reconstruction error and the error threshold includes: When the second reconstruction error is greater than or equal to the error threshold, it is determined that the test sample image has defects; or When the second reconstruction error is less than the error threshold, the test sample image is determined to be flawless.

9. An electronic device, characterized in that, The electronic device includes: Memory, storing at least one instruction; and The processor executes instructions stored in the memory to implement the defect detection method as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one instruction, which is executed by a processor in an electronic device to implement the defect detection method as described in any one of claims 1 to 8.

Images