Defect detection method, electronic device, and storage medium

Abstract

The application relates to image processing and provides a flaw detection method, an electronic device and a storage medium. The flaw detection method comprises the following steps: acquiring a to-be-detected sample image, a flaw image and a non-flaw image, obtaining a plurality of flaw detection accuracies based on the flaw image and the non-flaw image, and dividing the to-be-detected sample image into a plurality of to-be-detected regions; selecting a corresponding auto-encoder of each flaw detection accuracy from a plurality of constructed learners based on the flaw image; training the corresponding auto-encoder based on the non-flaw image to obtain a multi-scale auto-encoder model; inputting a plurality of to-be-detected image blocks segmented from each to-be-detected region into the corresponding multi-scale auto-encoder model to obtain a plurality of reconstructed image blocks; calculating a reconstruction error value between each reconstructed image block and the corresponding to-be-detected image block, and generating a detection result of a to-be-detected sample in the to-be-detected sample image. According to the application, different sizes of flaws in the image can be accurately detected.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing, and in particular to a flaw detection method, an electronic device and a storage medium. BACKGROUND

[0002] In the current flaw detection method, when there are multiple flaws of different sizes in the image to be detected, false positives or detection omissions are likely to occur, resulting in low accuracy of flaw detection. SUMMARY

[0003] In view of the above, it is necessary to provide a flaw detection method, an electronic device and a storage medium, which can accurately detect flaws of different sizes in an image.

[0004] The present application provides a flaw detection method, which comprises:

[0005] obtaining a sample image to be detected, a flaw image and a non-flaw image;

[0006] dividing the sample image to be detected into a plurality of detection regions, and generating a flaw detection accuracy corresponding to each detection region based on the flaw image and the non-flaw image;

[0007] constructing a plurality of learners;

[0008] selecting an autoencoder corresponding to each flaw detection accuracy from the plurality of learners;

[0009] training each autoencoder based on the non-flaw image to obtain a multi-scale autoencoder model;

[0010] segmenting each detection region into a plurality of image blocks to be detected, and inputting the plurality of image blocks to be detected into the corresponding multi-scale autoencoder model to obtain a plurality of reconstructed image blocks;

[0011] calculating a reconstruction error value between each reconstructed image block and the corresponding image block to be detected;

[0012] generating a detection result of the sample to be detected in the sample image to be detected based on the reconstruction error value.

[0013] According to an optional embodiment of the present application, the generation of a flaw detection accuracy corresponding to each detection region based on the flaw image and the non-flaw image comprises:

[0014] dividing the flaw image into a plurality of flaw regions, and dividing the non-flaw image into a plurality of non-flaw regions;

[0015] segmenting each flaw region into a plurality of flaw image blocks, and segmenting the non-flaw region into a plurality of non-flaw image blocks;

[0016] For each defect area, a similarity value between each defect image block and a corresponding non-defect image block is calculated;

[0017] A target average value corresponding to each defect area is obtained by calculating an average value of the similarity values of the plurality of defect image blocks in each defect area;

[0018] A parameter range of each defect area is generated based on the target average value;

[0019] The parameter range of each defect area is determined as the defect detection accuracy of each to-be-tested area at the corresponding position.

[0020] According to an optional embodiment of the present application, the selecting, from the plurality of learners, the autoencoder corresponding to each defect detection accuracy comprises:

[0021] A defect image block of each defect area and a non-defect image block of a corresponding non-defect area are determined as a test image block;

[0022] A label result of the test image block is obtained;

[0023] The test image block is input into each learner to obtain a plurality of first reconstructed image blocks;

[0024] An error value between each first reconstructed image block and a corresponding test image block is calculated;

[0025] A verification result is generated according to the calculated error value and a first preset value;

[0026] The verification result is compared with the label result to obtain a comparison result;

[0027] A correctness rate of each learner is calculated according to the comparison result;

[0028] The learner corresponding to the highest correctness rate is selected as the autoencoder corresponding to each defect area;

[0029] According to the correspondence between each defect area and each defect detection accuracy, the autoencoder corresponding to each defect detection accuracy is determined.

[0030] According to an optional embodiment of the present application, the calculating, according to the comparison result, the correctness rate of each learner comprises:

[0031] The verification result same as the label result is determined as a target result;

[0032] A first number of the target result is counted, and a second number of the verification result is counted;

[0033] According to the first quantity and the second quantity, a ratio of the target result in the verification result is calculated, and the ratio is determined as a correct rate of each learner.

[0034] According to an optional embodiment of the present application, the constructing a plurality of learners comprises:

[0035] Constructing a potential space with multiple scales;

[0036] According to each scale and an image size of the image block to be tested, a plurality of operation results are obtained by operation;

[0037] For each scale, a corresponding number of hidden layers are constructed as encoders according to the plurality of operation results, and a corresponding number of operation layers are constructed as decoders;

[0038] According to the encoders of each scale, the potential space and the decoders, the plurality of learners are generated.

[0039] According to an optional embodiment of the present application, the inputting the plurality of image blocks to be tested into the corresponding multi-scale autoencoder model to obtain a plurality of reconstructed image blocks comprises:

[0040] Each image block to be tested is input into a hidden layer of the corresponding multi-scale autoencoder model for feature extraction, and a feature vector output by each hidden layer is obtained, wherein the feature vector output by each hidden layer is used as an input vector of a next hidden layer;

[0041] A feature vector output by a last hidden layer is determined as a potential vector corresponding to the image block to be tested;

[0042] The potential vector is input into an operation layer of the corresponding multi-scale autoencoder model for reconstruction, and a reconstruction vector output by each operation layer is obtained, wherein the reconstruction vector output by each operation layer is used as an input vector of a next operation layer;

[0043] A reconstruction vector output by a last operation layer is converted to obtain the plurality of reconstructed image blocks.

[0044] According to an optional embodiment of the present application, the calculating a reconstruction error value between each reconstructed image block and the corresponding image block to be tested comprises:

[0045] A pixel value of a pixel point in each reconstructed image block is subtracted from a pixel value of a corresponding pixel point in the image block to be tested to obtain a difference image;

[0046] The difference image is binarized to obtain a binarized image;

[0047] From the binarized image, a pixel point with a configuration value is extracted as a target region;

[0048] The area of ​​the target region is calculated to obtain the reconstruction error value.

[0049] According to an optional embodiment of this application, generating the detection result of the test sample in the test sample image based on the reconstruction error value includes:

[0050] The reconstruction error value is compared with a second preset value;

[0051] If the reconstruction error value is greater than the second preset value, the image block to be tested corresponding to the reconstruction error value is determined as the target image block;

[0052] The defect location is determined based on the position of the target image block in the sample image to be tested.

[0053] The test sample and the location of the defect are determined as the test result.

[0054] This application provides an electronic device, the electronic device comprising:

[0055] Memory, storing at least one instruction; and

[0056] The processor retrieves instructions stored in the memory to implement the defect detection method.

[0057] This application provides a computer-readable storage medium storing at least one instruction, which is executed by a processor in an electronic device to implement the defect detection method described above.

[0058] As can be seen from the above technical solution, the sample image to be tested is divided into multiple regions to be tested, and a defect detection accuracy corresponding to each region to be tested is generated based on the defective and flawless images. A correspondence is established between each defect detection accuracy and each region to be tested. Multiple learners are constructed, and a multi-scale autoencoder model corresponding to each defect detection accuracy is selected from the multiple learners. Based on the correspondence, the corresponding autoencoder model can be selected to detect each region to be tested. Since there is an inverse relationship between each defect detection accuracy and the size of each defect (that is, each defect detection accuracy can characterize the size of the defect in the corresponding region to be tested), inputting the multiple image blocks to be tested into the corresponding multi-scale autoencoder model for detection can achieve… The present invention detects defects of different sizes in the test sample image. Since the multiple defect detection accuracies comprehensively characterize the size of all defects, the autoencoder corresponding to each defect detection accuracies is used to detect each test region. This enables comprehensive detection of defects of different sizes in the test image, avoiding detection omissions or misjudgments. Each test region is divided into multiple test image blocks and then input into the corresponding multi-scale autoencoder model for detection. Since it is not necessary to scale the test sample image, the loss of image details is avoided. By performing defect detection on each test image block, the image range of each defect detection is reduced. Therefore, the test sample image can be detected more finely, improving the accuracy of defect detection. Attached Figure Description

[0059] Figure 1 This is an application environment diagram of a preferred embodiment of the defect detection method of this application.

[0060] Figure 2 This is a flowchart of a preferred embodiment of the defect detection method of this application.

[0061] Figure 3 This is a schematic diagram illustrating the generation of the learner in a preferred embodiment of the defect detection method of this application.

[0062] Figure 4 This is a schematic diagram of the structure of an electronic device that implements the defect detection method of this application. Detailed Implementation

[0063] To make the objectives, technical solutions, and advantages of this application clearer, the application will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0064] like Figure 1The diagram illustrates an application environment of a preferred embodiment of the defect detection method of this application. The defect detection method can be applied to one or more electronic devices 1. The electronic devices 1 communicate with a camera device 2, which can be a webcam or other device for capturing images. For example, the camera device 2 can capture images of multiple defective products, thus obtaining multiple defective images, and simultaneously capture images of multiple flawless products, thus obtaining multiple flawless images. The products can be various devices, such as mobile phones.

[0065] The electronic device 1 is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions. Its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0066] The electronic device 1 can be any electronic product that can interact with the user, such as a personal computer, tablet computer, smartphone, personal digital assistant (PDA), game console, interactive network television (IPTV), smart wearable device, etc.

[0067] The electronic device 1 may further include network devices and / or user devices. The network devices include, but are not limited to, a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing consisting of a large number of hosts or network servers.

[0068] The network where the electronic device 1 is located includes, but is not limited to, the Internet, wide area network, metropolitan area network, local area network, and virtual private network (VPN).

[0069] like Figure 2 The diagram shown is a flowchart of a preferred embodiment of a defect detection method according to this application. Depending on different needs, the order of the steps in this flowchart can be adjusted according to actual detection requirements, and some steps can be omitted. The method is executed by an electronic device, for example... Figure 1 Electronic device 1 shown.

[0070] S10: Acquire the image of the sample to be tested, the image of the defective sample, and the image of the sample without defects.

[0071] In at least one embodiment of this application, each defective image refers to an image containing a defective product. It is understood that each flawless image refers to an image containing a flawless product. The defective product can be a phone case with scratches, a water cup with color differences, etc. The defective image can be used to generate a reconstructed image, and the flawless image can be used to train an autoencoder. It is understood that the defective image and the flawless image in this application are images of the same type of product. For example, when the defective image is an image of a water cup with color differences, the flawless image is an image of a water cup without color differences.

[0072] In at least one embodiment of this application, the sample image to be tested refers to an image for which defects need to be detected.

[0073] In at least one embodiment of this application, the electronic device acquires the image of the sample to be tested, the image of defects, and the image of no defects by means of:

[0074] The electronic device controls the camera to capture images of the sample under test at the same position and angle to obtain an image of the sample under test. The electronic device retrieves the defective image and the flawless image from a pre-configured first database.

[0075] The sample to be tested can be a product such as a water cup, and the camera device can be a camera.

[0076] S11, the sample image to be tested is divided into multiple regions to be tested, and a defect detection accuracy corresponding to each region to be tested is generated based on the defective image and the flawless image.

[0077] In at least one embodiment of this application, the plurality of areas to be tested refers to a plurality of areas that need to be tested for the presence of defects.

[0078] In at least one embodiment of this application, the defect detection accuracy refers to the average value of similarity values, wherein the similarity value refers to the similarity between the defective region in the defective image and the corresponding flawless region in the flawless image.

[0079] In at least one embodiment of this application, the electronic device divides the sample image to be tested into multiple test regions in various ways. For example, the electronic device divides the sample image to be tested into an upper region and a lower region.

[0080] In at least one embodiment of this application, the electronic device generates the defect detection accuracy corresponding to each test area based on the defective image and the flawless image, including:

[0081] The electronic device divides the flawed image into multiple flawed regions and the flawless image into multiple flawless regions. Further, the electronic device divides each flawed region into multiple flawed image blocks and the flawless region into multiple flawless image blocks. For each flawed region, the electronic device calculates the similarity value between each flawed image block and its corresponding flawless image block. It then calculates the average of the similarity values ​​of multiple flawed image blocks within each flawed region to obtain the target average value for each flawed region. Furthermore, the electronic device generates a parameter range for each flawed region based on the target average value, and determines the parameter range of each flawed region as the flaw detection accuracy for each region to be tested at the corresponding location.

[0082] Furthermore, in other embodiments of this application, a pre-defined division rule can be used to divide the flawed image, the flawless image, and the sample image to be tested. For example, the division rule can be set based on parameters such as preset size and preset position. The preset position can be determined based on a pre-constructed coordinate system. For example, the direction of the horizontal and vertical axes can be determined based on the image center point or any other point as a reference point, thereby constructing a two-dimensional coordinate system.

[0083] The target average value is the ratio of the similarity value to the number of the plurality of defective image blocks, and the similarity value ranges from [0,1].

[0084] Specifically, the similarity value can be calculated as follows:

[0085]

[0086] c1 = (K1L) 2 ;

[0087] c2 = (K2L) 2 ;

[0088] Where SSIM(x,y) is the similarity value, x is the defective image patch, y is the defect-free image patch corresponding to the defective image patch, and μ x μ is the average grayscale value of the defective image patch. y σ is the average grayscale value of the corresponding flawless image patch. x σ represents the standard deviation of the grayscale values ​​of the defective image patch. y The grayscale standard deviation σ of the corresponding flawless image patch xy Let c1 and c2 be the grayscale covariance between the defective image block and the corresponding flawless image block, respectively. c1 and c2 are parameters that keep the denominator of the similarity value from being zero. L is the dynamic range of the pixel values ​​of the pixels in the defective image block. K1 and K2 are pre-set constants, and K1 << 1 and K2 << 1.

[0089] Specifically, the parameter range for generating each defect area based on the target average value by the electronic device includes:

[0090] The electronic device sorts the target average values ​​in ascending order. The electronic device generates a first interval based on zero and the second target average value, and uses the first interval as the parameter range of the defect area corresponding to the first target average value. The electronic device generates a second interval based on the second target average value and the third target average value, and uses the second interval as the parameter range of the defect area corresponding to the second target average value. The above operation is repeated until the last target average value needs to be used for the generation operation. The electronic device generates a feature interval based on the last target average value and the value 1, and uses the feature interval as the parameter range of the defect area corresponding to the last target average value.

[0091] For example, the electronic device divides the flawed image into a first flawed region, a second flawed region, and a third flawed region, and divides the flawless image into a first flawless region, a second flawless region, and a third flawless region. The electronic device further divides each flawed region into multiple flawed image blocks and each flawless region into multiple flawless image blocks. It calculates the similarity value between each flawed image block and its corresponding flawless image block, and calculates the average of these similarity values ​​to obtain the target average value for each flawed region. If the target average value for the first flawed region is 0.4, the target average value for the second flawed region is 0.5, and the target average value for the third flawed region is 0.8, the electronic device... The electronic device generates a first interval [0, 0.5) that does not contain 0.5 based on zero and 0.5, and uses the first interval [0, 0.5) as the first parameter range of the first defect area. The electronic device generates a second interval [0.5, 0.8) that does not contain 0.8 based on 0.5 and 0.8, and uses the second interval [0.5, 0.8) as the second parameter range of the second defect area. The electronic device generates a feature interval [0.8, 1] based on 0.8 and the value 1, and uses the feature interval [0.8, 1] as the third parameter range, and uses the third parameter range as the third parameter range of the third defect area. The parameter range of each defect area is determined as the defect detection accuracy of each test area at the corresponding position.

[0092] Through the above implementation method, the similarity value between each defective area and the corresponding flawless area is calculated, and the similarity value is used as the defect detection accuracy for each area to be tested. Based on different defect detection accuracies, it is possible to preliminarily determine whether each area to be tested has defects. The smaller the defect detection accuracy, the greater the possibility that the area to be tested has defects. The parameter range of each target average value is used as the defect detection accuracy. Since the target similarity value can characterize the size of the defects in the corresponding defective area, and each parameter range refers to the range of each target similarity value, multiple defect detection accuracies can comprehensively characterize the size of different defects.

[0093] S12, construct multiple learners.

[0094] In at least one embodiment of this application, the plurality of learners can be used to generate reconstructed image patches based on input image patches.

[0095] In at least one embodiment of this application, the electronic device constructs multiple learners including:

[0096] The electronic device constructs a latent space of multiple scales, performs calculations based on each scale and the image size of the image block to be tested, and obtains multiple calculation results. For each scale, the electronic device constructs a corresponding number of hidden layers as encoders and a corresponding number of computational layers as decoders based on the multiple calculation results. The electronic device generates the multiple learners based on the encoder, latent space and decoder for each scale.

[0097] Each hidden layer consists of a cascaded convolutional layer, a batch normalization layer, and an activation function layer, while each operational layer consists of a cascaded deconvolutional layer, the batch normalization layer, and the activation function layer. The activation function layer is ReLU.

[0098] The latent space refers to the space where the extracted image features exist. The multiple scales can include 4*4, 8*8, and 16*16.

[0099] The image size refers to the number of pixels contained in the horizontal and vertical directions. The image size can be used to calculate the number of hidden layers in the plurality of learners. The image size includes, but is not limited to, 64*64 and 32*32.

[0100] Specifically, the electronic device constructs a potential space of multiple scales, including:

[0101] The electronic device sets the size of the potential space to multiple scales and constructs the potential space according to each scale, thereby obtaining multiple potential spaces.

[0102] Specifically, the electronic device performs calculations based on each scale and the image size of the image block to be measured, obtaining multiple calculation results, including:

[0103] The electronic device iteratively divides the image size by the target setting value to obtain a result. If the result differs from any scale, the next iterative division operation is performed, and the result is determined as the image size (referred to as the "target image size") for the next division operation. The electronic device divides the target image size by the target setting value until the result is the same as any of the multiple scales, at which point the division operation stops. Through iterative division, multiple results equivalent to each scale can be obtained. For example, if there are two scales, iterative calculations are required for each scale. Taking scale A as an example, during the iterative division calculation, multiple results can be obtained until a result equivalent to scale A is obtained, at which point the iteration stops, and the number of iterations required to obtain the result equivalent to scale A is determined. This number of iterations is used as the number of hidden layers in each learner, and the multiple results are used as the size of the feature map generated by the encoder.

[0104] The target setting values ​​include, but are not limited to, 2 and 4.

[0105] It is understood that the generation processes of the encoder and the decoder are reversible, and this application will not elaborate on the generation process of the decoder.

[0106] like Figure 3The diagram shown is a schematic representation of the learner generation method of a preferred embodiment of the present application's defect detection method. When the image size is 32*32, the latent space scale is 4*4, and the target setting value is 2, the image size is divided by the target setting value for the first time to obtain a first division result of 16*16. The first division result is then divided by the target setting value for the second time to obtain a second division result of 8*8. The second division result of 8*8 is the same as the latent space scale of 8*8. The calculation is stopped, resulting in a calculation count of 2. A network with 2 hidden layers is constructed as the encoder. The first calculation result of 16*16 is used as the size of the feature map output by the first hidden layer, and the second calculation result of 8*8 is used as the size of the feature map output by the second hidden layer. The learner is constructed and includes an encoder, a latent space with a scale of 16*16, and a decoder network. The encoder network contains two hidden layers, wherein the feature map output by the first hidden layer is 16*16 in size, and the feature map output by the second hidden layer is 8*8 in size, which is the result of the first division. The decoder network contains two operational layers, wherein the feature map output by the first operational layer is 16*16 in size, and the feature map output by the second operational layer is 32*32 in size.

[0107] Through the above implementation method, when using each learner for feature extraction, the size of the features extracted by each learner can be controlled according to the operation result. Since the size of the features extracted by each learner is different, the reconstruction accuracy of each learner is also different.

[0108] S13, Select an autoencoder corresponding to the defect detection accuracy from the plurality of learners.

[0109] In at least one embodiment of this application, the autoencoder is selected from the plurality of learners based on the flawed image blocks and the flawless image blocks.

[0110] In at least one embodiment of this application, the electronic device selects an autoencoder corresponding to each defect detection accuracy from the plurality of learners, including:

[0111] The electronic device identifies the defective image block of each defective region and the corresponding flawless image block of the flawless region as test image blocks. The electronic device acquires the labeling result of the test image blocks and inputs the test image blocks into each learner to obtain multiple first reconstructed image blocks. Further, the electronic device calculates the error value between each first reconstructed image block and the corresponding test image block, and generates a verification result based on the calculated error value and a first preset value. Further still, the electronic device compares the verification result with the labeling result to obtain a comparison result. Further still, the electronic device calculates the accuracy of each learner based on the comparison result, selects the learner corresponding to the highest accuracy as the autoencoder corresponding to each defective region, and determines the autoencoder corresponding to each defect detection accuracy based on the correspondence between each defective region and each defect detection accuracy.

[0112] The marking results can be obtained from a pre-configured second database. The marking results include whether any test image block has defects or not. The marking results can be used to compare with the verification results.

[0113] The comparison results include cases where the verification result is the same as the labeling result, and cases where the verification result is different from the labeling result.

[0114] The first preset value can be set by the user, and this application does not impose any restrictions.

[0115] The accuracy rate refers to the probability that the verification result is the same as the labeling result.

[0116] Specifically, the electronic device generates a verification result based on the calculated error value and a first preset value, including:

[0117] If the calculated error value is equal to or greater than the first preset value, the electronic device determines the verification result corresponding to the calculated error value as indicating that the test image block has defects; or, if the calculated error value is less than the first preset value, the electronic device determines the verification result corresponding to the calculated error value as indicating that the test image block does not have defects.

[0118] Specifically, the electronic device calculates the accuracy of each learner based on the comparison results, including:

[0119] The electronic device identifies the verification result that is the same as the labeling result as the target result, counts a first number of the target results, and counts a second number of the verification results. Further, the electronic device calculates the ratio of the target result in the verification results based on the first number and the second number, and determines the ratio as the accuracy of each learner.

[0120] In the above implementation method, the defective image block and the corresponding flawless image block are used as the test image block, and the test image block is input into each learner to obtain the plurality of first reconstructed image blocks. Since the difference between the defective image block and the flawless image block is large, the difficulty of generating an accurate first reconstructed image block by each learner is increased. Therefore, the accuracy can accurately reflect the reconstruction capability of each learner. By selecting the learner with the highest accuracy as the autoencoder corresponding to each defect detection accuracy, the reconstruction accuracy of the autoencoder can be improved.

[0121] S14, the autoencoder is trained based on the flawless image to obtain a multi-scale autoencoder model.

[0122] In at least one embodiment of this application, the multi-scale autoencoder model is generated by training the autoencoder on the flawless image, and the multi-scale autoencoder model can be used to generate images similar to the flawless image.

[0123] In at least one embodiment of this application, the process of training the autoencoder to obtain the multi-scale autoencoder model is consistent with the process of generating multiple reconstruction blocks described below, so this application will not repeat it here.

[0124] In this embodiment, the flawless image is used to train the autoencoder to generate the multi-scale autoencoder model, and the multi-scale autoencoder model learns the features of the flawless image.

[0125] S15, each region to be tested is divided into multiple image blocks to be tested, and the multiple image blocks to be tested are input into the corresponding multi-scale autoencoder model to obtain multiple reconstructed image blocks.

[0126] In at least one embodiment of this application, the plurality of reconstructed image blocks refer to image blocks reconstructed by the multi-scale autoencoder model based on the plurality of image blocks to be tested.

[0127] In at least one embodiment of this application, the electronic device inputs the plurality of image blocks to be tested into a corresponding multi-scale autoencoder model to obtain a plurality of reconstructed image blocks, including:

[0128] The electronic device inputs each image patch to be tested into the hidden layer of the corresponding multi-scale autoencoder model for feature extraction, obtaining a feature vector output by each hidden layer. The feature vector output by each hidden layer serves as the input vector for the next hidden layer. The electronic device determines the feature vector output by the last hidden layer as the latent vector corresponding to the image patch to be tested. Further, the electronic device inputs the latent vector into the operational layer of the corresponding multi-scale autoencoder model for reconstruction, obtaining a reconstructed vector output by each operational layer. The reconstructed vector output by each operational layer serves as the input vector for the next operational layer. The electronic device performs a transformation process on the reconstructed vector output by the last operational layer to obtain the multiple reconstructed image patches.

[0129] The latent vector refers to the feature obtained by the multi-scale autoencoder model after compressing each image block to be tested. The reconstructed vector refers to the feature obtained by the multi-scale autoencoder model after decompressing the latent vector. There are multiple ways to transform the reconstructed vector output by the last operational layer, and all of these methods are existing technologies, which will not be elaborated here.

[0130] Through the above implementation method, the corresponding autoencoder is selected according to the defect detection accuracy of each test area to detect the test area. Since the multiple defect detection accuracies comprehensively characterize the size of different defects, using the autoencoder corresponding to each defect detection accuracy to detect each test area can comprehensively detect defects of different sizes in the test image, avoid detection omissions or misjudgments, and improve the accuracy of defect detection.

[0131] S16, calculate the reconstruction error value between each reconstructed image block and the corresponding image block to be tested.

[0132] In at least one embodiment of this application, the reconstruction error value refers to the difference between each reconstructed image block and the corresponding image block to be tested, and the reconstruction error value can be used to determine whether the image block to be tested has defects.

[0133] In at least one embodiment of this application, the electronic device calculates the reconstruction error value between each reconstructed image block and the corresponding image block to be tested, including:

[0134] The electronic device subtracts the pixel value of each pixel in the reconstructed image block from the pixel value of the corresponding pixel in the image block to be tested to obtain a difference image. Further, the electronic device binarizes the difference image to obtain a binarized image. Even further, the electronic device extracts the pixel values ​​with configured values ​​from the binarized image as target regions and calculates the area of ​​the target regions to obtain the reconstruction error value.

[0135] The configuration values ​​can be set by the user, and this application does not impose any restrictions on them.

[0136] Through the above implementation method, the reconstruction error value between each reconstructed image block and the corresponding image block to be tested can be calculated. Since the multi-scale autoencoder model has learned the features of the flawless image, the multiple reconstructed image blocks can be regarded as flawless image blocks. The reconstruction error value can accurately reflect the difference between the reconstructed image block and the corresponding image block to be tested. Using the reconstruction error value, it is possible to accurately detect whether there are defects in the image block to be tested.

[0137] S17, Based on the reconstruction error value, generate the detection result of the sample to be tested in the sample image to be tested.

[0138] In at least one embodiment of this application, the detection result refers to whether the sample to be tested has defects, and if defects are present, the specific location of the defects.

[0139] In at least one embodiment of this application, the electronic device generates a detection result of the sample to be tested in the sample image to be tested based on the reconstruction error value, including:

[0140] The electronic device compares the reconstruction error value with a second preset value. If the reconstruction error value is greater than the second preset value, the electronic device determines the image block to be tested corresponding to the reconstruction error value as the target image block, and locates the defect location according to the position of the target image block in the image to be tested. The electronic device determines the image to be tested and the defect location as the detection result.

[0141] The second preset value can be set as needed, and this application does not impose any restrictions on it. The target image block refers to an image block with defects.

[0142] Through the above implementation method, the image block to be tested corresponding to the reconstruction error value greater than the second preset value is determined as the target image block. Since the larger the reconstruction error value, the greater the difference, the greater the possibility that the image block to be tested has defects. By comparing the reconstruction error value with the second preset value, it is possible to quickly and accurately detect whether the sample to be tested has defects and locate the location of the defects.

[0143] In at least one embodiment of this application, if the target image block is not present in the image block to be tested, the electronic device takes the sample to be tested as a flawless sample and the image of the sample to be tested as a flawless image as the detection result.

[0144] As can be seen from the above technical solution, the sample image to be tested is divided into multiple regions to be tested, and a defect detection accuracy corresponding to each region to be tested is generated based on the defective and flawless images. A correspondence is established between each defect detection accuracy and each region to be tested. Multiple learners are constructed, and a multi-scale autoencoder model corresponding to each defect detection accuracy is selected from the multiple learners. Based on the correspondence, the corresponding autoencoder model can be selected to detect each region to be tested. Since there is an inverse relationship between each defect detection accuracy and the size of each defect (that is, each defect detection accuracy can characterize the size of the defect in the corresponding region to be tested), inputting the multiple image blocks to be tested into the corresponding multi-scale autoencoder model for detection can achieve… The present invention detects defects of different sizes in the test sample image. Since the multiple defect detection accuracies comprehensively characterize the size of all defects, the autoencoder corresponding to each defect detection accuracies is used to detect each test region. This enables comprehensive detection of defects of different sizes in the test image, avoiding detection omissions or misjudgments. Each test region is divided into multiple test image blocks and then input into the corresponding multi-scale autoencoder model for detection. Since it is not necessary to scale the test sample image, the loss of image details is avoided. By performing defect detection on each test image block, the image range of each defect detection is reduced. Therefore, the test sample image can be detected more finely, improving the accuracy of defect detection.

[0145] like Figure 4 The diagram shown is a schematic representation of the structure of an electronic device that implements the defect detection method of this application.

[0146] In one embodiment of this application, the electronic device 1 includes, but is not limited to, a memory 12, a processor 13, and a computer program, such as a defect detection program, stored in the memory 12 and executable on the processor 13.

[0147] Those skilled in the art will understand that the schematic diagram is merely an example of electronic device 1 and does not constitute a limitation on electronic device 1. It may include more or fewer components than shown in the diagram, or combine certain components, or different components. For example, electronic device 1 may also include input / output devices, network access devices, buses, etc.

[0148] The processor 13 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processor 13 is the computing core and control center of the electronic device 1, connecting various parts of the electronic device 1 through various interfaces and lines, and obtaining the operating system of the electronic device 1 and various installed applications and program code.

[0149] The processor 13 acquires the operating system and various installed applications of the electronic device 1. The processor 13 acquires these applications to implement the steps in the various defect detection method embodiments described above, for example... Figure 2 The steps are shown.

[0150] For example, the computer program may be divided into one or more modules / units, which are stored in the memory 12 and retrieved by the processor 13 to complete this application. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the process of retrieving the computer program from the electronic device 1.

[0151] The memory 12 can be used to store the computer programs and / or modules. The processor 13 implements various functions of the electronic device 1 by running or retrieving the computer programs and / or modules stored in the memory 12, and by calling the data stored in the memory 12. The memory 12 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 12 may include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other non-volatile solid-state storage device.

[0152] The memory 12 can be the external memory and / or internal memory of the electronic device 1. Furthermore, the memory 12 can be a physical memory, such as a memory module, a TF card (Trans-flash Card), etc.

[0153] If the modules / units integrated in the electronic device 1 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when the computer program is acquired by a processor, it can implement the steps of the various method embodiments described above.

[0154] The computer program includes computer program code, which may be in the form of source code, object code, accessible file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, and read-only memory (ROM).

[0155] Combination Figure 2 The memory 12 in the electronic device 1 stores multiple instructions to implement a defect detection method. The processor 13 can acquire the multiple instructions to implement: acquiring a sample image to be tested, a defective image, and a defect-free image; dividing the sample image to be tested into multiple regions to be tested, and generating a defect detection accuracy corresponding to each region to be tested based on the defective image and the defect-free image; constructing multiple learners; selecting an autoencoder corresponding to each defect detection accuracy from the multiple learners; training each autoencoder based on the defect-free image to obtain a multi-scale autoencoder model; dividing each region to be tested into multiple image blocks to be tested, and inputting the multiple image blocks to be tested into the corresponding multi-scale autoencoder model to obtain multiple reconstructed image blocks; calculating the reconstruction error value between each reconstructed image block and the corresponding image block to be tested; and generating the detection result of the sample to be tested in the sample image to be tested based on the reconstruction error value.

[0156] Specifically, the processor 13's implementation method for the above instructions can be found in [reference needed]. Figure 2 The descriptions of the relevant steps in the corresponding embodiments are not repeated here.

[0157] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0158] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0159] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0160] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of this application is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within this application. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0161] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices described in this application may also be implemented by a single unit or device through software or hardware. The terms "first," "second," etc., are used to indicate names and do not indicate any specific order.

[0162] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application and are not intended to limit it. Although this application has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of this application without departing from the spirit and scope of the technical solutions of this application.

Claims

1. A defect detection method, characterized in that, The defect detection method includes: Acquire images of the product's test sample, images of defects, and images of products without defects; The sample image to be tested is divided into multiple regions to be tested, and the defect detection accuracy corresponding to each region to be tested is determined based on the similarity value between the defective region corresponding to each region in the defective image and the flawless region corresponding to each region in the flawless image. Constructing multiple learners includes: constructing a latent space of multiple scales; performing calculations based on each scale and the image size of the image patch to be tested to obtain multiple calculation results; for each scale, constructing a corresponding number of hidden layers as encoders and a corresponding number of computation layers as decoders based on the multiple calculation results; and generating the multiple learners based on the encoder, latent space, and decoder for each scale. Select an autoencoder from the plurality of learners that corresponds to the defect detection accuracy for each defect; Each autoencoder is trained based on the flawless image to obtain a multi-scale autoencoder model; Each region to be tested is divided into multiple image blocks to be tested, and the multiple image blocks to be tested are input into the corresponding multi-scale autoencoder model to obtain multiple reconstructed image blocks; Calculate the reconstruction error value between each reconstructed image block and the corresponding image block to be tested; The detection result of the sample to be tested in the sample image to be tested is determined based on the reconstruction error value.

2. The defect detection method as described in claim 1, characterized in that, The method for determining the accuracy of defect detection includes: The flawed image is divided into multiple flawed regions, and the flawless image is divided into multiple flawless regions; Each defective region is divided into multiple defective image blocks, and the defect-free region is divided into multiple defect-free image blocks; For each defective region, calculate the similarity value between each defective image patch and its corresponding flawless image patch; The target average value for each defective region is obtained by calculating the average similarity value of multiple defective image patches in each defective region. Generate the parameter range for each defect area based on the target average value; The parameter range for each defect area is determined as the defect detection accuracy for each area to be tested at the corresponding location.

3. The defect detection method as described in claim 2, characterized in that, The step of selecting an autoencoder corresponding to the defect detection accuracy from the plurality of learners includes: The flawed image block of each flawed area and the corresponding flawless image block of the flawless area are determined as the test image block; Obtain the labeling results of the test image blocks; The test image blocks are input into each learner to obtain multiple first reconstructed image blocks; Calculate the error value between each first reconstructed image block and its corresponding test image block; The verification result is generated based on the calculated error value and the first preset value; The verification result is compared with the labeling result to obtain the comparison result; Calculate the accuracy of each learner based on the comparison results; The learner with the highest accuracy is selected as the autoencoder for each defective region. Based on the correspondence between each defect area and each defect detection accuracy, the autoencoder corresponding to each defect detection accuracy is determined.

4. The defect detection method as described in claim 3, characterized in that, The calculation of the accuracy of each learner based on the comparison results includes: The verification result that is identical to the marked result is identified as the target result; Count the first number of the target results, and count the second number of the verification results; The target result is calculated as a percentage of the validation results based on the first quantity and the second quantity, and the percentage is determined as the accuracy of each learner.

5. The defect detection method as described in claim 1, characterized in that, The step of inputting the multiple image blocks to be tested into the corresponding multi-scale autoencoder model to obtain multiple reconstructed image blocks includes: Each image block to be tested is input into the hidden layer of the corresponding multi-scale autoencoder model for feature extraction, and the feature vector output by each hidden layer is obtained, wherein the feature vector output by each hidden layer is used as the input vector of the next hidden layer. The feature vector output from the last hidden layer is determined as the latent vector corresponding to the image to be tested. The latent vector is input into the corresponding multi-scale autoencoder model's operational layer for reconstruction, resulting in a reconstructed vector output by each operational layer, where the reconstructed vector output by each operational layer is the input vector of the next operational layer. The reconstruction vector output from the last computational layer is transformed to obtain the multiple reconstructed image blocks.

6. The defect detection method as described in claim 1, characterized in that, The calculation of the reconstruction error value between each reconstructed image block and the corresponding image block to be tested includes: The difference image is obtained by subtracting the pixel value of the corresponding pixel in the image block under test from the pixel value of the reconstructed image block. The difference image is binarized to obtain a binarized image; Extract the pixels with the configured values ​​from the binarized image as the target region; The area of ​​the target region is calculated to obtain the reconstruction error value.

7. The defect detection method as described in claim 1, characterized in that, The determination of the detection result of the sample to be tested in the sample image to be tested based on the reconstruction error value includes: The reconstruction error value is compared with a second preset value; If the reconstruction error value is greater than the second preset value, the image block to be tested corresponding to the reconstruction error value is determined as the target image block; The defect location is determined based on the position of the target image block in the sample image to be tested. The test sample and the location of the defect are determined as the test result.

8. An electronic device, characterized in that, The electronic device includes: Memory, storing at least one instruction; and The processor executes the at least one instruction to implement the defect detection method as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one instruction, which is executed by a processor in an electronic device to implement the defect detection method as described in any one of claims 1 to 7.

Images