Image defect detection model building method, image detection method and electronic device
By generating training images and optimizing the automatic encoder and image segmentation network parameters, an image defect detection model is established, which solves the problem of poor accuracy in neural network algorithms when detecting image defects, and achieves higher detection accuracy and accuracy.
Patent Information
- Application Number
- CN202310690370.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-06-12
- Publication Date
- 2024-12-13
AI Technical Summary
In the prior art, neural network algorithms have poor accuracy when detecting image defects, especially when the number of training samples is insufficient or unknown defects, the automatic encoder model cannot effectively learn the defect characteristics, resulting in inaccurate detection results.
By using defect-free samples and defect-samples to generate training images, input them to the automatic encoder to calculate reconstruction errors, and generate masked images in combination with the image segmentation network, calculate the total reconstruction errors, optimize the automatic encoder and image segmentation network parameters, repeat this process Until the threshold is reached, an image defect detection model is established.
It significantly improves the accuracy and accuracy of image defect detection, can automatically amplify training data, and improves the accuracy of detection results, especially in the judgment of defects during the testing phase.
Smart Images
Figure CN119130886A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the establishment of an image processing model and an image processing method, and more particularly to a method for establishing an image defect detection model, a method for detecting defective images, and an electronic device for executing these methods. Background Art
[0002] Image recognition technology is widely used in defect detection, which improves the detection speed and accuracy compared with traditional manual detection. Defect detection technology relies on machine vision to compare the image to be read and the known defect image, so the defect features in the defect image must be defined in advance. Based on different application scenarios, the content of the defect image will also vary with the image in the application field. Users need to define the corresponding defect features according to the application scenario, however, this approach takes a lot of time to prepare for the preparatory work.
[0003] In order to efficiently find defects in images, neural network algorithms are used to learn the defect features in images to achieve automated image defect detection. Neural network algorithms can infer image features from a large number of unknown images input by themselves, however, there is no efficient method to infer defect features from unknown images during the training phase.
[0004] In addition, the autoencoder of the related art can be used for image reconstruction, filtering out defects during the reconstruction process to produce a flawless reconstructed image. However, if the number of samples in the training phase is insufficient or there are no defects in the training image, the autoencoder model cannot learn the defect features. In other words, the defects in the test image input to the autoencoder will still appear in the reconstructed image, resulting in incorrect output results of the autoencoder, that is, the accuracy of the judgment results in the test phase is low.
[0005] Accordingly, in the method of using a neural network algorithm to detect defects in an image, the problem of poor accuracy in interpreting defects in an image remains to be solved. Summary of the invention
[0006] According to one embodiment of the present application, a method for establishing an image defect detection model is disclosed, and the image defect detection model is used to detect whether an image is abnormal. The method for establishing the image defect detection model includes: a) using a flawless sample image and a flawed sample image to generate a first training image; b) inputting the first training image to an autoencoder to output a reconstructed image and calculate a first reconstruction error; c) calculating the difference between the first training image and the reconstructed image to generate a reconstruction error image; d) using the first training image, the reconstructed image, and the reconstruction error image to generate a second training image; e) inputting the second training image to an image segmentation network to generate a masked image and calculate a second reconstruction error; f) calculating a total reconstruction error based on the first reconstruction error and the second reconstruction error; g) optimizing the parameters of the autoencoder and the image segmentation network based on the total reconstruction error; and h) repeating steps a) to g) to complete the establishment and parameter optimization of the image defect detection model, wherein the image defect detection model includes an autoencoder and an image segmentation network.
[0007] In one embodiment, before step a), the method further comprises:
[0008] Generate a noise sample image using a random function and synthesize the noise sample image and the flawless sample image to obtain the flawed sample image; or
[0009] The defective sample image is transformed and the transformed defective sample image and the flawless sample image are synthesized to obtain the defective sample image.
[0010] In one embodiment, step b) comprises:
[0011] The pixel value of the flawless sample image and the pixel value of the reconstructed image are input into a first error function for calculation to obtain the first reconstruction error.
[0012] In one embodiment, step d) comprises:
[0013] The first training image, the reconstructed image and the reconstructed error image are combined to generate the second training image.
[0014] In one embodiment, step e) comprises:
[0015] Classifying pixels of the second training image by the image segmentation network to obtain a retained area corresponding to defects and a filtered area corresponding to the background; and
[0016] The mask image including the retained area and the filtered area is output.
[0017] In one embodiment, step e) comprises:
[0018] The pixel value of the defect sample image and the pixel value of the mask image are input into the second error function for calculation to obtain the second reconstruction error.
[0019] In one embodiment, step f) comprises:
[0020] The first reconstruction error and the second reconstruction error are weightedly calculated using the first weight and the second weight respectively to obtain the total reconstruction error.
[0021] In one embodiment, step g) comprises:
[0022] The parameters of the automatic encoder and the image segmentation network are adjusted according to the parameter value of the second error function, the first weight and the second weight.
[0023] In one embodiment, step h) comprises:
[0024] Repeat steps a) to g) until the total reconstruction error is less than a threshold value to complete the establishment of the image defect detection model and parameter optimization.
[0025] According to an embodiment of the present application, an electronic device for establishing an image defect detection model is disclosed, comprising:
[0026] A storage medium configured to store the image defect detection model; and
[0027] The processor is coupled to the storage medium and is configured to execute the aforementioned method for establishing the image defect detection model.
[0028] According to one embodiment of the present application, a method for detecting defective images is disclosed, comprising: a1) acquiring an image to be detected; b1) inputting the image to be detected into an automatic encoder and outputting a reconstructed image; c1) calculating the difference between the image to be detected and the reconstructed image to generate a reconstructed error image; d1) generating a test image based on the image to be detected, the reconstructed image and the reconstructed error image; e1) inputting the test image into an image segmentation network to generate a defect-shielding test image; and f1) determining whether the image to be detected is a defective image based on the size of the indicated area of the defect-shielding test image.
[0029] In one embodiment, step d1) comprises:
[0030] The test image is generated by combining the image to be detected, the detected reconstructed image and the detected reconstructed error image.
[0031] In one embodiment, step e1) comprises:
[0032] Classifying the pixels of the test image by the image segmentation network to obtain defect areas and background areas; and
[0033] The defect shielding test image including the defect area and the background area is output.
[0034] In one embodiment, step f1) comprises:
[0035] Calculating the area of the defect region of the defect masking test image to obtain the indicated area;
[0036] If the indicated area is larger than the threshold, the image to be detected is determined to be the defective image; and
[0037] If the indicated area is smaller than or equal to the threshold, the image to be detected is determined to be a flawless image.
[0038] According to an embodiment of the present application, an electronic device for detecting defective images is disclosed, comprising:
[0039] A storage medium configured to store the defect image detection model; and
[0040] The processor is coupled to the storage medium and is configured to execute the above-mentioned method for detecting defective images.
[0041] The image processing method proposed in the present application can automatically amplify the images used for training, and in the subsequent testing stage of testing whether there are defects in the images, greatly improve the accuracy of judging whether there are defects in the images and the positions of the defects in the images.
[0042] The above summary is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments and features described above, further aspects, embodiments and features of the present application will be readily apparent by reference to the accompanying drawings and the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the multiple drawings represent the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings only depict some embodiments disclosed in the present application and should not be regarded as limiting the scope of the present application.
[0044] Figure 1 is a block diagram of an electronic device according to an embodiment of the present application.
[0045] Figure 2 FIG. 4 is a flow chart of a method for establishing an image defect detection model according to an embodiment of the present application.
[0046] Figure 3 This is a schematic diagram of generating a first training image according to an embodiment of the present application.
[0047] Figure 4It is a schematic diagram of outputting a reconstructed image according to an embodiment of the present application.
[0048] Figure 5 This is a schematic diagram of calculating a first reconstruction error according to an embodiment of the present application.
[0049] Figure 6 It is a schematic diagram of generating a reconstructed error image according to an embodiment of the present application.
[0050] Figure 7 This is a schematic diagram of generating a second training image according to an embodiment of the present application.
[0051] Figure 8 It is a schematic diagram of generating a defect masking image according to an embodiment of the present application.
[0052] Fig. 9 This is a schematic diagram of calculating a second reconstruction error according to an embodiment of the present application.
[0053] Fig.10 It is a schematic diagram of calculating the total reconstruction error according to an embodiment of the present application.
[0054] Fig.11 Detailed description is a flow chart of a defective image detection method according to an embodiment of the present application.
[0055] Description of reference numerals:
[0056] 10: Electronic devices;
[0057] 110: storage media;
[0058] 120: processor;
[0059] 150: Image defect detection model;
[0060] 155: Autoencoder;
[0061] 157: Image Segmentation Network;
[0062] 159: operation module;
[0063] 302: flawless sample image;
[0064] 304: defective sample image;
[0065] 310: First training image;
[0066] 312: Reconstruction of images;
[0067] 314: Defects;
[0068] 316: Reconstruct error image;
[0069] 318: Defects;
[0070] 320: second training image;
[0071] 328: Defects;
[0072] 334: Defect masking image;
[0073] 336: Filter area;
[0074] 338: Reserved area;
[0075] E1: first error function;
[0076] E2: second error function;
[0077] ES: weighted formula;
[0078] K1: first reconstruction error;
[0079] K2: second reconstruction error;
[0080] KS: total reconstruction error;
[0081] S210~S280, S1110~S1180: steps. DETAILED DESCRIPTION
[0082] The present application is further described below in conjunction with drawings and embodiments so that relevant personnel in the technical field to which the present application belongs can better understand the present application and implement it accordingly, but the embodiments are not intended to limit the present application.
[0083] As used herein, terms such as "first" and "second" describe various elements, components, regions, layers and / or portions, which should not be limited by these terms. These terms may only be used to distinguish one element, component, region, layer or portion from another. Unless the context clearly indicates, terms such as "first" and "second" as used herein do not imply a sequence or order.
[0084] Please refer to Figure 1 , which is a block diagram of an electronic device according to an embodiment of the present application. The electronic device 10 includes a storage medium 110 and a processor 120. The storage medium 110 is coupled to the processor 120. The electronic device 10 is used to establish an image defect detection model 150.
[0085] In one embodiment, the image defect detection model 150 is used to detect whether an image has abnormality.
[0086] The storage medium 110 is configured to store an image defect detection model 150 . In one embodiment, the image defect detection model 150 is implemented using a neural network model architecture. The image defect detection model 150 includes an auto encoder 155 , an image segmentation network 157 , and a computing module 159 .
[0087] In one embodiment, the autoencoder 155 may be a convolutional autoencoder (CNN AutoEncoder), a sparse autoencoder (Sparse AutoEncoder), a denoising autoencoder (Denoising AutoEncoder), or other autoencoders implemented in a multi-layer neural network architecture using an unsupervised learning algorithm.
[0088] In one embodiment, the image segmentation network 157 may be an object detection algorithm (such as a U-Net or Mask RCNN algorithm) for detecting objects in an image and marking the object contour in the image and generating a mask image.
[0089] The operation module 159 is used to execute other neural network operations other than the auto encoder 155 and the image segmentation network 157 .
[0090] In one embodiment, the image defect detection model 150 includes an automatic encoder 155, an image segmentation network 157, and a computing module 159. In one embodiment, the automatic encoder 155, the image segmentation network 157, and the computing module 159 are software models, which are implemented by a plurality of program codes, so that the processor 120 executes the operations of the automatic encoder 155, the image segmentation network 157, and the computing module 159 after loading the plurality of program codes.
[0091] In one embodiment, the image defect detection model 150 may be a deep learning algorithm, such as a convolutional neural network (CNN), a recurrent neural network (RNN), a generative adversarial network (GAN), a multilayer perceptron (MLP), a deep Boltzmann machine (DBM), or a long short-term memory network (LSTM).
[0092] In one embodiment, the processor may be, but is not limited to, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a central processing unit (CPU), a system on chip (SoC), a field programmable gate array (FPGA), a network processor chip, or a combination of the above elements.
[0093] In one embodiment, the storage medium may be, but is not limited to, a random access memory (RAM), a flash memory (Flash memory), a read-only memory (ROM), a hard disk drive (HDD), a solid state drive (SSD), an optical storage device, or a combination of the above elements.
[0094] In one embodiment, the electronic device 10 may be, but is not limited to, a tablet computer, a notebook computer, a personal computer, a desktop computer, a host computer system, a computer workstation, an imaging computer, or other electronic device having a processor and / or a storage device that can execute applications installed thereon and / or in the cloud or can execute instructions stored locally or in the cloud.
[0095] In one embodiment, the image defect detection model 150 can be applied to, for example, factory production line product inspection, wafer inspection, printed circuit board inspection, public scene anomaly detection, or any scenario that uses machine vision to achieve purposeful inspection.
[0096] Please refer to Figure 2 , which is a flow chart of a method for establishing an image defect detection model according to an embodiment of the present application. The method for establishing an image defect detection model can be Figure 1 The processor 120 of the electronic device 10 loads a plurality of program codes in the storage medium 110 to execute a plurality of operations to achieve the establishment and training of the image defect detection model 150 .
[0097] In one embodiment, the user may obtain a plurality of flawless sample images in advance for the purpose of detection. The flawless sample images refer to images whose contents are normal or correct based on the scene or application. For example, in the application of printed circuit boards, the flawless sample images refer to printed circuit board images on which the circuit elements are correct, the positions of the circuit elements are correct, and the wiring is correct. In another embodiment, the flawless sample images may be images used for training and learning.
[0098] The flawless sample images may be stored in advance in an image set (not shown) of the storage medium 110 .
[0099] In step S210 , the operation module 159 generates a first training image using the non-defective sample image and the defective sample image.
[0100] In one embodiment, the first training image is generated by synthesizing a flawless sample image and a flawed sample image.
[0101] Please refer to Figure 3 , which is a schematic diagram of generating a first training image according to an embodiment of the present application.
[0102] like Figure 3 As shown, the flawless sample image 302 and the flawed sample image 304 are synthesized to generate a first training image 310. The first training image 310 includes image features that are the same or similar to those of the flawless sample image 302 and the flawed sample image 304.
[0103] In one embodiment, the flawed sample image 304 includes a noise sample image and a defect sample image.
[0104] In one embodiment, the operation module 159 generates image noise using a random function, and performs affine transformation, perspective transformation, color transformation or a combination thereof to process the image noise and then synthesizes the image noise with the flawless sample image to generate a noisy sample image.
[0105] In one embodiment, the operation module 159 performs affine transformation, perspective transformation, color transformation or a combination thereof to process any image that can be regarded as a defect and then synthesizes the image with the non-defective sample image to generate a defective sample image.
[0106] In one embodiment, the image set stored in the storage medium 110 includes a plurality of flawless sample images and a plurality of flawless sample images. There is a ratio between the number of flawless sample images and the number of flawless sample images. For example, the ratio is 1 to 100, that is, the operation module 159 takes the same flawless sample image and 100 flawless sample images in the image set to generate 100 first training images respectively. This ratio is used to adjust the difference between the first training image and the flawless sample image.
[0107] In step S220 , the operation module 159 inputs the first training image to the auto encoder 155 , and the auto encoder 155 outputs the reconstructed image and calculates the first reconstruction error.
[0108] Please refer to Figure 4 , which is a schematic diagram of an output reconstructed image according to an embodiment of the present application.
[0109] like Figure 4 As shown, the input image of the auto encoder 155 is the first training image 310. The encoder (not shown) and the decoder (not shown) of the auto encoder 155 perform image processing on the first training image 310 and output a reconstructed image 312 of the flawless sample image 302.
[0110] In one embodiment, the auto encoder 155 has the ability to process defects in the first training image 310 (e.g., filter out defects 314) and convert the first training image 310 into an image that is the same as or similar to the defect-free sample image 302, namely, the reconstructed image 312. The reconstructed image 312 is the output image of the auto encoder 155.
[0111] Continuing with the above embodiment, in step S220, the electronic device 10 further calculates a first reconstruction error. In one embodiment, the electronic device 10 inputs the pixel values of the flawless sample image 302 and the pixel values of the reconstructed image 312 into a first error function to calculate the first reconstruction error.
[0112] In one embodiment, the first error function is: Where X i is the pixel value of the flawless sample image 302, Y i is the pixel value of the reconstructed image 312 .
[0113] Please refer to Figure 5 , which is a schematic diagram of calculating a first reconstruction error according to an embodiment of the present application.
[0114] The electronic device 10 calculates the square sum of the difference between the pixel value of the flawless sample image 302 and the pixel value of the reconstructed image 312 for each pixel using the first error function E1, and then calculates the average of the square sums to obtain the first reconstruction error K1. The first reconstruction error K1 presents the overall error value or difference between the flawless sample image 302 and the reconstructed image 312, reflecting the degree to which the automatic encoder 155 currently processes (eliminates) noise.
[0115] The first reconstruction error K1 may be fed back to the auto-encoder 155 as one of the elements for adjusting the parameters of the auto-encoder 155 to optimize the ability of the auto-encoder 155 to handle noise.
[0116] In step S230 , the calculation module 159 calculates the difference between the first training image and the reconstructed image to generate a reconstructed error image.
[0117] Please refer to Figure 6 , which is a schematic diagram of generating a reconstructed error image according to an embodiment of the present application.
[0118] like Figure 6 As shown, the difference between the reconstructed image 312 and the first training image 310 is a reconstructed error image 316 .
[0119] In one embodiment, the first training image 310 includes a defect 314, and the reconstructed error image 316 includes a defect 318. The defect 314 is a defect feature before image processing, and the defect 318 is a defect feature after image processing. The defect 318 is similar to the defect 314, so the reconstructed error image 316 can be used as another defect sample image similar to the defect sample image 304 (e.g., Figure 3 ) defective image.
[0120] In step S240 , the operation module 159 generates a second training image using the first training image 310 , the reconstructed image 312 , and the reconstructed error image 316 .
[0121] Please refer to Figure 7 , which is a schematic diagram of generating a second training image according to an embodiment of the present application.
[0122] like Figure 7 As shown, the operation module 159 combines or superimposes the first training image 310 , the reconstructed image 312 , and the reconstructed error image 316 to generate a multi-dimensional image (ie, the second training image 320 ).
[0123] The second training image 320 is generated by merging or superimposing the first training image 310 , the reconstructed image 312 , and the reconstructed error image 316 . Therefore, the image features of the second training image 320 include a defect 328 . The defect 328 is associated with the defect 314 in the first training image 310 and the defect 318 in the reconstructed error image 316 .
[0124] In step S250 , the operation module 159 inputs the second training image to the image segmentation network 157 , the image segmentation network 157 generates a masked image, and the operation module 159 calculates a second reconstruction error.
[0125] Please refer to Figure 8 , which is a schematic diagram of generating a defect masking image according to an embodiment of the present application.
[0126] like Figure 8 As shown, the second training image 320 is input to the image segmentation network 157. The image segmentation network 157 classifies the pixels of the second training image 320 as defects or background according to the pixels and image features of the second training image 320.
[0127] In one embodiment, the image segmentation network 157 generates a defect mask image 334 after classifying all pixels of the second training image 320. The defect mask image 334 includes a retained area 338 and a filtered area 336. The retained area 338 corresponds to the defect 328 of the second training image 320. The filtered area 336 corresponds to the background of the second training image 320.
[0128] In one embodiment, the operation module 159 uses the defect mask image 334 to perform image processing, filter out the image block corresponding to the filter area 336 of the processed image, and leave the image block corresponding to the reserved area 338. In this embodiment, the left image block is used to indicate the defect block of the processed image.
[0129] In step S250 , the operation module 159 further calculates the second reconstruction error. In one embodiment, the operation module 159 inputs the defect sample image 304 and the defect mask image 334 into the second error function E2 to calculate the second reconstruction error K2.
[0130] In one embodiment, the second error function is: E2 = -α t (1-p t ) γ log(p t ), where p t In order to use the defect mask image 334 to correctly determine the probability of the processed image being a defect image or a non-defect image, α tand γ are coefficients of the second error function E2, which are hyperparameters used for subsequent feedback to adjust the weight values of the image segmentation network 157.
[0131] Please refer to Fig. 9 , which is a schematic diagram of calculating the second reconstruction error according to an embodiment of the present application.
[0132] The operation module 159 calculates the difference between the defect sample image 304 and the defect mask image 334 using the second error function E2 to obtain a second reconstruction error K2 that represents the overall error value or difference between the defect sample image 304 and the defect mask image 334 .
[0133] In step S260 , a total reconstruction error is calculated according to the first reconstruction error K1 and the second reconstruction error K2 .
[0134] In one embodiment, the operation module 159 calculates a weighted formula to obtain a total reconstruction error.
[0135] In one embodiment, the weighting formula is: ES=w1E1+w2E2, wherein w1 and w2 are weighting coefficients (eg, a ratio whose total is 100%).
[0136] Please refer to Fig.10 , which is a schematic diagram of calculating the total reconstruction error according to an embodiment of the present application.
[0137] As described above, the first reconstruction error K1 calculated in step S220 and the second reconstruction error K2 calculated in step S250 are respectively input into the weighting formula ES. The operation module 159 uses the weighting formula ES to proportionally distribute the first reconstruction error K1 and the second reconstruction error K2 with weighting coefficients w1 and w2, respectively, and calculates the total reconstruction error KS after summing them up.
[0138] In step S270 , the operation module 159 optimizes the parameters of the auto-encoder 155 and the image segmentation network 157 according to the total reconstruction error.
[0139] In one embodiment, the total reconstruction error KS may be fed back to the auto-encoder 155 and the image segmentation network 157 via back-propagation for parameter optimization.
[0140] After the parameters of the autoencoder 155 and the image segmentation network 157 are adjusted, the electronic device 10 once again executes the aforementioned steps S210 to S270 to continuously train the autoencoder 155 and the image segmentation network 157 and adjust the parameters of the autoencoder 155 and the image segmentation network 157 to improve the accuracy of the autoencoder 155 and the image segmentation network 157.
[0141] In step S280 , the electronic device 10 completes the image training and parameter optimization of the auto-encoder 155 and the image segmentation network 157 .
[0142] In one embodiment, the electronic device 10 determines whether the total reconstruction error KS is less than a threshold value each time. If the total reconstruction error KS is less than the threshold value, it is determined that the establishment and parameter optimization of the image defect detection model 150 have been completed.
[0143] After the image defect detection model 150 is established and optimized, the image defect detection model 150 can be applied to detect whether the image to be detected is a defective image.
[0144] Please refer to Fig.11 , which is a flow chart of a defect image detection method according to an embodiment of the present application. The defect image detection method can be Figure 1 The processor 120 of the electronic device 10 loads a plurality of program codes in the storage medium 110 to execute a plurality of operations to realize the detection of defective images.
[0145] In step S1110 , the electronic device 10 acquires an image to be detected.
[0146] In one embodiment, the image to be inspected may be a product image of a factory production line, a wafer image, a printed circuit board image, a public scene image, or an image taken from any scene.
[0147] In one embodiment, the electronic device 10 includes an image capturing module (such as a camera) for photographing the object to be detected to obtain an image to be detected.
[0148] In step S1120 , the operation module 159 inputs the image to be detected to the automatic encoder 155 , and the automatic encoder 155 outputs the detected and reconstructed image.
[0149] In one embodiment, the automatic encoder 155 performs encoder operation and decoder operation on the image to be detected and outputs a detected reconstructed image of the image to be detected.
[0150] In this embodiment, the automatic encoder 155 has been optimized, so that the defects of the image to be detected can be filtered out in this step. In other words, the detected reconstructed image is a more correct image than the image to be detected.
[0151] In step S1130 , the calculation module 159 calculates the difference between the image to be detected and the detected reconstructed image to generate a detected reconstructed error image.
[0152] The detection and reconstruction error image is a difference image between the image to be detected and the detection and reconstruction image. In other words, the purpose of this step is to extract the image block determined to be a defect of the image to be detected and represent it with the detection and reconstruction error image.
[0153] In step S1140 , the operation module 159 generates a test image according to the image to be detected, the detected and reconstructed image, and the detected and reconstructed error image.
[0154] In one embodiment, the operation module 159 combines or superimposes the image to be detected, the detection reconstructed image, and the detection reconstructed error image to generate a multi-dimensional image (ie, a test image).
[0155] Since there are objectively defective blocks in the image to be detected, the detected reconstructed image is an image with defects filtered out (the image without defects considered by the automatic encoder 155), and the detected reconstructed error image is an image of the defective blocks of the image to be detected (the image considered by the automatic encoder 155 to be defective), the test image generated by merging or superimposing these three images can highlight the intersection of the objectively existing defects and the defects determined by the automatic encoder 155.
[0156] In step S1150 , the operation module 159 inputs the test image to the image segmentation network 157 , and the image segmentation network 157 generates a defect masking test image.
[0157] In one embodiment, the test image is input to the image segmentation network 157. The image segmentation network 157 classifies a plurality of pixel blocks of the test image into defect areas or background areas according to the pixels and image features of the test image.
[0158] In one embodiment, the image segmentation network 157 generates a defect masking test image after classifying all pixel blocks of the test image. The defect masking test image includes a defect area and a background area. The defect area corresponds to the image block of the defect in the test image. The background area corresponds to the image block of the non-defect (background) of the test image.
[0159] In step S1160, the operation module 159 determines whether the indicated area of the defect masking test image is greater than a threshold value. If the indicated area is equal to or less than the threshold value, step S1170 is executed. If the indicated area is greater than the threshold value, step S1180 is executed.
[0160] In one embodiment, the indicated area is the area of the defect region of the defect masking test image.
[0161] In step S1170 , since the indicated area is less than or equal to the threshold, it means that the defect amount of the image to be tested does not exceed the tolerance value, so the operation module 159 determines that the image to be tested is a flawless image.
[0162] In step S1180 , since the indicated area is larger than the threshold, it means that the defect amount of the image to be tested exceeds the tolerance value, so the operation module 159 determines that the image to be tested is a defective image.
[0163] In one embodiment, the electronic device 10 will issue a notification to the user (eg, a notification that the appearance of an object is abnormal) so that the user can perform corresponding actions.
[0164] In summary, the present application proposes to establish an image defect detection model and use the image defect detection model to detect defective images, and to improve the diversity of the training model by adding noise samples and bad samples to the detection model. In the detection stage, the detection results include the position and size of the defect in the image, and a notification is issued based on the detection results as to whether the image to be detected has defects (that is, whether the photographed object to be detected has abnormalities). The parameter optimization of the model is improved by an image processing operation composed of multiple steps. The image defect detection model established according to the method of the present application can improve the accuracy of detecting defective images.
[0165] The above description is only a preferred specific example of the present application, and does not limit the patent scope of the present application. Therefore, all equivalent changes made using the contents of the present application are similarly included in the scope of the present application and are hereby stated.
Claims
1. A method for establishing an image defect detection model, characterized in that: The image defect detection model is used to detect whether the image is abnormal. The method includes: a) generating a first training image using a flawless sample image and a flawed sample image; b) inputting the first training image into an automatic encoder to output a reconstructed image and calculating a first reconstruction error; c) calculating the difference between the first training image and the reconstructed image to generate a reconstructed error image; d) generating a second training image using the first training image, the reconstructed image and the reconstructed error image; e) inputting the second training image into an image segmentation network to generate a masked image and calculate a second reconstruction error; f) calculating a total reconstruction error according to the first reconstruction error and the second reconstruction error; g) performing optimization of parameters of the autoencoder and the image segmentation network according to the total reconstruction error; and h) Repeating steps a) to g) to complete the establishment and parameter optimization of the image defect detection model, wherein the image defect detection model includes the autoencoder and the image segmentation network.
2. The method according to claim 1, characterized in that Prior to step a), the method comprises: Generate a noise sample image using a random function and synthesize the noise sample image and the flawless sample image to obtain the flawed sample image; or The defective sample image is transformed and the transformed defective sample image and the flawless sample image are synthesized to obtain the defective sample image.
3. The method according to claim 1, characterized in that Step b) comprises: The pixel value of the flawless sample image and the pixel value of the reconstructed image are input into a first error function for calculation to obtain the first reconstruction error.
4. The method according to claim 1, characterized in that: Step d) comprises: The first training image, the reconstructed image and the reconstructed error image are combined to generate the second training image.
5. The method according to claim 1, characterized in that Step e) comprises: Classifying pixels of the second training image by the image segmentation network to obtain a retained area corresponding to defects and a filtered area corresponding to the background; and The mask image including the retained area and the filtered area is output.
6. The method according to claim 5, characterized in that Step e) comprises: The pixel value of the defect sample image and the pixel value of the mask image are input into the second error function for calculation to obtain the second reconstruction error.
7. The method according to claim 6, characterized in that Step f) comprises: The first reconstruction error and the second reconstruction error are weightedly calculated using the first weight and the second weight respectively to obtain the total reconstruction error.
8. The method according to claim 7, characterized in that Step g) comprises: The parameters of the automatic encoder and the image segmentation network are adjusted according to the parameter value of the second error function, the first weight and the second weight.
9. The method according to claim 8, characterized in that Step h) comprises: Repeat steps a) to g) until the total reconstruction error is less than a threshold value to complete the establishment of the image defect detection model and parameter optimization.
10. An electronic device for establishing an image defect detection model, characterized in that: include: A storage medium configured to store the image defect detection model; as well as A processor, coupled to the storage medium, is configured to execute the method according to any one of claims 1 to 9.
11. A method for detecting defective images using the image defect detection model established by the method according to any one of claims 1 to 9, characterized in that: include: a1) Acquire the image to be detected; b1) inputting the image to be detected into the automatic encoder and outputting the detected reconstructed image; c1) calculating the difference between the image to be detected and the detected reconstructed image to generate a detected reconstructed error image; d1) generating a test image according to the image to be detected, the detected reconstructed image and the detected reconstructed error image; e1) inputting the test image into the image segmentation network to generate a defect masking test image; and f1) judging whether the image to be inspected is a defective image according to the size of the indicated area of the defect shielding test image.
12. The method according to claim 11, characterized in that Step d1) comprises: The test image is generated by combining the image to be detected, the detected reconstructed image and the detected reconstructed error image.
13. The method according to claim 11, characterized in that Step e1) comprises: Classifying the pixels of the test image by the image segmentation network to obtain defect areas and background areas; and The defect shielding test image including the defect area and the background area is output.
14. The method according to claim 12, characterized in that Step f1) comprises: Calculating the area of the defect region of the defect masking test image to obtain the indicated area; If the indicated area is larger than the threshold, determining that the image to be detected is the defective image; and If the indicated area is smaller than or equal to the threshold, the image to be detected is determined to be a flawless image.
15. An electronic device for detecting defective images, characterized in that: include: A storage medium configured to store the defect image detection model; as well as A processor, coupled to the storage medium, is configured to execute the method according to claim 11.