Unsupervised electronic component surface defect detection method based on defect repair

Abstract

The present invention discloses an unsupervised electronic component surface defect detection method based on defect repair, comprising: 1) collecting electronic component images; 2) dividing and annotating the electronic component images into data sets; 3) constructing defective electronic component images using a random defect generation method and a migration defect generation method for normal electronic component images in a training set; 4) training an improved DRAEM model using the generated defective electronic component images; 5) testing the trained improved DRAEM model on a test set, merging intermediate variables generated during the process, completing the detection results, and visually outputting them. The present invention can be effectively used for electronic component surface defect detection under conditions where only normal samples are used for training, achieving detection model training under conditions where there are no defective samples, reducing model development costs, optimizing the problem of small defect samples in defect detection tasks, and improving the defect detection accuracy of electronic components.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial electronic component defect detection, and in particular to an unsupervised electronic component surface defect detection method based on defect repair. Background Art

[0002] Surface defect detection is an extremely important part of industrial production and is widely used in electronics, textiles, steel, glass, wood, parts, tiles, steel plates, and paper. Surface defect detection technologies primarily include manual visual inspection and machine vision. Manual visual inspection relies on human visual inspection to find defects. This method is costly, inefficient, has limited resolution, and is highly subjective, making it difficult to meet the demands of large-scale and high-speed product quality inspections. Machine vision inspection utilizes optical imaging equipment, image processing, and machine learning to distinguish between good and defective products. It offers advantages such as non-contact, high accuracy, high efficiency, and strong real-time capabilities. However, it also presents many difficulties, such as a lack of defect samples, an imbalance in defect types, slow detection speeds, and the presence of small and weak defects. This makes it difficult to simultaneously meet the practical requirements of high precision and high speed. Consequently, numerous problems remain to be solved in its application. Summary of the Invention

[0003] The purpose of the present invention is to overcome the shortcomings and disadvantages of the existing technology and provide an unsupervised electronic component surface defect detection method based on defect repair, which can be effectively used for electronic component surface defect detection under conditions with only normal sample training, and realize detection model training under zero defect sample conditions, effectively reducing the model development cost, optimizing the problem of small defect samples in defect detection tasks, and improving the defect detection accuracy of electronic components.

[0004] To achieve the above objectives, the present invention provides a technical solution: an unsupervised electronic component surface defect detection method based on defect repair, comprising the following steps:

[0005] 1) Collect images of electronic components;

[0006] 2) performing data set division and data labeling on the electronic component images collected in step 1); wherein data set division refers to dividing the collected images into a training set and a test set, wherein the training set contains images of normal electronic components, and the test set contains images of normal electronic components and images of defective electronic components; and data labeling refers to identifying the type of electronic component images and the location of defects;

[0007] 3) For the normal electronic component images in the training set in step 2), defective electronic component images are constructed using the random defect generation method and the transfer defect generation method. The random defect generation method draws on the pseudo-defect generation algorithm in the DRAEM model and completes defect generation by cutting and pasting random blocks from the COCO dataset; the transfer defect generation method generates defects by transferring defect features from the public dataset.

[0008] 4) Using the defective electronic component images generated in step 3), an improved DRAEM model is trained. The original DRAEM model includes a defect repair module responsible for repairing defects and a defect identification module responsible for determining defect locations. In the defect repair module, the improved DRAEM model uses vector quantization technology for defect repair, resulting in greater robustness. In the defect identification module, the improved DRAEM model uses a multi-scale feature comparison method, resulting in higher positioning accuracy.

[0009] 5) Use the trained improved DRAEM model to test on the test set, merge the intermediate variables generated in the process, complete the detection results and visualize the output.

[0010] Further, in step 1), the specific implementation method is as follows:

[0011] First, the electronic components to be inspected are placed on a circular vibrating disk. The posture of the electronic components is adjusted through a fixed vibration frequency. The guiding effect of the slide rail allows the electronic components to enter the inspection process with a fixed posture. The disk is then rotated to drive the electronic components to different cameras, thereby completing the acquisition of the electronic component image.

[0012] Further, in step 2), the specific implementation method is as follows:

[0013] For the electronic component images collected in step 1), first, it is necessary to divide some normal electronic component images into a training set, and then divide the remaining electronic component images into a test set; in addition, for the defective electronic component images in the test set, use the EISEG tool to perform pixel-level annotation of the defect location.

[0014] Further, in step 3), the specific implementation method is as follows:

[0015] For the normal electronic component image I in the training set divided in step 2) N , using random defect generation method and migration defect generation method to construct defective electronic component image I DAmong them, the random defect generation method randomly cuts random texture blocks in the COCO dataset and pastes them into the normal electronic component image to form a defective electronic component image; the migration defect generation method first collects public defect features on different public datasets to form a feature database, and then selects the corresponding defects from the feature database according to the specified defect category, and finally fuses them into the normal electronic component image through Poisson editing to form a defective electronic component image.

[0016] Further, in step 4), the specific implementation method is as follows:

[0017] The defective electronic component images generated in step 3) are used to train the improved DRAEM model. The improved DRAEM model is optimized for two modules:

[0018] The defect repair module in the improved DRAEM model uses vector quantization technology to quantize and record normal texture features, and uses normal texture features to replace defect texture features during the test phase to achieve defect repair; the defect repair module consists of an encoder, a texture dictionary D and a decoder; the encoder is used to extract the defective electronic component image I D The function of the texture dictionary D is to convert F into a quantized feature F' by nearest neighbor search; the function of the encoder is to map the feature F' into a repaired image I with the same scale as the input image R During the training process, the loss constraints of the defect repair module include the encoding loss L enc ode, codebook loss L codebook and restoration loss L repair , defined as follows:

[0019] L encode =MSE(F,Sg[D])

[0020] L codebook =MSE(Sg[F],D)

[0021] L reapir =MSE(I D ,I R )+SSIM(I D ,I R )

[0022] Where Sg stands for Stop Gradient. After the Sg operation, the gradient of the specified variable will not be updated forward. MSE stands for mean square error loss. SSIM stands for structural similarity loss.

[0023] The defect identification module in the improved DRAEM model consists of a characterization module and a multi-scale feature similarity identification module; wherein the characterization module consists of the first three residual module layers Layer1, Layer2 and Layer3 of Resnet18, which can identify defective electronic component images I D and the repaired image I R Characterize and get F D1 、F D2 、F D3 、F R1 、F R2 and F R3 , where F Di and F Ri They are defective electronic component images I D and the repaired image I R In Layer i The output representation at i = 1, 2, 3; the multi-scale feature similarity identification module is composed of multiple feature similarity identification modules FSDM. For the same scale feature F Di and F Ri , the feature similarity identification module FSDM is used to locate defects. First, the receptive field of the model is expanded by convolution operations with different void rates, and the features are recombined to obtain the recombined features F Di ' and F Ri '; Then, the positioning result of the corresponding scale is obtained through splicing and 1×1 convolution operation, and the segmentation result consistent with the input sample size is obtained through upsampling operation, that is, the defect positioning result P i ; For the defect localization results P1, P2 and P3 obtained by the three scale features, the final defect definition result P4 is obtained by splicing and fusion of 1×1 convolution operation; During the training process, the identification loss L discriminate Constraints, L discriminate Use deep supervision technology to perform imbalance loss BCE constraints on all output defect localization results P1, P2, P3 and P4, as shown below:

[0024] L discriminate =BCE(G,P1)+BCE(G,P2)+BCE(G,P3)+BCE(G,P4)

[0025]

[0026] In the formula, G represents the mask of the defect area, P represents the predicted defect area, N represents the number of samples in a batch, α represents the imbalance rate, and G jk Represents the pixel value of G at pixel point (j, k), P jk It represents the probability that P is a defect at pixel point (j, k), where j and k represent the horizontal and vertical coordinates of the pixel point, respectively.

[0027] Further, in step 5), the intermediate variables include the original image I, the repaired image I R and defect location results P1, P2, P3, and P4.

[0028] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0029] 1. The model of the present invention has stronger adaptability and detection performance, and can better repair and locate defects under high-noise conditions. The present invention not only performs well on low-resolution simple texture surfaces, but also performs well on high-resolution complex texture images.

[0030] 2. The present invention has shorter development time and lower development costs. The present invention only requires normal samples during model training, eliminating the need for tedious work such as data labeling, effectively reducing development costs and time.

[0031] 3. This invention effectively solves the problems of small samples and sample imbalance in defect detection tasks. During model training, this invention only requires normal samples, not defective samples. Therefore, it is not affected by imbalanced defect categories and a small number of defective samples.

[0032] 4. This invention has a wide range of industrial applications. It can not only perform pixel-level defect detection, but can also be used for data annotation and provide prior knowledge for supervised defect detection tasks, thus possessing high application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 This is the overall framework diagram of the present invention; in the figure, Repair Module represents the defect repair module, Discriminate Module represents the defect identification module, FSDM represents the feature similarity identification module, H represents the height of the input image, W represents the width of the input image, and Multi-Scale Feature Similarity Discriminate Module represents the multi-scale feature similarity identification module.

[0034] Figure 2 Figure 2 is the structure diagram of the feature similarity discrimination module FSDM. In the figure, Feature Extraction represents feature extraction, concate represents image splicing, upsampling represents image sampling, and FSDM: Feature Similarity Discriminate Module represents the feature similarity discrimination module.

[0035] Figure 3 Schematic diagram of the detection results of the present invention. DETAILED DESCRIPTION

[0036] The present invention will be further described below with reference to specific embodiments.

[0037] like Figures 1 to 3 As shown, this embodiment discloses an unsupervised electronic component surface defect detection method based on defect repair, the details of which are as follows:

[0038] 1) Collect images of electronic components. The specific implementation method is as follows:

[0039] First, the electronic components to be inspected are placed on a circular vibrating disk. The posture of the electronic components is adjusted through a fixed vibration frequency. The guiding effect of the slide rail allows the electronic components to enter the inspection process with a fixed posture. The disk is then rotated to drive the electronic components to different cameras, thereby completing the acquisition of the electronic component image.

[0040] 2) Perform data set division and data annotation on the electronic component images collected in step 1); wherein, data set division refers to dividing the collected images into a training set and a test set. First, some normal electronic component images need to be divided into the training set, and then the remaining electronic component images need to be divided into the test set. The training set contains normal electronic component images, and the test set contains normal electronic component images and defective electronic component images; data annotation refers to identifying the type of electronic component image and the location of the defect. For the defective electronic component images in the test set, use the EISEG tool to perform pixel-level annotation of the defect location.

[0041] 3) For the normal electronic component image I in the training set divided in step 2) N , using random defect generation method and migration defect generation method to construct defective electronic component image I D Among them, the random defect generation method randomly cuts random texture blocks in the COCO dataset and pastes them into the normal electronic component image to form a defective electronic component image; the migration defect generation method first collects public defect features on different public datasets to form a feature database, and then selects the corresponding defects from the feature database according to the specified defect category, and finally fuses them into the normal electronic component image through Poisson editing to form a defective electronic component image.

[0042] 4) Using the defective electronic component images generated in step 3), an improved DRAEM model is trained. The original DRAEM model includes a defect repair module responsible for repairing defects and a defect identification module responsible for determining defect locations. In the defect repair module, the improved DRAEM model uses vector quantization technology for defect repair, resulting in greater robustness. In the defect identification module, the improved DRAEM model uses a multi-scale feature comparison method, resulting in higher positioning accuracy.

[0043] The improved DRAEM model is optimized for two modules:

[0044] The defect repair module in the improved DRAEM model uses vector quantization technology to quantize and record normal texture features, and uses normal texture features to replace defect texture features during the test phase to achieve defect repair; the defect repair module consists of an encoder, a texture dictionary D and a decoder; the encoder is used to extract the defective electronic component image I D The function of the texture dictionary D is to convert F into a quantized feature F' by nearest neighbor search; the function of the encoder is to map the feature F' into a repaired image I with the same scale as the input image R During the training process, the loss constraints of the defect repair module include the encoding loss L encode , codebook loss L codebook and restoration loss L repair , defined as follows:

[0045] L encode =MSE(F,Sg[D])

[0046] L codebook =MSE(Sg[F],D)

[0047] L reapir =MSE(I D ,I R )+SSIM(I D ,I R )

[0048] Where Sg stands for Stop Gradient. After the Sg operation, the gradient of the specified variable will not be updated forward. MSE stands for mean square error loss. SSIM stands for structural similarity loss.

[0049] The defect identification module in the improved DRAEM model consists of a characterization module and a multi-scale feature similarity identification module FSDM; wherein the characterization module consists of the first three residual module layers Layer1, Layer2 and Layer3 of Resnet18, which can identify defective electronic component images I D and the repaired image I R Characterize and get F D1 、F D2 、F D3 、F R1 、F R2 and F R3 , where F Di and F Ri They are defective electronic component images I D and the repaired image IR In Layer i The output representation at i = 1, 2, 3; the multi-scale feature similarity identification module is composed of multiple feature similarity identification modules FSDM. For the same scale feature F Di and F Ri , the feature similarity identification module FSDM is used to locate defects. First, the receptive field of the model is expanded by convolution operations with different void rates, and the features are recombined to obtain the recombined features F Di ' and F Ri '; Then, the positioning result of the corresponding scale is obtained through splicing and 1×1 convolution operation, and the segmentation result consistent with the input sample size is obtained through upsampling operation, that is, the defect positioning result P i ; For the defect localization results P1, P2 and P3 obtained by the three scale features, the final defect definition result P4 is obtained by splicing and fusion of 1×1 convolution operation; During the training process, the identification loss L discriminate Constraints, L discriminate Use deep supervision technology to perform imbalance loss BCE constraints on all output defect localization results P1, P2, P3 and P4, as shown below:

[0050] L discriminate =BCE(G,P1)+BCE(G,P2)+BCE(G,P3)+BCE(G,P4)

[0051]

[0052] In the formula, G represents the mask of the defect area, P represents the predicted defect area, N represents the number of samples in a batch, α represents the imbalance rate, and G jk Represents the pixel value of G at pixel point (j, k), P jk It represents the probability that P is a defect at pixel point (j, k), where j and k represent the horizontal and vertical coordinates of the pixel point, respectively.

[0053] 5) Use the trained improved DRAEM model to test on the test set, merge the intermediate variables generated in the process, complete the detection results and visualize the output; the intermediate variables include the original image I, the repaired image I R and defect location results P1, P2, P3, and P4.

[0054] The above-described embodiments are only preferred embodiments of the present invention and are not intended to limit the scope of implementation of the present invention. Therefore, any changes made based on the shape and principle of the present invention should be included in the scope of protection of the present invention.

Claims

1. An unsupervised electronic component surface defect detection method based on defect repair, characterized in that: The following steps are involved: 1) Collect images of electronic components; 2) Dataset division and data labeling are performed on the electronic component images collected in step 1); wherein dataset division refers to dividing the collected images into a training set and a test set, where the training set contains images of normal electronic components, and the test set contains images of normal electronic components and images of defective electronic components; data labeling refers to identifying the type of electronic component images and the location of defects; 3) For the normal electronic component images in the training set in step 2), defective electronic component images are constructed using the random defect generation method and the transfer defect generation method. The random defect generation method draws on the pseudo-defect generation algorithm in the DRAEM model and completes defect generation by cutting and pasting random blocks from the COCO dataset. The transfer defect generation method generates defects by transferring defect features from the public dataset. 4) Using the defective electronic component images generated in step 3), an improved DRAEM model is trained. The original DRAEM model consists of a defect repair module responsible for repairing defects and a defect identification module responsible for determining defect locations. In the defect repair module, the improved DRAEM model uses vector quantization technology for defect repair, resulting in greater robustness. In the defect identification module, the improved DRAEM model uses a multi-scale feature comparison method, resulting in higher positioning accuracy. The defect repair module uses vector quantization technology to quantize and record normal texture features, and uses normal texture features to replace defect texture features during the test phase to repair defects. The defect repair module consists of an encoder, a texture dictionary, and a and decoder; the encoder is used to extract defective electronic component images Features in ; Texture dictionary The role of is to use the nearest neighbor search method to Convert to quantitative features ; The function of the encoder is to transform the features Mapped to a restored image of the same scale as the input image ; The defect identification module consists of a representation module and a multi-scale feature similarity identification module; the representation module consists of the first three residual module layers of Resnet18 、 and Composition, which can image defective electronic components and repair the image To characterize, we can obtain 、 、 、 、 and ,in and Defective electronic component images and repair the image exist The output representation at ; The multi-scale feature similarity identification module is composed of multiple feature similarity identification modules FSDM. and , using the feature similarity identification module FSDM to locate defects, firstly, the receptive field of the model is expanded by convolution operations with different void rates, and the features are recombined to obtain recombined features and ; Then, the positioning result of the corresponding scale is obtained through splicing and 1×1 convolution operation, and the segmentation result consistent with the input sample size is obtained through upsampling operation, that is, the defect positioning result ; Defect location results obtained for three scale features 、 and The final defect definition result is obtained by splicing and 1×1 convolution operation fusion ; 5) Use the trained improved DRAEM model to test on the test set, merge the intermediate variables generated in the process, complete the detection results and visualize the output.

2. The unsupervised electronic component surface defect detection method based on defect repair according to claim 1 is characterized in that: In step 1), the specific implementation method is as follows: First, the electronic components to be inspected are placed on a circular vibrating disk. The posture of the electronic components is adjusted through a fixed vibration frequency. The guiding effect of the slide rail allows the electronic components to enter the inspection process with a fixed posture. The disk is then rotated to drive the electronic components to different cameras, thereby completing the acquisition of the electronic component image.

3. The unsupervised electronic component surface defect detection method based on defect repair according to claim 2 is characterized in that: In step 2), the specific implementation method is as follows: For the electronic component images collected in step 1), first, it is necessary to divide some normal electronic component images into a training set, and then divide the remaining electronic component images into a test set; in addition, for the defective electronic component images in the test set, use the EISEG tool to perform pixel-level annotation of the defect locations.

4. The unsupervised electronic component surface defect detection method based on defect repair according to claim 3 is characterized in that: In step 3), the specific implementation method is as follows: For the normal electronic component images in the training set divided in step 2) , using random defect generation method and migration defect generation method to construct defective electronic component images Among them, the random defect generation method randomly cuts random texture blocks in the COCO dataset and pastes them into the normal electronic component image to form a defective electronic component image; the migration defect generation method first collects public defect features on different public datasets to form a feature database, and then selects the corresponding defects from the feature database according to the specified defect category, and finally fuses them into the normal electronic component image through Poisson editing to form a defective electronic component image.

5. The unsupervised electronic component surface defect detection method based on defect repair according to claim 4 is characterized in that: In step 4), the defective electronic component image generated in step 3) is used to train the improved DRAEM model; During training, the loss constraints of the defect repair module include the encoding loss , codebook loss and restore losses , defined as follows: ； ； ； Where, Indicates Stop Gradient, passing The gradient of the specified variable will not be updated forward after the operation; represents the mean square error loss; represents the structural similarity loss; During the training process, the defect identification module is subjected to the identification loss Constraints, Defect localization results for all outputs using deep supervision technology 、 、 and Conduct imbalance loss Constraints are expressed as follows: ； ； Where, A mask representing the defect area, represents the predicted defect area, represents the number of samples in a batch, represents the imbalance rate, express At the pixel The pixel value at express At the pixel is the probability of defect, Represent the horizontal and vertical coordinates of the pixel respectively.

6. The unsupervised electronic component surface defect detection method based on defect repair according to claim 5 is characterized in that: In step 5), the intermediate variable contains the original image , repair the image and defect location results 、 、 and .

Images