Electronic component defect detection method and equipment

Abstract

The invention discloses an electronic component defect detection method. The method comprises the following steps: acquiring infrared images corresponding to three resolutions of an electronic component; extracting a low-resolution feature, a medium-resolution feature and a high-resolution feature of the infrared image; respectively extracting and fusing fine-grained associated information and coarse-grained associated information of the low-resolution features, the medium-resolution features and the high-resolution features; generating an adversarial sample graph through an adversarial network DCGAN; constructing a defect detection model based on deep learning, and fusing detection results to obtain an electronic component defect detection result; evaluating the defect detection result of the electronic component, and optimizing the defect detection model based on the evaluation result; according to the method, based on the defect detection model of deep learning, fine-grained and coarse-grained information of infrared images of different resolutions of electronic components is identified and detected, and the adversarial sample graph is generated by using the adversarial network DCGAN, so that the defect detection precision, accuracy and robustness are improved.

Description

Technical Field

[0001] The present invention belongs to the field of electronic components, and specifically refers to a method and device for defect detection of electronic components. Background Art

[0002] With the rapid development of electronic technology, electronic components are developing towards miniaturization, integration, and high performance. These components play a crucial role in fields such as communication, computers, medical equipment, and automotive electronics. However, due to the complexity of the manufacturing process, various defects may occur in electronic components during the production process, and these defects may affect the performance of the components and even lead to product failure. Therefore, effective defect detection of electronic components is a key step in ensuring product quality and reliability.

[0003] Chinese invention patent with publication number CN115018828A discloses a method for defect detection of electronic components. The present invention belongs to the field of image data processing and specifically relates to a method for defect detection of electronic components. An image of a PCB board to be detected is obtained, and a grayscale image of the PCB board to be detected is obtained; the grayscale image of the PCB board to be detected is divided into regions to obtain different regions, the grayscale histogram of each region is obtained, and the maximum inter-class variance algorithm is used to determine the segmentation threshold, and the grayscale image of the PCB board to be detected is segmented to obtain a segmented image; the threshold segmentation image of the PCB board image to be detected is matched with the standard binary image of the PCB board, and an exclusive OR operation is performed on the two matched images to obtain an exclusive OR image; the connected regions in the exclusive OR image are obtained, and the area of each connected region is counted; the area of each connected region is compared with the threshold. If there is a connected region with an area greater than or equal to the threshold, the region corresponding to the connected region has a defect; that is, the solution of the present invention can accurately perform image threshold segmentation through local threshold segmentation.

[0004] Currently, in the field of defect detection of electronic components, through image recognition and analysis for detection, a single resolution is used for defect detection. The defects shown by the same electronic component in images with different resolutions may be different, and in images with different brightnesses, the defect detection may also be different. This results in low recognition accuracy and precision of the existing image recognition methods. In addition, when using a deep learning model for training to detect defects of electronic components, in the prior art, the training defect samples are insufficient and the defect sample data lacks diversity, which may also lead to a lack of robustness in defect detection. Summary of the Invention

[0005] In order to solve the problems in the above-mentioned existing technologies, in defect detection by image recognition and analysis, a single resolution is used for defect detection. The defects presented by the same electronic component in images with different resolutions may be different, and in images with different brightness levels, the defect detection may also be different. This results in low recognition accuracy and precision of the existing image recognition methods. In addition, when using a deep learning model for training in the defect detection of electronic components, there are insufficient training defect samples and a lack of diversity in defect sample data in the existing technologies, which may also lead to problems such as a lack of robustness in defect detection. The present invention proposes a method and device for defect detection of electronic components to improve the above problems.

[0006] The specific implementation of this application is as follows:

[0007] A method for defect detection of electronic components, comprising:

[0008] S1: Obtain infrared images corresponding to three resolutions of the electronic component, where the three resolutions include low resolution, medium resolution, and high resolution;

[0009] S2: Extract low-resolution features, medium-resolution features, and high-resolution features of the infrared images;

[0010] S3: Respectively extract and fuse the fine-grained association information and coarse-grained association information of the low-resolution features, medium-resolution features, and high-resolution features to obtain low-resolution association features, medium-resolution association features, and high-resolution association features;

[0011] S4: Perform fusion processing on the pre-processed high-resolution association features and the medium-resolution association features of the image to obtain a first fusion feature map; perform fusion processing on the first fusion feature and the pre-processed low-resolution association features of the image to obtain a second fusion feature map; fuse the second fusion feature and the first fusion feature to obtain a third fusion feature map; fuse the third fusion feature and the high-resolution association features to obtain a fourth fusion feature map;

[0012] S5: Respectively generate second adversarial feature maps, third adversarial feature maps, and fourth adversarial feature maps for the second fusion feature map, third fusion feature map, and fourth fusion feature map through the adversarial network DCGAN;

[0013] S6: Construct a defect detection model based on deep learning. The defect detection model respectively detects the second fusion feature map, third fusion feature map, fourth fusion feature map, second adversarial feature map, third adversarial feature map, and fourth adversarial feature map, and fuses the detection results to obtain the defect detection result of the electronic component;

[0014] S7: Evaluate the defect detection results of the electronic components, and optimize the defect detection model based on the evaluation results.

[0015] Further, the specific implementation process of S1 is as follows:

[0016] S11: Collect the initial infrared images containing multiple electronic components. For each electronic component, three types of resolution images are collected. The three types of resolution images are specifically divided as follows:

[0017] Low resolution: w < 320, h < 320, 320 < w ≤ 640, 320 < h ≤ 640;

[0018] Medium resolution: 640 < w ≤ 1080, 640 < h ≤ 1080;

[0019] High resolution: 1080 < w, 1080 < h, where w represents the width of the image and h represents the height of the image;

[0020] S12: Segment the initial infrared images of the electronic components using the Canny edge detection algorithm according to the contour of each electronic component to obtain multiple infrared images. The initial infrared images need to collect corresponding images of three types of resolutions.

[0021] Further, S1 also includes: Feature extraction of the image brightness needs to be performed on the initial infrared images. The method of calculating the average brightness of the image using a histogram is used for feature extraction of the image brightness, which is specifically divided as follows:

[0022] First-level brightness: 0 ≤ B ≤ 50;

[0023] Second-level brightness: 50 < B ≤ 100;

[0024] Third-level brightness: 100 < B ≤ 180;

[0025] Fourth-level brightness: 180 < B ≤ 255, where B represents the average brightness value of the image.

[0026] Further, the image preprocessing in S4 includes:

[0027] Perform image preprocessing on the features obtained by fusion processing to obtain image preprocessing features. The image preprocessing includes denoising, enhancing, and binarizing the image;

[0028] Map the preprocessed image features to the features of the fusion process, including: for each position of the preprocessed image features, match the corresponding position of the features of the fusion process, and perform dot product and linear transformation operations on the local area centered on this position and the predicted preprocessed image features to generate the first fusion feature map, the second fusion feature map, the third fusion feature map, and the fourth fusion feature map.

[0029] Further, in S6, before detecting the second fusion feature map, the third fusion feature map, the fourth fusion feature map, the second adversarial feature map, the third adversarial feature map, and the fourth adversarial feature map respectively, it includes:

[0030] Perform cross-attention mechanism processing, hybrid attention mechanism processing, point-wise spatial attention mechanism processing, and effective channel attention mechanism processing on the second fusion feature map, the third fusion feature map, the fourth fusion feature map, the second adversarial feature map, the third adversarial feature map, and the fourth adversarial feature map in sequence.

[0031] Further, the specific implementation process of S6 is as follows:

[0032] S61: Collect the image data of the second fusion feature map, the third fusion feature map, the fourth fusion feature map, the second adversarial feature map, the third adversarial feature map, and the fourth adversarial feature map. Part of it is used for the test set, and part of it is used for the training set. Train a deep learning defect detection model through the training set;

[0033] S62: Divide the defect detection model into a feature extraction network Inception, a feature pyramid, and a classification and regression model. Use the feature extraction network Inception to extract fine-grained association information and coarse-grained association information, connect the feature pyramid to the feature extraction network Inception in series, perform convolution on each layer of the feature pyramid, and perform classification and regression through the classification and regression model;

[0034] S63: Use the test set to perform network iterative testing on the feature extraction network Inception and output the detection result;

[0035] S64: Determine whether the detection result includes the types of electronic component defects, and the types of electronic component defects include: scratches, cracks, bubbles, impurities, burrs, spots, holes, and oil stains.

[0036] Further, the specific construction steps of the feature extraction network Inception are as follows:

[0037] M1: Divide the feature extraction network Inception into Inception-A and Inception-B. Use Inception-A to extract fine-grained association information, which at least includes edges and textures. Use Inception-B to extract coarse-grained association information, which at least includes shapes and objects. The Inception-A includes a first convolutional layer, a second convolutional layer, and a third convolutional layer. The Inception-B includes a fourth convolutional layer, a fifth convolutional layer, and a sixth convolutional layer;

[0038] M2: Use the adversarial network DCGAN to generate second, third, and fourth adversarial feature maps for the second, third, and fourth fusion feature maps respectively. The images of the generated second, third, and fourth adversarial feature maps are used as training set samples;

[0039] M3: Add the generated training set samples to the database for storage, which are used for training set data and test set data;

[0040] The specific process of M2 is as follows: According to the original images of the second, third, and fourth fusion feature maps given in the current training, generate image samples of adversarial second, third, and fourth adversarial feature maps. The specifications of the adversarial image samples are: the size diversification feature of electronic components, the non-fixed position feature of electronic components, and the image brightness diversification feature. Input 1 given original image, and add perturbation factors through the adversarial network DCGAN to generate 100 adversarial sample images.

[0041] Further, the specific process of concatenating the feature pyramid to the feature extraction network Inception is as follows:

[0042] Sample the feature map of the first convolutional layer, that is, the fine-grained association information, and then superimpose it on the feature map of the coarse-grained association information of the fourth convolutional layer to obtain the first layer of the pyramid. Continue to perform this step for the second and third convolutional layers. Superimpose the feature maps of every three adjacent layers in the channels to obtain one layer of the pyramid. Finally, a total of three layers of feature pyramids are obtained. The specific processing process of the three layers of the pyramid is as follows:

[0043] The first layer of the pyramid is used to process low-resolution images;

[0044] The second layer of the pyramid is used to process medium-resolution images;

[0045] The third layer of the pyramid is used to process high-resolution images.

[0046] Further, in S7, the mean square error loss function is used to measure the difference between the predicted value and the true value, so as to evaluate the detection result of the electronic component defects:

[0047]

[0048] Among them, Loss represents the loss function; T represents the step size in the time series data; N represents the total number of output defect types; g t+1,i represents the i-th element in the One-Hot encoding form of the true electronic component defect at time point t + 1; represents the predicted probability for the i-th defect category at time t + 1;

[0049] The parameters of the defect detection model are adjusted by the whale optimization algorithm to minimize the loss function, so as to optimize and improve the defect detection model. The specific parameters adjusted by the whale optimization algorithm are:

[0050] P m = P - c·δ P Loss;

[0051] Among them, P m represents the parameters of the updated model; c represents the defect detection rate, which is used to control the frequency of parameter update; δ P Loss represents the gradient of the loss function with respect to the parameter P; P represents the parameters of the defect detection model.

[0052] An electronic component defect detection device, the device includes:

[0053] At least one processor, a memory and an input-output unit;

[0054] Among them, the memory is used to store computer programs, and the processor is used to call the computer programs stored in the memory to execute any one of the methods in an electronic component defect detection method.

[0055] The beneficial effects of an electronic component defect detection method of the present invention are as follows:

[0056] The present invention obtains infrared images corresponding to three resolutions of electronic components; extracts low-resolution features, medium-resolution features, and high-resolution features of the infrared images; separately extracts and fuses the fine-grained association information and coarse-grained association information of the low-resolution features, medium-resolution features, and high-resolution features to obtain low-resolution association features, medium-resolution association features, and high-resolution association features; respectively generates second adversarial feature maps, third adversarial feature maps, and fourth adversarial feature maps for the second fusion feature map, third fusion feature map, and fourth fusion feature map through the adversarial network DCGAN; constructs a defect detection model based on deep learning, fuses the detection results to obtain the defect detection result of the electronic component; evaluates the defect detection result of the electronic component, and optimizes the defect detection model based on the evaluation result; the defect detection model based on deep learning of the present invention identifies and detects the fine-grained and coarse-grained of infrared images with different resolutions of electronic components, and simultaneously uses the adversarial network DCGAN to generate adversarial sample maps, improving the defect detection accuracy, accuracy, and robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 It is a schematic flowchart of a method for detecting defects of electronic components according to the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0058] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0059] Embodiment 1

[0060] First, the technical terms involved in the embodiments of the present application are explained.

[0061] (1) Inception

[0062] The Inception network structure was proposed by the Google team. It won the championship in the ImageNet Challenge in 2014 and was named GoogLeNet. The core idea of the Inception network is to use multiple convolutional kernels of different sizes to observe the input data to capture features of different scales. This design inspiration comes from the fact that in the real world, the same object appears in different sizes due to different distances. Therefore, by using convolutional kernels of different sizes, the network can better adapt to features of different scales, thereby improving the image recognition ability;

[0063] High computational performance: By introducing sparsity and dimensionality reduction modules, the Inception network effectively reduces computational complexity and improves computational performance, giving it significant advantages when processing large-scale datasets.

[0064] Good generalization ability: The Inception module integrates information at different scales, enhancing the network's adaptability to scales. When facing inputs of different scales, the network can extract more robust feature representations, thereby improving the network's generalization ability.

[0065] Easy to expand and optimize: The modular design of the Inception network structure makes it easy to expand and optimize. By adding more Inception modules, the network performance can be further improved. At the same time, for specific tasks and datasets, the Inception module can also be customized and optimized to meet different requirements.

[0066] (2) Adversarial network DCGAN

[0067] Deep Convolutional Generative Adversarial Networks (DCGAN for short) is a model that combines deep learning and adversarial networks. It performs well in the field of image generation. The structure of DCGAN includes a Generator and a Discriminator, and the two are jointly trained through an adversarial process.

[0068] Figure 1 This is a schematic flowchart of a method for detecting defects in electronic components according to the present invention.

[0069] The method includes steps S1 - S7:

[0070] S1: Obtain infrared images corresponding to three resolutions of the electronic component, and the three resolutions include low resolution, medium resolution, and high resolution;

[0071] In this embodiment, the infrared images are obtained by deploying infrared cameras. There can be multiple infrared cameras, which are respectively arranged by the staff at different positions according to the detection requirements. The cameras can be communicatively connected to the electronic component defect detection device of this application embodiment, and send the detected images of the electronic components to be measured to the electronic component defect detection device. The infrared cameras are preset to three image acquisition modes, and the three image acquisition modes can acquire images of the electronic components to be measured corresponding to low resolution, medium resolution, and high resolution.

[0072] S2: Extract the low-resolution features, medium-resolution features, and high-resolution features of the infrared images;

[0073] S3: Extract and fuse the fine-grained association information and coarse-grained association information of the low-resolution features, medium-resolution features, and high-resolution features respectively to obtain low-resolution association features, medium-resolution association features, and high-resolution association features;

[0074] S4: Perform fusion processing on the high-resolution association features and the medium-resolution association features after image preprocessing to obtain a first fusion feature map; perform fusion processing on the first fusion feature and the low-resolution association features after image preprocessing to obtain a second fusion feature map; fuse the second fusion feature and the first fusion feature to obtain a third fusion feature map; fuse the third fusion feature and the high-resolution association features to obtain a fourth fusion feature map;

[0075] S5: Use the adversarial network DCGAN to generate second adversarial feature maps, third adversarial feature maps, and fourth adversarial feature maps for the second fusion feature map, third fusion feature map, and fourth fusion feature map respectively;

[0076] S6: Construct a defect detection model based on deep learning. The defect detection model detects the second fusion feature map, third fusion feature map, fourth fusion feature map, second adversarial feature map, third adversarial feature map, and fourth adversarial feature map respectively, and fuses the detection results to obtain the electronic component defect detection result;

[0077] S7: Evaluate the electronic component defect detection result, and optimize the defect detection model based on the evaluation result.

[0078] Further, the specific implementation process of S1 is as follows:

[0079] S11: Collect the initial infrared images containing multiple electronic components. Each electronic component collects three types of resolution images, and the three types of resolution images are specifically divided into:

[0080] Low resolution: w < 320, h < 320, 320 < w ≤ 640, 320 < h ≤ 640;

[0081] Medium resolution: 640 < w ≤ 1080, 640 < h ≤ 1080;

[0082] High resolution: 1080 < w, 1080 < h, where w represents the width of the image and h represents the height of the image;

[0083] S12: Segment the initial infrared images of the electronic components according to the contour of each electronic component by using the Canny edge detection algorithm to obtain multiple infrared images. The initial infrared images need to collect corresponding images of three types of resolutions.

[0084] Specifically, by analyzing three resolutions of the initial infrared image of electronic components, different defects of electronic components can be quickly identified. Because a single-resolution image is likely to lead to missed detection of different defects of electronic components, analyzing different image resolutions for the same electronic component can speed up the detection speed of the model and enable fine-grained and diverse identification of electronic component defects.

[0085] Furthermore, S1 further includes: the initial infrared image needs to extract features of the image brightness, and the method of calculating the average brightness of the image by using a histogram is adopted for feature extraction of the image brightness, which is specifically divided into:

[0086] First-level brightness: 0 ≤ B ≤ 50;

[0087] Second-level brightness: 50 < B ≤ 100;

[0088] Third-level brightness: 100 < B ≤ 180;

[0089] Fourth-level brightness: 180 < B ≤ 255, where B represents the average brightness value of the image.

[0090] Specifically, by extracting image features of the same electronic component under different brightnesses, the defect categories existing in the electronic component can be more accurately identified.

[0091] Furthermore, the image preprocessing in S4 includes:

[0092] Performing image preprocessing on the features obtained by fusion processing to obtain image preprocessing features, and the image preprocessing includes denoising, enhancing, and binarizing the image;

[0093] Mapping the image preprocessing features to the fusion processing features includes: for each position of the image preprocessing features, matching the corresponding position of the fusion processing features, and performing dot product and linear transformation operations on the local area centered on this position and the predicted image preprocessing features to generate the first fusion feature map, the second fusion feature map, the third fusion feature map, and the fourth fusion feature map.

[0094] Furthermore, before detecting the second fusion feature map, the third fusion feature map, the fourth fusion feature map, the second adversarial feature map, the third adversarial feature map, and the fourth adversarial feature map in S6 respectively, it includes:

[0095] Perform cross-attention mechanism processing, hybrid attention mechanism processing, point-wise spatial attention mechanism processing, and effective channel attention mechanism processing on the second fusion feature map, third fusion feature map, fourth fusion feature map, second adversarial feature map, third adversarial feature map, and fourth adversarial feature map in sequence.

[0096] Further, the specific implementation process of S6 is as follows:

[0097] S61: Collect the image data of the second fusion feature map, third fusion feature map, fourth fusion feature map, second adversarial feature map, third adversarial feature map, and fourth adversarial feature map. Part of it is used for the test set, and part of it is used for the training set. Train a defect detection model for deep learning through the training set.

[0098] S62: Divide the defect detection model into a feature extraction network Inception, a feature pyramid, and a classification and regression model. Use the feature extraction network Inception to extract fine-grained association information and coarse-grained association information. Connect the feature pyramid to the feature extraction network Inception, perform convolution on each layer of the feature pyramid, and perform classification and regression through the classification and regression model.

[0099] S63: Use the test set to perform network iterative testing on the feature extraction network Inception and output the detection result.

[0100] S64: Determine whether the detection result includes the types of electronic component defects. The types of electronic component defects include: scratches, cracks, bubbles, impurities, burrs, spots, holes, and oil stains.

[0101] Further, the specific construction steps of the feature extraction network Inception are as follows:

[0102] M1: Divide the feature extraction network Inception into Inception-A and Inception-B. Use Inception-A to extract fine-grained association information, where the fine-grained association information includes at least edges and textures. Use Inception-B to extract coarse-grained association information, where the coarse-grained association information includes at least shapes and objects. Inception-A includes a first convolutional layer, a second convolutional layer, and a third convolutional layer. Inception-B includes a fourth convolutional layer, a fifth convolutional layer, and a sixth convolutional layer.

[0103] Specifically, the extraction of fine-grained association information:

[0104] It targets the minute defects of electronic components, which appear as detailed features in images and require high-resolution and high-precision recognition capabilities. The defects that can be recognized by fine-grained correlation information include: scratches, cracks, burrs, spots, and holes. These defects need to be extracted and analyzed through high-resolution images because the damage they cause on the component surface is subtle.

[0105] In this embodiment, for scratches and cracks, they are recognized by detecting minute fractures and indentations on the component surface; for burrs, they are recognized by detecting minute protruding parts at the edges of the components; for spots and holes, they need to be recognized by detecting the discontinuities or missing parts on the material surface.

[0106] Coarse-grained correlation information extraction:

[0107] It targets the overall defects or large-scale defects of electronic components, which appear as relatively obvious features in images and are recognized through lower-resolution imaging techniques. The defects that can be recognized by coarse-grained correlation information extraction include: bubbles, impurities, and oil stains. These defects usually form relatively obvious abnormal areas on or inside the component surface and can be recognized through macroscopic observation.

[0108] In this embodiment, bubbles are recognized by detecting circular or oval transparent or semi-transparent areas inside the component; impurities are recognized by detecting non-uniformly distributed particles or clumps in the material; oil stains are recognized by detecting irregular shiny areas on the component surface.

[0109] M2: Use the adversarial network DCGAN to generate second adversarial feature maps, third adversarial feature maps, and fourth adversarial feature maps for the second fusion feature map, third fusion feature map, and fourth fusion feature map respectively, and use the images of the generated second adversarial feature maps, third adversarial feature maps, and fourth adversarial feature maps as training set samples.

[0110] M3: Add the generated training set samples to the database for storage, which are used for training set data and test set data.

[0111] The specific process of M2 is as follows: According to the original images of the second fusion feature map, third fusion feature map, and fourth fusion feature map given in the current training, generate adversarial image samples of the second adversarial feature map, third adversarial feature map, and fourth adversarial feature map. The specifications of the adversarial image samples are: diverse size features of electronic components, non-fixed position features of electronic components, and diverse image brightness features. Input 1 given original image, and add perturbation factors through the adversarial network DCGAN to generate 100 adversarial sample images.

[0112] Specifically, the model sample data volume is expanded through the adversarial network DCGAN to obtain a diverse set of electronic component generation images, solving the problem of insufficient model sample quantity. Simply put, the more times an electronic component has defects in the past, the more rigorous the inspection is required. The more model construction sample data volume required in the model performance requirement information, the higher the model accuracy requirement will be (for example, 98%), and vice versa. This provides support for ensuring the accuracy of model recognition and the robustness of the model.

[0113] Furthermore, the specific process of connecting the feature pyramid in series to the feature extraction network Inception is as follows:

[0114] The first convolution layer, that is, the feature map of fine-grained association information, is sampled and then superimposed with the feature map of coarse-grained association information of the fourth convolution layer to obtain the first layer of the pyramid. This step is continued for the second and third convolution layers. Each three adjacent layers of feature maps are superimposed on the channel to obtain one layer of the pyramid. Finally, a total of three layers of feature pyramid are obtained. The specific processing process of the three layers of the pyramid is as follows:

[0115] The first level of the pyramid is used to process low-resolution images;

[0116] The second level of the pyramid is used to process medium-resolution images;

[0117] The third level of the pyramid is used to process high-resolution images.

[0118] Furthermore, in S7, a mean square error loss function is used to measure the difference between the predicted value and the true value, so as to evaluate the electronic component defect detection result:

[0119]

[0120] Where Loss represents the loss function; T represents the step size in the time series data; N represents the total number of output defect types; g t+1,i The i-th element in the One-Hot encoding form representing the real electronic component defect at time point t+1; represents the predicted probability for the i-th defect category at time t+1.

[0121] Furthermore, the parameters of the defect detection model are adjusted by the whale optimization algorithm to minimize the loss function and achieve optimization and improvement of the defect detection model. The specific adjustment parameters of the whale optimization algorithm are:

[0122] P m =Pc·δ P Loss;

[0123] Among them, Pm P represents the parameters of the updated model; c represents the defect detection rate, which is used to control the frequency of parameter update; δ P Loss represents the gradient of the loss function with respect to the parameter P; P represents the parameters of the defect detection model.

[0124] The beneficial effects of an electronic component defect detection method of the present invention are as follows:

[0125] The present invention obtains infrared images corresponding to three resolutions of an electronic component; extracts low-resolution features, medium-resolution features, and high-resolution features of the infrared images; respectively extracts and fuses the fine-grained association information and coarse-grained association information of the low-resolution features, medium-resolution features, and high-resolution features to obtain low-resolution association features, medium-resolution association features, and high-resolution association features; respectively generates second adversarial feature maps, third adversarial feature maps, and fourth adversarial feature maps for the second fusion feature map, third fusion feature map, and fourth fusion feature map through the adversarial network DCGAN; constructs a defect detection model based on deep learning, fuses the detection results to obtain the electronic component defect detection result; evaluates the electronic component defect detection result, and optimizes the defect detection model based on the evaluation result; the defect detection model based on deep learning of the present invention identifies and detects the fine-grained and coarse-grained information of infrared images with different resolutions of the electronic component, and at the same time uses the adversarial network DCGAN to generate adversarial sample maps, improving the defect detection accuracy, accuracy, and robustness.

[0126] The above describes the present invention and its implementation manners. This description is not restrictive. What is shown in the drawings is only one of the implementation manners of the present invention. The actual content is not limited thereto. Generally speaking, if those of ordinary skill in the art are inspired by it and design similar structural manners and embodiments without creative efforts without departing from the purpose of the present invention, they should all fall within the protection scope of the present invention.

Claims

1. A method for detecting defects in electronic components, characterized in that: Including: S1: Obtain infrared images corresponding to three resolutions of electronic components, where the three resolutions include low resolution, medium resolution, and high resolution; S2: Extract the low-resolution features, medium-resolution features, and high-resolution features of the infrared images; S3: Respectively extract and fuse the fine-grained association information and coarse-grained association information of the low-resolution features, medium-resolution features, and high-resolution features to obtain low-resolution association features, medium-resolution association features, and high-resolution association features; S4: Perform fusion processing on the high-resolution association features and the medium-resolution association features after image preprocessing to obtain a first fusion feature map; perform fusion processing on the first fusion feature and the low-resolution association features after image preprocessing to obtain a second fusion feature map; Fuse the second fusion feature with the first fusion feature to obtain a third fusion feature map; fuse the third fusion feature with the high-resolution association features to obtain a fourth fusion feature map; S6: Respectively generate second adversarial feature maps, third adversarial feature maps, and fourth adversarial feature maps for the second fusion feature map, third fusion feature map, and fourth fusion feature map through the adversarial network DCGAN; S7: Construct a defect detection model based on deep learning. The defect detection model respectively detects the second fusion feature map, third fusion feature map, fourth fusion feature map, second adversarial feature map, third adversarial feature map, and fourth adversarial feature map, and fuses the detection results to obtain the defect detection result of the electronic component; S8: Evaluate the defect detection result of the electronic component, and optimize the defect detection model based on the evaluation result.

2. The electronic component defect detection method according to claim 1, characterized in that: The specific implementation process of S1 is as follows: S11: Collect initial infrared images containing multiple electronic components. Each electronic component collects three types of resolution images, and the three types of resolution images are specifically divided into: Low resolution: w < 320, h < 320, 320 < w ≤ 640, 320 < h ≤ 640; Medium resolution: 640 < w ≤ 1080, 640 < h ≤ 1080; High resolution: 1080 < w, 1080 < h, where w represents the width of the image and h represents the height of the image; S12: Segment the initial infrared images of the electronic components according to the contour of each electronic component by using the Canny edge detection algorithm to obtain multiple infrared images. The initial infrared images need to collect images corresponding to three resolutions.

3. The electronic component defect detection method according to claim 2, characterized in that: S1 also includes: Feature extraction needs to be performed on the image brightness of the initial infrared image. The method of calculating the average brightness of the image by using a histogram is adopted for feature extraction of the image brightness, and it is specifically divided into: First-level brightness: 0 ≤ B ≤ 50; Second-level brightness: 50 < B ≤ 100; Third-level brightness: 100 < B ≤ 180; Fourth-level brightness: 180 < B ≤ 255, where B represents the average brightness value of the image.

4. The electronic component defect detection method according to claim 3, characterized in that: The image preprocessing in S4 includes: Performing image preprocessing on the fused features to obtain image preprocessing features, wherein the image preprocessing includes denoising, enhancing and binarizing the image; Mapping the image preprocessing features to the fused features includes: matching each position of the image preprocessing features to the corresponding position of the fused features, and performing point multiplication and linear transformation operations on a local area centered on the position and the predicted image preprocessing features to generate the first fused feature map, the second fused feature map, the third fused feature map, and the fourth fused feature map.

5. The electronic component defect detection method according to claim 4, characterized in that: In the S6, before respectively detecting the second fused feature map, the third fused feature map, the fourth fused feature map, the second adversarial feature map, the third adversarial feature map and the fourth adversarial feature map, the steps include: The second fused feature map, the third fused feature map, the fourth fused feature map, the second adversarial feature map, the third adversarial feature map and the fourth adversarial feature map are respectively processed by cross attention mechanism, mixed attention mechanism, point-wise spatial attention mechanism and effective channel attention mechanism.

6. The electronic component defect detection method according to claim 5, characterized in that: The specific implementation process of S6 is as follows: S61: collecting image data of the second fused feature map, the third fused feature map, the fourth fused feature map, the second adversarial feature map, the third adversarial feature map, and the fourth adversarial feature map, a portion of which is used as a test set, and a portion of which is used as a training set, and a deep learning defect detection model is trained through the training set; S62: Divide the defect detection model into a feature extraction network Inception, a feature pyramid, and a classification regression model, use the feature extraction network Inception to extract fine-grained association information and coarse-grained association information, connect the feature pyramid in series to the feature extraction network Inception, perform convolution on each layer of the feature pyramid, and perform classification and regression through the classification regression model; S63: Performing a network iteration test on the feature extraction network Inception using the test set, and outputting a detection result; S64: Determine whether the detection result includes electronic component defect types, where the electronic component defect types include scratches, cracks, bubbles, impurities, burrs, spots, holes and oil stains.

7. The electronic component defect detection method according to claim 6, characterized in that: The specific steps of building the feature extraction network Inception are as follows: M1: The feature extraction network Inception is divided into Inception-A and Inception-B, Inception-A is used to extract fine-grained association information, wherein the fine-grained association information at least includes edges and textures, and Inception-B is used to extract coarse-grained association information, wherein the coarse-grained association information at least includes shapes and objects, wherein the Inception-A includes the first convolutional layer, the second convolutional layer, and the third convolutional layer, and the Inception-B includes the fourth convolutional layer, the fifth convolutional layer, and the sixth convolutional layer; M2: Use the adversarial network DCGAN to generate a second adversarial feature map, a third adversarial feature map and a fourth adversarial feature map for the second fused feature map, the third fused feature map and the fourth fused feature map, respectively, and generate images of the second adversarial feature map, the third adversarial feature map and the fourth adversarial feature map as training set samples; M3: Add the generated training set samples to the database for storage, which is used for training set data and test set data; The specific process of M2 is: according to the original images of the second fusion feature map, the third fusion feature map and the fourth fusion feature map given by the current training, generate adversarial image samples of the second adversarial feature map, the third adversarial feature map and the fourth adversarial feature map, and the adversarial image sample specifications are: the size diversity feature of electronic components, the non-fixed location feature of electronic components and the image brightness diversity feature, input 1 given original image, add perturbation factors through the adversarial network DCGAN, and generate 100 adversarial sample images.

8. The electronic component defect detection method according to claim 7, characterized in that: The specific process of connecting the feature pyramid in series to the feature extraction network Inception is as follows: The first convolution layer, that is, the feature map of fine-grained association information, is sampled and then superimposed with the feature map of coarse-grained association information of the fourth convolution layer to obtain the first layer of the pyramid. This step is continued for the second and third convolution layers. Each three adjacent layers of feature maps are superimposed on the channel to obtain one layer of the pyramid. Finally, a total of three layers of feature pyramid are obtained. The specific processing process of the three layers of the pyramid is as follows: The first level of the pyramid is used to process low-resolution images; The second level of the pyramid is used to process medium-resolution images; The third level of the pyramid is used to process high-resolution images.

9. The electronic component defect detection method according to claim 8, characterized in that: In S7, a mean square error loss function is used to measure the difference between the predicted value and the true value, so as to evaluate the electronic component defect detection result: Where Loss represents the loss function; T represents the step size in the time series data; N represents the total number of output defect types; g t+1,i The i-th element in the One-Hot encoding form representing the real electronic component defect at time point t+1; represents the predicted probability for the i-th defect category at time t+1; The parameters of the defect detection model are adjusted by the whale optimization algorithm to minimize the loss function and optimize and improve the defect detection model. The specific adjustment parameters of the whale optimization algorithm are: P m =P-c·δ P Loss; Among them, P m represents the parameters of the updated model; c represents the defect detection rate, which is used to control the frequency of parameter updating; δ P Loss represents the gradient of the loss function with respect to the parameter P; P represents the parameter of the defect detection model.

10. An electronic component defect detection device, characterized in that: The device comprises: at least one processor, memory, and input-output unit; The memory is used to store a computer program, and the processor is used to call the computer program stored in the memory to execute the method as claimed in any one of claims 1 to 9.

Images