Electronic component appearance defect detection method and system based on multispectral imaging

Abstract

The present invention proposes an electronic component appearance defect detection method and system based on multispectral imaging, which belongs to the field of optical measurement technology, including: obtaining an appearance image dataset of an electronic component to be detected, performing image preprocessing, extracting an area of ​​interest of the appearance of the electronic component to be detected, and obtaining a multispectral image of the area of ​​interest; extracting the spectral reflectance characteristics and spatial characteristics of the multispectral image of the area of ​​interest through a spectral reflectance reconstruction method combining a genetic algorithm and a neural network, and performing feature fusion to form a spatial-spectral joint feature dataset of the area of ​​interest, which is input into a trained appearance defect detection model to obtain the defect type and size of the appearance of the electronic component to be detected; and displaying the appearance defect detection results of the electronic component to be detected. The present invention improves the accuracy and efficiency of appearance defect detection of electronic components.

Description

Technical Field

[0001] The present invention belongs to the technical field of optical measurement, and in particular relates to a method and system for detecting appearance defects of electronic components based on multi-spectral imaging. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] As the basic unit of electronic equipment and information systems, the quality of electronic components directly affects the performance and function realization of instruments and equipment. In the actual production process of electronic components, there may be defects such as bubbles, discoloration, rupture, deformation, cracks, electrode deformity, electrode defects, electrode shedding, poor adhesion, edge scarcity, scratches, bumps, and missing corners. These defects will seriously affect the yield rate of components. Therefore, electronic component defect detection is an important link to ensure the quality and performance of components.

[0004] Traditional manual visual inspection is the main defect detection method used. This method of sampling manual visual inspection has the problems of low detection efficiency, high labor intensity, and high false detection rate, which greatly affects the detection effect of surface defects of components and cannot meet the needs of real-time online detection in automated production processes. With the increasing types of electronic components, the requirements and process difficulties of quality inspection are also gradually increasing, and automatic optical inspection equipment is gradually applied to surface defect detection of electronic components. However, factors such as shooting environment, lighting conditions, and different types of equipment will have a certain impact on the quality of the captured image, especially for the contamination of the electrode surface, the traditional visible light image cannot be accurately identified, resulting in the limited detection accuracy of automatic optical inspection equipment for surface defects of electronic components and limited application scope. Therefore, it is urgent to provide a high-precision and high-stability appearance defect detection method for electronic components. Summary of the invention

[0005] In order to overcome the shortcomings of the above-mentioned prior art, the present invention provides a method and system for detecting appearance defects of electronic components based on multispectral imaging. The multispectral imaging system is used to measure the spectral reflectivity of any position in space, and the appearance defect feature recognition and detection of electronic components are realized by combining high-resolution two-dimensional spatial images, thereby improving the accuracy and efficiency of electronic component appearance defect detection.

[0006] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions: A first aspect of the present invention provides a method for detecting appearance defects of electronic components based on multispectral imaging.

[0007] A method for detecting appearance defects of electronic components based on multispectral imaging, comprising: Acquire a data set of appearance images of electronic components to be inspected; Performing image preprocessing on the appearance image data set of the electronic component to be inspected, extracting the region of interest of the appearance of the electronic component to be inspected, and obtaining a multispectral image of the region of interest; Extracting the spectral reflectance features of the multispectral image of the region of interest by combining a genetic algorithm with a neural network spectral reflectance reconstruction method; By quantifying the image texture features, the spatial features of the multispectral image in the area of ​​interest are extracted; Fusing the spectral reflectance feature and the spatial feature to form a spatial-spectral joint feature data set of an area of ​​interest; Inputting the spatial-spectral joint feature data set of the region of interest into the trained appearance defect detection model to obtain the defect type and size of the appearance of the electronic component to be detected; Display the appearance defect detection results of the electronic components to be tested.

[0008] A second aspect of the present invention provides an electronic component appearance defect detection system based on multi-spectral imaging.

[0009] An electronic component appearance defect detection system based on multispectral imaging, comprising: Light source module, imaging module, control module and data processing module; The light source module is composed of light emitting diodes of different emission spectrum types; The imaging module is composed of an imaging lens and a detector; The control module is used to control the light source driver and the data acquisition card of the industrial camera; The control module drives the light source module to perform time-sharing stroboscopic lighting, and synchronously triggers the imaging module to perform image acquisition, so as to obtain a data set of appearance images of the electronic components to be inspected; The data processing module performs image preprocessing on the image data set of the appearance of the electronic component to be detected, extracts the region of interest of the appearance of the electronic component to be detected, and obtains a multispectral image of the region of interest; Extracting the spectral reflectance features of the multispectral image of the region of interest by combining a genetic algorithm with a neural network spectral reflectance reconstruction method; By quantifying the image texture features, the spatial features of the multispectral image in the area of ​​interest are extracted; Fusing the spectral reflectance feature and the spatial feature to form a spatial-spectral joint feature data set of an area of ​​interest; Inputting the spatial-spectral joint feature data set of the region of interest into the trained appearance defect detection model to obtain the defect type and size of the appearance of the electronic component to be detected; Display the appearance defect detection results of the electronic components to be tested.

[0010] The third aspect of the present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in a method as described in the first aspect of the present invention are implemented.

[0011] A fourth aspect of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps in a method as described in the first aspect of the present invention.

[0012] A fifth aspect of the present invention provides a computer program product comprising instructions, which, when running on a computer, enables the computer program to implement the steps in a method as described in the first aspect of the present invention when executed by a processor.

[0013] One or more of the above technical solutions have the following beneficial effects: The present invention provides a device and method for detecting appearance defects of electronic components based on a multi-spectral system. The imaging technology is combined with the spectral technology to quickly obtain the two-dimensional spatial information and one-dimensional spectral information of the object to be tested, and can accurately detect the type, size, shape, distribution and other information of the electronic components. It has the advantages of real-time observation, non-contact detection, and strong resistance to environmental interference, and is particularly suitable for the detection and measurement of defect types of electronic components under complex conditions.

[0014] The present invention provides a spectral reflectance reconstruction method combining a genetic algorithm and a neural network. The multispectral image response value is used as an input variable of the neural network. The optimal neural network threshold and weight are obtained in the genetic algorithm, and the obtained values ​​are assigned to the corresponding network parameters. The optimized neural network reconstructs the spectral reflectance with higher accuracy. Compared with the traditional method of directly measuring the spectral reflectance by a spectral imaging system based on a narrow-band filter, the method has the advantages of simple overall structure and low cost.

[0015] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0017] Figure 1 A flow chart of a method for detecting appearance defects of electronic components based on multispectral imaging according to an embodiment of the present invention; Figure 2 Schematic diagram of an electronic component appearance defect detection system based on multispectral imaging according to an embodiment of the present invention; Figure 3 The graph is a graph of measured spectra of 17 light-emitting diodes selected for use in the embodiments of the present invention. DETAILED DESCRIPTION

[0018] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0019] It should be noted that the terms used herein are for describing specific embodiments only and are not intended to be limiting of exemplary embodiments according to the present invention.

[0020] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0021] Embodiment 1 This embodiment discloses a method for detecting appearance defects of electronic components based on multispectral imaging, comprising: Step 1: Obtain an appearance image dataset of electronic components to be inspected; Step 2: performing image preprocessing on the image data set of the appearance of the electronic component to be detected, extracting the region of interest of the appearance of the electronic component to be detected, and obtaining a multispectral image of the region of interest; Step 3, extracting the spectral reflectance characteristics of the multispectral image of the region of interest by combining a genetic algorithm and a neural network spectral reflectance reconstruction method; Step 4: Extracting the spatial features of the multispectral image of the region of interest by quantifying the image texture features; Step 5: Fusing the spectral reflectance feature and the spatial feature to form a spatial-spectral joint feature data set of the region of interest; Step 6: Input the spatial-spectral joint feature data set of the region of interest into the trained appearance defect detection model to obtain the defect type and size of the appearance of the electronic component to be detected; and display the appearance defect detection result of the electronic component to be detected.

[0022] In order to explain this embodiment more clearly, Figure 1 As shown, a method for detecting appearance defects of electronic components based on multispectral imaging can be specifically described as follows: Step 1: Obtain a dataset of appearance images of electronic components to be inspected.

[0023] In this embodiment, a multispectral imaging system is used to collect images of electronic components to obtain image data sets of the electronic components to be tested under different lighting conditions, including 17 spectral images of the electronic components to be tested and 1 background image.

[0024] The specific method is as follows: first, control the first light source in the light source system to turn on, and turn off the other light sources to collect the first picture; then turn off the first light source, and turn on the second light source to collect the second image; following the same operation, each type of light source is turned on and off in turn, and 17 images of the electronic components to be tested under lighting conditions of different spectral types are collected; finally, turn off the light source module, and collect a background image under only ambient light conditions to obtain all the image data required for the detection of the electronic components to be tested.

[0025] Step 2: performing image preprocessing on the image data set of the appearance of the electronic component to be inspected, extracting the region of interest of the appearance of the electronic component to be inspected, and obtaining a multispectral image of the region of interest.

[0026] In this embodiment, the image data set is subjected to image preprocessing, specifically, the region of interest in the image is identified and divided through background correction processing and image segmentation processing. The background correction process is to process each multispectral image Image with background light Subtract to remove the influence of background ambient light. The calculation formula is as follows: (1) The image segmentation method is to obtain the region of interest of the electronic components in each group of images through an edge detection algorithm: first, the grayscale image is binarized through an adaptive threshold method, and finally, the region of interest of the electronic components is extracted through a binary image template to obtain a multispectral image of the region of interest of the electronic components.

[0027] The specific steps are as follows: 1) Obtain a binary image of electronic components by using the Otsu thresholding method on a single-channel image; 2) Make the outline of the electronic components clear by dilation and erosion operations on the binary image; 3) Use the watershed algorithm to segment the electronic components into individual images; 4) Extract the region of interest of the electronic component image.

[0028] By performing background correction and image segmentation processing on the image data set of electronic components, the interference of background light when collecting multispectral images is eliminated, and the region of interest of the electronic components of interest is extracted.

[0029] Step 3: extracting the spectral reflectance features of the multispectral image of the region of interest by combining a genetic algorithm with a neural network spectral reflectance reconstruction method.

[0030] In this embodiment, the spectral reflectance characteristics of the region of interest are extracted, and the multispectral image data of the region of interest is used as input through a trained spectral reflectance reconstruction model to obtain the full-spectrum spectral reflectance information of the spectral region with a wavelength of 400nm~1000nm.

[0031] In this embodiment, the specific implementation of constructing the spectral reflectance reconstruction model includes: 1) Collect the image response values ​​of the standard color card under different lighting conditions and divide them into training set and test set: Specifically, a 24-color standard color card is used as the experimental object, and the spectral reflectance value of each color block is measured by a spectrophotometer. The spectral wavelength range is 400nm-1000nm, with an interval of 1nm. In the experiment, 23 color blocks are selected for training and 1 color block is selected for testing.

[0032] Before neural network training, the multispectral image set in step 2 is normalized to obtain the response matrix G, from which 23 color blocks are extracted to form a training set and 1 color block is extracted to form a test set.

[0033] 2) Determine the structural parameters of the BP neural network: Specifically, the input is a 24×17 response matrix, and the output is a 24×601 spectral reflectance matrix, that is, the input layer is 17 nodes, the output layer is 601 nodes, and 601 wavelength values ​​in the 400-1000nm band are selected at 1nm intervals. The number of hidden layer nodes is determined according to the empirical formula: The empirical formula is as follows: (2) Among them, m is the number of input layer nodes, n is the number of output layer nodes, and a is an integer between 1 and 10.

[0034] In this embodiment, m=17, n=601, so the range of the number of hidden layer nodes x is 25 to 35. Integers in this range are sequentially substituted into the BP neural network to calculate the error, and the number of nodes corresponding to the minimum error is selected as the optimal number of hidden layer nodes for the next training model.

[0035] 3) Input the training set and response set of the standard color card into the neural network, set the initial weight, bias parameter, training target and other parameters of the neural network model, build the BP neural network model and test it.

[0036] 4) In view of the problems that the BP neural network model is prone to slow convergence, easy to fall into local extreme values, and easy to oscillate during the learning process, a genetic algorithm is used to calculate the global optimal weights and bias parameters so that the optimized BP neural network can better perform sample prediction.

[0037] The basic idea of ​​​​the genetic algorithm to optimize the BP neural network is to first randomly generate an initial population of a certain size. Each individual in the population represents a set of weights and bias parameters of the neural network. The fitness function of each individual in the initial population is calculated, and a new population is generated by selecting operators, crossover operators, and mutation operators to replace the original population. After iterative calculations, check whether the termination conditions are met, such as whether the optimal fitness value or the maximum number of iterations is reached. Finally, the best weight value and bias parameter value are output to obtain the optimized BP neural network model.

[0038] Among them, the fitness function of the neural network is set to the mean square error (MSE) between the true spectral reflectance vector Re and the vector Rp predicted by the neural network, which is expressed as: (3) Wherein, U is the band beam, and in this embodiment, U=601.

[0039] 5) Use the trained neural network to perform simulation prediction on the test color block to obtain the spectral reflectance of the test color block and verify the reconstruction accuracy of the spectral reflectance.

[0040] The image response values ​​of the electronic components to be tested under different lighting conditions are input into the improved BP neural network to obtain the spectral reflectance of each pixel at each wavelength position. The spectral reflectance curve is composed of the reflectances of all wavelengths, the spectral reflectance is reconstructed, and the full spectral characteristics of the electronic components are obtained.

[0041] The multispectral image response value is used as the input variable of the neural network, the optimal neural network threshold and weight are obtained in the genetic algorithm, and the obtained values ​​are assigned to the corresponding network parameters. The optimized neural network reconstructs the spectral reflectance with higher accuracy. Compared with the traditional method of directly measuring spectral reflectance based on a spectral imaging system with a narrow-band filter, it has the advantages of simple overall structure and low cost.

[0042] Step 4: Extract the spatial features of the multispectral image of the region of interest by quantifying the image texture features.

[0043] The spatial feature extraction of the multispectral image is performed, and the image texture features are extracted using the grayscale-gradient co-occurrence matrix method. The energy value, entropy value, contrast, and correlation are calculated to quantify the texture features of the sample, thereby describing the spatial feature attributes such as uniformity, complexity, clarity, and directionality of the image texture.

[0044] The specific process is as follows: Energy value: reflects the uniformity of grayscale distribution and texture coarseness in the image area. The calculation method is: (4) Among them, i and j represent gray levels, P(i,j) represents the frequency of pixel pairs under a specific position relationship, and Ng represents the matrix dimension.

[0045] Entropy value: Indicates the degree of non-uniformity or complexity of the texture in the image. The calculation method is as follows: (5) Among them, i and j represent gray levels, P(i,j) represents the frequency of pixel pairs under a specific position relationship, and Ng represents the matrix dimension.

[0046] Contrast: This feature is used to describe the clarity of the image and is calculated as follows (6) Among them, i and j represent gray levels, P (i, j) represents the frequency of pixel pairs under a specific position relationship, Ng represents the matrix dimension; n = |ij| represents the grayscale difference.

[0047] Correlation: This feature is used to measure the similarity of elements in the gray level co-occurrence matrix in the row or column direction. The calculation method is as follows: (7) in, and denote the average values ​​in row and column directions respectively, and Represent the variance in the row and column directions respectively; i and j represent the grayscale, P(i,j) represents the frequency of pixel pairs under a specific position relationship, and Ng represents the matrix dimension.

[0048] Step 5: Fusing the spectral reflectance feature and the spatial feature to form a spatial-spectral joint feature data set of the region of interest.

[0049] In this embodiment, the spectral features and spatial features are fused by principal component analysis method to form a spectral-spatial feature data set which is input into the defect detection module.

[0050] Step 6: Input the spatial-spectral joint feature data set of the region of interest into the trained appearance defect detection model to obtain the defect type and size of the appearance of the electronic component to be detected; and display the appearance defect detection result of the electronic component to be detected.

[0051] In this embodiment, it specifically refers to inputting the spectral features and spatial features of the multispectral image set of the region of interest into the training to obtain the appearance defect detection model, obtain the appearance defect type of the electronic component to be detected, determine the defect size calculation method of the electronic component to be detected, and determine the defect detection result of the electronic component to be detected according to the calculation result. Among them, the training set and test set of the appearance defect detection model include multiple features, as well as the defect position and type of the defect area in each image. The defect classification detection method can use the least squares support vector machine method to establish an appearance defect detection model to achieve the classification and identification of the appearance defects of electronic components. The defect detection result includes the defect category, defect position, and defect size of the detected electronic component.

[0052] In this embodiment, the defect detection model is determined in the following manner, and the specific steps include: 1) Construct a good product image library and a defect image library of electronic components, where the good product image library can include qualified electronic component images and qualified labels, and the defect image library can include defective electronic component images and defect type and location labels; 2) Dividing the image library into a training set and a test set; 3) Taking the feature vector in the training set as input and the defect type corresponding to the input training set as output, a defect detection model is obtained by training; 4) Taking the feature vector in the test set as input and the defect type corresponding to the input test machine as output, testing the defect detection model; 5) Select a defect model whose test accuracy meets the detection conditions as the defect detection model.

[0053] In this embodiment, the defect classification detection model adopts the least squares support vector machine method (LS-SVM) to establish a detection model for the appearance defects of electronic components. The basic idea is to find an optimal hyperplane in the data space to separate data samples of different categories. It has the advantages of short training time and low complexity.

[0054] The specific method for determining the appearance defect classification detection model based on the least squares support vector machine method is as follows: 1) Construct a data set and set the set of spatial-spectral feature vectors and label data of N samples as , where the input feature vector is the feature vector of the sample, represents the category of the sample, and n represents the number of samples; 2) Using nonlinear functions The input spectral data set consisting of the spatial-spectral feature vectors and label data of N samples is mapped to the high-dimensional feature space and the relationship model is established as follows: (8) Among them, w represents the weight vector and b represents the bias value.

[0055] 3) Apply the least squares support vector machine to solve the problem. The function fitting problem can be expressed as an optimization problem: (9) in, represents the penalty coefficient, represents the error variable, w represents the weight vector; 4) Solve the model in the dual space and obtain the fitting model of the least squares support vector machine: (10) in, is the kernel function, and b are model parameters, which are determined through the training process.

[0056] In this embodiment, a nonlinear radial basis function is selected as the kernel function of the support vector machine classifier.

[0057] In this embodiment, the defect size of the electronic component can be quantified by different calculation methods, including defect area calculation, defect length calculation, aspect ratio, etc. According to the preset threshold range of different types of defect areas, it can be judged whether the electronic component to be tested is qualified.

[0058] The appearance defect detection module is used to classify and mark the surface defects of the electronic components to be inspected, and the defect size calculation method of the electronic components to be tested is determined according to the defect type. The module can automatically and quickly process a large number of product images, and finally output the detected image marked with relevant information such as defect size, defect location, defect type, etc., and save the results in the local database, thereby completing the automatic detection of surface defects of electronic components.

[0059] Embodiment 2 The purpose of this embodiment is to provide an electronic component appearance defect detection system based on multi-spectral imaging.

[0060] An electronic component appearance defect detection system based on multispectral imaging is provided in an embodiment of the present invention, such as Figure 2As shown, it includes S101 light source module, S102 imaging module, S103 control module and S104 data processing module. Among them, the light source module provides different types of spectral lighting; the control module is used to control each type of light source to light up and turn off in sequence, and synchronously trigger the imaging module to collect images; the imaging module obtains images of electronic components, and sends the multi-spectral image set and background image to the data processing module, thereby completing the automatic detection of surface defects.

[0061] The light source module is composed of light emitting diodes of different spectral types, including broad-spectrum light sources and narrow-spectrum light sources. In this embodiment, the light source module preferably comprises 17 types of light emitting diodes, the spectral range of which completely covers 400nm-1000nm, and its spectral characteristics are as follows: Figure 3 shown.

[0062] The light source module has high requirements for lighting distance, lighting area and lighting uniformity. In this embodiment, all light-emitting diode light sources are divided into two groups and placed symmetrically. The spatial position distribution of each type of light-emitting diode needs to be optimized. The position and incident angle of different types of light sources are used as independent variables. The illumination distribution of different light source arrays is calculated by optical simulation methods until the error of illumination distribution of different types of spectra is minimized and optimal. The optimal structural configuration of the light source module can be obtained to improve the accuracy and efficiency of defect detection.

[0063] In this embodiment, the light source module is configured as an array for uniform illumination, and each LED light source is equipped with a uniform light lens. Light sources of different wavelengths can illuminate the detection area with the same illumination uniformity, so that different types of LED light sources have the same uniform spectral power distribution within the camera field of view.

[0064] The imaging module consists of an imaging lens and a detector. The choice of the imaging lens has a great impact on the quality of the image. It is generally determined according to the characteristics of the object to be detected and the working distance of the visual detection system. Multiple factors need to be considered, such as the number of pixels, pixel size, spectral response range, and cost. According to the characteristics of the object to be detected and the working distance of the visual detection system, a high-resolution industrial telecentric optical lens is used in this embodiment to quickly obtain the dimensions of high-quality electronic components. The detector uses a high-resolution, high-frame-rate CMOS camera module. The spectral response range of the sensor is 360nm-1000nm, covering the spectral range of the light source. The camera has a maximum frame rate of up to 120fps and has a good frequency response.

[0065] The control module is used to control the light source driver and the data acquisition card of the industrial camera. It sets the exposure parameters, integration time and start-up order of the imaging module, drives the light source module to perform time-sharing stroboscopic lighting, and synchronously triggers the imaging module to perform image acquisition, so as to obtain the image data set of the electronic components to be inspected.

[0066] That is, each type of light-emitting diode is controlled to be turned on or off at a certain time through control instructions. When the light-emitting diode light source is turned on, the camera is synchronously triggered to take pictures and collect a two-dimensional image of the target object. By imaging in sequence according to the wavelength distribution of the light-emitting diode array, a complete multispectral image set of the target object can be obtained.

[0067] The data processing module is used to implement the steps of the above method when executing the software program deployed on the computer, including extracting the area of ​​interest of the electronic components in each image, reconstructing the spectral reflectance using the brightness information of each pixel, and obtaining the spatial feature information of the electronic components through a feature extraction method, inputting the spectral-spatial feature information into the defect detection model to classify and label the surface defects, and obtaining the defect information of the electronic component samples.

[0068] This system uses light-emitting diodes with different emission spectrum types to form the light source module. The control module drives different types of light-emitting diodes to perform time-sharing stroboscopic lighting, and synchronously triggers the imaging module to collect images, thereby achieving low-cost and rapid acquisition of multi-spectral images. The imaging lens is used to converge the light beam with the reflectivity information of the target object, and the detector outputs a two-dimensional image of the target object based on the light intensity information of the beam. The data processing module has the functions of image preprocessing, image spectral feature extraction, image spatial feature extraction and defect detection, and can realize the appearance defect detection of electronic components.

[0069] Embodiment 3 The purpose of this embodiment is to provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the program.

[0070] Embodiment 4 The purpose of this embodiment is to provide a computer-readable storage medium.

[0071] A computer-readable storage medium stores a computer program, which executes the steps of the above method when executed by a processor.

[0072] Embodiment 5 The purpose of this embodiment is to provide a computer program product containing instructions, which, when running on a computer, enables the computer to execute the methods and functions involved in any of the above embodiments. The steps involved in the apparatus of the above embodiment correspond to the method embodiment 1, and the specific implementation method can refer to the relevant description part of embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood to include any medium that can store, encode or carry an instruction set for execution by a processor and enable the processor to execute any method in the present invention.

[0073] Those skilled in the art should understand that the modules or steps of the present invention described above can be implemented by a general-purpose computer device, or alternatively, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0074] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A method for detecting appearance defects of electronic components based on multispectral imaging, characterized in that: include: Acquire a data set of appearance images of electronic components to be inspected; Performing image preprocessing on the appearance image data set of the electronic component to be inspected, extracting the region of interest of the appearance of the electronic component to be inspected, and obtaining a multispectral image of the region of interest; Extracting the spectral reflectance features of the multispectral image of the region of interest by combining a genetic algorithm with a neural network spectral reflectance reconstruction method; By quantifying the image texture features, the spatial features of the multispectral image in the area of ​​interest are extracted; Fusing the spectral reflectance feature and the spatial feature to form a spatial-spectral joint feature data set of an area of ​​interest; Inputting the spatial-spectral joint feature data set of the region of interest into the trained appearance defect detection model to obtain the defect type and size of the appearance of the electronic component to be detected; Display the appearance defect detection results of the electronic components to be tested.

2. The method for detecting appearance defects of electronic components based on multispectral imaging according to claim 1, characterized in that: Performing image preprocessing on the appearance image data set of the electronic components to be inspected, specifically including: identifying and dividing the region of interest in the image through background correction processing and image segmentation processing; The background correction process is to process each multispectral image Image with background light Subtract to remove the influence of background ambient light. The calculation formula is as follows: The image segmentation method is to binarize the grayscale image through an adaptive threshold method, and finally extract the region of interest of the electronic component through a binary image template to obtain a multispectral image of the region of interest of the electronic component.

3. The method for detecting appearance defects of electronic components based on multispectral imaging according to claim 1, characterized in that: The spectral reflectance reconstruction method of the combined genetic algorithm and neural network specifically includes: collecting image response values ​​of standard color cards under different lighting conditions, and dividing them into a training set and a test set: Determine the structural parameters of the BP neural network: the input is a 24×17 response matrix, the output is a 24×601 spectral reflectance matrix, 601 wavelength values ​​are selected from the 400-1000nm band at 1nm intervals, and the number of hidden layer nodes is determined according to an empirical formula; Input the training set and response set of the standard color card into the neural network, set the initial weight, bias parameter and training target parameter of the neural network model, and build the BP neural network model: Genetic algorithm is used to calculate the global optimal weight and bias parameters to obtain the optimized BP neural network model; The sample prediction is performed through the obtained training set to obtain the trained BP neural network model; The trained BP neural network model is applied to the test color block for simulation prediction to obtain the spectral reflectance of the test color block and verify the reconstruction accuracy of the spectral reflectance.

4. The method for detecting appearance defects of electronic components based on multispectral imaging according to claim 1, characterized in that: The step of extracting the spatial features of the multispectral image of the region of interest specifically includes: The grayscale-gradient co-occurrence matrix method is used to extract image texture features, and the energy value, entropy value, contrast, and correlation are calculated to quantify the texture features of the sample, thereby describing the spatial characteristic attributes of the image texture, such as uniformity, complexity, clarity, and directionality.

5. The method for detecting appearance defects of electronic components based on multispectral imaging according to claim 1, characterized in that: The spectral reflectance feature and the spatial feature are fused, specifically, the spectral feature and the spatial feature are fused through a principal component analysis method to form a spectral-spatial feature data set which is input into an appearance defect detection module.

6. The method for detecting appearance defects of electronic components based on multispectral imaging according to claim 1, characterized in that: The trained appearance defect detection model is constructed using the least squares support vector machine method, specifically including: Construct a data set, set the input spectral data set consisting of N samples' spatial-spectral feature vectors and label data; Using nonlinear functions The input spectral data set consisting of the spatial-spectral feature vectors and label data of N samples is mapped to the high-dimensional feature space and the relationship model is established as follows: Among them, w represents the weight vector, b represents the bias value; The least squares support vector machine is used to solve the problem, and the function fitting problem is expressed as an optimization problem: in, represents the penalty coefficient, represents the error variable; Solving the model in the dual space, the fitting model of the least squares support vector machine is obtained as follows: in, is the kernel function, and b are model parameters, which are determined through the training process; and the nonlinear radial basis is selected as the kernel function of the support vector machine classifier.

7. An electronic component appearance defect detection system based on multispectral imaging, characterized in that: include: Light source module, imaging module, control module and data processing module; The light source module is composed of light emitting diodes of different emission spectrum types; The imaging module is composed of an imaging lens and a detector; The control module is used to control the light source driver and the data acquisition card of the industrial camera; The control module drives the light source module to perform time-sharing stroboscopic lighting, and synchronously triggers the imaging module to perform image acquisition, so as to obtain a data set of appearance images of the electronic components to be inspected; The data processing module performs image preprocessing on the image data set of the appearance of the electronic component to be detected, extracts the region of interest of the appearance of the electronic component to be detected, and obtains a multispectral image of the region of interest; Extracting the spectral reflectance features of the multispectral image of the region of interest by combining a genetic algorithm with a neural network spectral reflectance reconstruction method; By quantifying the image texture features, the spatial features of the multispectral image in the area of ​​interest are extracted; Fusing the spectral reflectance feature and the spatial feature to form a spatial-spectral joint feature data set of an area of ​​interest; Inputting the spatial-spectral joint feature data set of the region of interest into the trained appearance defect detection model to obtain the defect type and size of the appearance of the electronic component to be detected; Display the appearance defect detection results of the electronic components to be tested.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method described in any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 6 are performed.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images