A method for identifying defects in electronic components
Through edge detection technology and artificial intelligence in electronic component defect detection, noise reduction processing and feature matrix analysis of X-ray structure images are used to solve the problem of error detection caused by low image clarity of X-ray machines, and high-precision Automated detection.
Patent Information
- Application Number
- CN202411080774.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-08
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-08-08
AI Technical Summary
In the detection of internal defects of electronic components, the image captured by the X-ray machine is low in sharpness and is prone to false detection, making it difficult to accurately identify the assembly situation.
Edge detection technology and artificial intelligence are adopted to collect X-ray structure images of electronic components for noise reduction processing, image segmentation, significant point extraction, feature matrix calculation and neural network model analysis, to improve the accuracy of image recognition and realize automated detection.
It improves the accuracy of electronic components defect identification, reduces false detection and missed detection, realizes automated detection, and saves human resources.
Smart Images

Figure CN118608523B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image recognition, and in particular to a method for identifying defects of electronic components. Background Art
[0002] With the development of social economy and the advancement of industrialized society, artificial intelligence systems in the field of industrial production have developed rapidly, and the application of electronic products has increased accordingly. As important components of electronic products, the quality of electronic components directly affects the use effect of electronic products.
[0003] Electronic components are usually made up of many parts, and the components are small in size. It is easy for parts to be missed during assembly during the production process. In addition, it is difficult to obtain the internal assembly status of the components through RGB cameras. Therefore, X-ray machines are needed to collect internal information images of the components.
[0004] However, since the image resolution of X-ray machines is relatively low and the color information is relatively less rich, it is easy to make false detections during detection. Therefore, it is necessary to design a method for identifying defects in electronic components. Summary of the invention
[0005] The purpose of the present invention is to provide a method for identifying defects in electronic components, so as to improve the accuracy of image recognition through edge detection technology and realize automatic detection through artificial intelligence.
[0006] To achieve the above object, the present invention provides the following solutions:
[0007] A method for identifying defects in electronic components comprises the following steps:
[0008] Collecting structural images of electronic components and performing noise reduction processing on the structural images to obtain noise-reduced images;
[0009] The denoised image is divided into several segmented images by using image segmentation technology;
[0010] Extract salient points of all segmented images;
[0011] Calculate the feature matrix of all pixels in the segmented image respectively;
[0012] Obtain the eigenvectors of all feature matrices through the neural network model;
[0013] The similarity of the feature vectors is calculated. If the calculation result does not exceed the similarity threshold, it is considered that there are defects in the structural image corresponding to the feature vector.
[0014] Optionally, the specific method for performing noise reduction processing on the structural image is: performing filtering and noise reduction processing on the structural image through a Gaussian filter; the specific formula for filtering and noise reduction is: ;in , To denoise the image, is the structural image, x is the horizontal coordinate of the pixel in the image, y is the vertical coordinate of the pixel in the image, i is the increment of the horizontal axis, j is the increment of the ordinate, G ( x,y ) is the weight of the Gaussian filter, k is the size of the Gaussian filter.
[0015] Optionally, the image segmentation technology adopts a grayscale threshold segmentation method.
[0016] Optionally, extract the salient points of all segmented images, the specific steps are as follows:
[0017] The feature amplitude and feature direction of each pixel in the segmented image are calculated, the feature amplitudes of pixels with the same feature direction are compared, and the pixel with the largest feature amplitude is selected as the salient point.
[0018] Optionally, the characteristic amplitude is calculated as: ; Where M is the characteristic amplitude, They are the rates of change of pixels in eight directions; the eight directions of pixels are: horizontal left and right, vertical up and down, upper left, lower left, upper right and lower right.
[0019] Optionally, the characteristic direction is calculated as: ;in, is the angle value of the feature direction, is the horizontal direction parameter, is the vertical direction parameter.
[0020] Optionally, the feature matrix of all pixels in the segmented image is calculated separately, and the specific steps are: vector combination of unit vectors in 8 directions of the pixels is performed to obtain a feature matrix; the expression of the feature matrix is: A=(α1, α2, ...., α8); wherein A is the feature matrix, and α1~α8 are unit vectors in 8 directions respectively.
[0021] Optionally, the neural network model is a convolutional neural network model.
[0022] Optionally, the calculation formula for the similarity of feature vectors is:
[0023] ;
[0024] Among them, T is the similarity, is the target feature vector, is the nth eigenvector adjacent to the target eigenvector.
[0025] Optionally, the specific steps of the grayscale threshold segmentation method are:
[0026] Get the grayscale values of all pixels in the denoised image;
[0027] Count the frequency of occurrence of different gray values and generate a gray histogram;
[0028] The grayscale threshold is obtained by performing a weighted average operation on the grayscale value and the occurrence frequency in the grayscale histogram;
[0029] Compare all pixels with the grayscale threshold and select pixels whose grayscale values are higher than the grayscale threshold;
[0030] The segmented images of different sizes are divided with the selected pixels as the center.
[0031] According to the specific embodiment provided by the present invention, the present invention discloses the following technical effects: the electronic component defect recognition method provided by the present invention comprises: collecting X-ray structural images of electronic components, and performing noise reduction processing on the structural images to obtain noise-reduced images; dividing the noise-reduced images into several segmented images by image segmentation technology; extracting significant points of all segmented images; calculating the feature matrices of all significant points respectively; obtaining the feature vectors of all feature matrices by a neural network model; calculating the similarity of the feature vectors, and if the calculation result is higher than the similarity threshold, it is considered that there are defects in the structural image corresponding to the feature vector. The method improves the accuracy of image recognition by edge detection technology, and realizes automated detection by artificial intelligence. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0033] Figure 1 A flow chart of a method for identifying defects of electronic components according to an embodiment of the present invention;
[0034] Figure 2 A flow chart of the gray threshold segmentation method according to an embodiment of the present invention;
[0035] Figure 3 A flowchart of salient point extraction according to an embodiment of the present invention;
[0036] Figure 4 This is a characteristic amplitude and characteristic direction data diagram of an embodiment of the present invention. DETAILED DESCRIPTION
[0037] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0038] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and understandable, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.
[0039] like Figure 1 As shown, an embodiment of the present invention provides a method for identifying defects of electronic components, comprising the following steps:
[0040] Step 100: Capture a structural image of an electronic component, and perform noise reduction processing on the structural image to obtain a noise-reduced image.
[0041] Specifically, the Gaussian filtering method is used to reduce the noise of the structural image. The specific steps are as follows:
[0042] Define the Gaussian function in the Gaussian filtering method; the specific expression of the Gaussian function is:
[0043] ;
[0044] in, is a Gaussian function, is the standard deviation, x is the horizontal coordinate of the pixel in the image, y is the vertical coordinate of the pixel in the image, e The larger the standard deviation, the stronger the blurring effect of the Gaussian function on the image. In this embodiment, the standard deviation ranges from 1 to 3.
[0045] A Gaussian matrix is constructed according to the size of the Gaussian function and the Gaussian filter; the size of the Gaussian matrix in this embodiment is 5*5.
[0046] The pixel value of each pixel in the Gaussian matrix is calculated by the Gaussian function, and normalized to obtain the weight of the Gaussian filter; the calculation formula for calculating the pixel value of each pixel in the Gaussian matrix by the Gaussian function is:
[0047] ;
[0048] in, a is the row index of the Gaussian matrix, b is the column index of the Gaussian matrix, is the pixel value.
[0049] For each pixel, a weighted average calculation is performed on it and the pixel values in the surrounding neighborhood to obtain a new pixel value of the current pixel, and a denoised image is obtained based on the new pixel value; the specific calculation formula is:
[0050] ;
[0051] in , To denoise the image, is the structural image, i is the increment of the horizontal axis, j is the increment of the ordinate, k is the size of the Gaussian filter, G ( x,y ) is the weight of the Gaussian filter. The weight is determined according to the pixel value distribution of the Gaussian matrix. The farther away from the center pixel, the smaller the weight of the pixel. The Gaussian filter has the characteristics of a low-pass filter. It can remove high-frequency noise and detail information in the image, making the image smoother, and retain the main features of the original image (such as contours and edges) on the basis of the smoothed image. It is also separable, which can save computing resources and time, making it more efficient in computing. The blurring degree of the Gaussian filter can be controlled by adjusting the size and standard deviation of the Gaussian filter to meet different application requirements.
[0052] Step 200: Divide the denoised image into a number of segmented images using image segmentation technology.
[0053] Specifically, the image segmentation technology uses gray threshold segmentation method, such as Figure 2 As shown, the specific steps are as follows:
[0054] Step 201: Obtain the grayscale values of all pixels of the denoised image; the grayscale value calculation formula is: ;in, u is the red component of the pixel, p is the green component of the pixel, q The blue component of the pixel.
[0055] Step 202: Count the frequencies of occurrence of different grayscale values and generate a grayscale histogram.
[0056] Step 203: Obtain the grayscale threshold by performing a weighted average operation on the grayscale value and the occurrence frequency in the grayscale histogram; the specific calculation formula of the grayscale threshold is:
[0057] ;
[0058] Where D is the grayscale threshold, n is the number of different grayscale values, For the i The frequency of occurrence of gray values, For the i Gray value.
[0059] Step 204: Compare all pixels with the grayscale threshold, and filter out pixels with grayscale values higher than the grayscale threshold.
[0060] Step 205: Based on the difference between the grayscale value of the screened pixel point and the grayscale threshold, segmented images of different sizes are divided with different pixel points as the center.
[0061] Specifically, the difference between the gray value and the gray threshold is a gradient of 50. When the difference between the gray value of a pixel and the gray threshold is less than or equal to 50, the size of the segmented image centered on the pixel is 5*5; when the difference between the gray value of a pixel and the gray threshold is greater than 50 and less than or equal to 100, the size of the segmented image centered on the pixel is 8*8; when the difference between the gray value of a pixel and the gray threshold is greater than 100, the size of the segmented image centered on the pixel is 16*16. The gray threshold segmentation method divides the pixels of an image into different regions according to the gray value by setting the gray threshold. It does not require complex calculation processes and image processing algorithms, and has the advantages of simple calculation, high computational efficiency, and fast image processing speed.
[0062] Step 300: extract salient points of all segmented images.
[0063] Furthermore, the gradient is a vector (vector), which indicates that the directional derivative of a function at that point has the maximum value along that direction, that is, the function at that point changes fastest along that direction (the direction of this gradient) and the rate of change is the largest (the modulus of the gradient). In an image, the gradient calculates the magnitude of the image change. For the edge of the image, the grayscale value changes greatly, and the gradient value changes greatly; on the contrary, for the smoother part of the image, the grayscale value changes less, and the corresponding gradient value changes less. However, the image gradient is only obtained by calculating the rate of change of the X-axis and Y-axis, which lacks comprehensiveness in detecting the edge of the image.
[0064] In this embodiment, the change rates of the pixel points in multiple directions are considered and composed into feature amplitudes, the gradient amplitude in the image gradient is replaced by the feature amplitude, and the feature direction is obtained according to the direction parameters in the horizontal and vertical directions, and the feature amplitude and feature direction are used as the basis for extracting salient points.
[0065] like Figure 3 As shown, the specific steps are as follows:
[0066] Step 301: Calculate the characteristic amplitude of each pixel point of the segmented image.
[0067] Specifically, the characteristic amplitude is obtained by squaring the rate of change of the pixel in eight directions, and then taking the sum and square root; the eight directions of the pixel are the horizontal left and right direction, the vertical up and down direction, the upper left, the lower left, the upper right and the lower right direction of the pixel. The specific calculation formula of the characteristic amplitude is:
[0068] ;
[0069] Where M is the characteristic amplitude, They are the rates of change of pixels in eight directions; the eight directions of pixels are: horizontal left and right, vertical up and down, upper left, lower left, upper right and lower right.
[0070] Step 302: Calculate the characteristic direction of each pixel point of the segmented image.
[0071] Specifically, the feature direction is represented by the angle value of the vector, which is obtained by the inverse tangent transformation of the horizontal direction parameter and the vertical direction parameter of the pixel point. The specific calculation formula of the feature direction is:
[0072] ;
[0073] in, is the angle value of the feature direction, is the horizontal direction parameter, is the vertical direction parameter.
[0074] More specifically, the eight directions of the pixel point, namely the horizontal left and right direction, the vertical up and down direction, the upper left, lower left, upper right and lower right, correspond to eight angle values of 180°, 0°, 90°, 270°, 135°, 225°, 45° and 315° respectively. The angle value of the feature direction is subtracted from the eight angle values respectively, and the direction corresponding to the angle value with the smallest absolute value of the difference is the feature direction.
[0075] Step 303 includes: comparing the characteristic amplitudes of the pixels with the same characteristic direction, and selecting the pixel with the largest characteristic amplitude as the salient point.
[0076] Specifically, the embodiment of the present invention specifically illustrates a segmented image of 5*5, and the feature amplitude and feature direction are represented as follows: Figure 4 As shown, the value "2↑" of the upper left corner vertex represents a binary number pair "(2,90)", which means that the characteristic amplitude of the pixel point is 2, the angle value corresponding to the characteristic direction is 90°, and the characteristic direction is the vertical upward direction, and the rest of the values are similar.
[0077] More specifically, step 303 also includes: selecting the maximum value of the characteristic amplitude as a significant point; the specific steps are: comparing the characteristic amplitude of the pixel point with the characteristic amplitude of the same characteristic direction in the adjacent pixel points, if the characteristic amplitude of the pixel point is the maximum value within the adjacent neighborhood, it does not need to be the maximum value, and the pixel point is still selected as a significant point. Compared with the general image gradient method that only considers the rate of change in the horizontal and vertical directions of the image, the present invention adds consideration of the rate of change in the diagonal direction, making the extraction of significant points in the image more accurate and reducing the occurrence of false detection and missed detection.
[0078] Step 400: Calculate the feature matrices of all pixels in the segmented image respectively; the specific steps are: select 8 unit vectors in 8 directions of the pixels respectively, combine the 8 unit vectors to obtain the feature matrix, and collect all the feature matrices as input data of the neural network model.
[0079] Specifically, the expression of the feature matrix is: A=(α1,α2,....,α8);where A is the feature matrix, and α1~α8 are unit vectors in 8 directions respectively. Representing the segmented image through the feature matrix makes the feature representation of the segmented image more obvious. Using the feature matrix as the input data of the neural network model simplifies the format change operation of the neural network model on the segmented image, making subsequent operations more convenient and efficient.
[0080] Step 500: Obtain the eigenvectors of all eigenmatrices through a neural network model.
[0081] Specifically, the neural network model is a convolutional neural network model; the convolutional neural network model consists of multiple convolutional layers, pooling layers and fully connected layers.
[0082] More specifically, the convolution layer uses the output data of the previous layer as the input data of the next layer, and each layer extracts the features of the current layer by convolving the input data with the convolution kernel of the current layer. The convolution layer includes multiple convolution kernels, each of which is used to extract different features; the convolution kernel of the convolution layer is a small matrix that can slide on the input data and perform convolution operations with each local area of the input data; the convolution operation is used to convert each local area into a new feature, which is used for processing in the next layer. The convolution layer effectively reduces the number of parameters in the network and has translation invariance, which can maintain the ability to recognize features when the input data is translated.
[0083] The pooling layer uses the output data of the convolutional layer as input data and performs downsampling operations on it to reduce the spatial size of the output data, thereby reducing the dimension of the output data; the dimensionality reduction operation of the pooling layer reduces the number of parameters in the network and reduces the consumption of computing resources. The pooling layer uses maximum pooling, and the specific steps are: sliding a fixed-size window on the input data of the pooling layer, and then outputting the maximum value in the window as the result after downsampling.
[0084] Furthermore, when performing pooling operations in the pooling layer, the neural network may be too complex to easily remember every sample and its corresponding label in all input data, but it cannot accurately predict new data, resulting in overfitting. Therefore, a Dropout layer is added to the pooling layer; the Dropout layer has n neurons and 2 n The overfitting problem is solved by temporarily discarding multiple neurons and substructures from the neural network with a certain probability and averaging the weight results of the remaining neurons and substructures.
[0085] The fully connected layer is the last layer of the neural network. It obtains the neural network output by nonlinearly combining the features extracted by the convolution layer and the pooling layer, and maps the neural network output to the target output to obtain a feature vector.
[0086] Furthermore, each input neuron in the fully connected layer is connected to all neurons in the next layer, and each input neuron and each neuron in the next layer have corresponding weights, which control the influence of the input neuron on the neurons in the next layer.
[0087] Step 600: Calculate the similarity of the feature vectors. If the calculation result does not exceed the similarity threshold, it is considered that there is a defect in the structural image corresponding to the feature vector.
[0088] Specifically, the specific steps for calculating the similarity of feature vectors are: taking the feature vector of a certain salient point as the target feature vector, respectively calculating the cosine values of the target feature vector and the feature vector of the target salient point in eight directions, averaging the eight cosine values and combining them with the feature amplitude of the target salient point to obtain the similarity of the target feature vector. The specific calculation formula is:
[0089] ;
[0090] Among them, T is the similarity, is the target feature vector, is the nth eigenvector adjacent to the target eigenvector.
[0091] More specifically, the similarity threshold is 1.8. When the feature vector similarity is less than or equal to the similarity threshold, it is determined that there is a defect in the structural image corresponding to the feature vector; when the feature vector similarity is greater than the similarity threshold, it is determined that there is no defect in the structural image corresponding to the feature vector.
[0092] The beneficial effects of the present invention are as follows:
[0093] Improved the accuracy of image recognition;
[0094] Automated detection is achieved through artificial intelligence, saving manpower.
[0095] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.
[0096] The present invention uses specific examples to illustrate the principles and implementation methods of the present invention. The above examples are only used to help understand the method and core ideas of the present invention. At the same time, for those skilled in the art, according to the ideas of the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.
Claims
1. A method for identifying defects in electronic components, characterized in that: The steps include: Collecting a structural image of an electronic component, and performing noise reduction processing on the structural image to obtain a noise-reduced image; Dividing the denoised image into a plurality of segmented images by using an image segmentation technique; Extracting salient points of all the segmented images; Calculating feature matrices of all pixels in the segmented image respectively; Obtaining the eigenvectors of all the eigenmatrices through a neural network model; Calculating the similarity of the feature vectors. If the calculation result does not exceed the similarity threshold, it is considered that there is a defect in the structural image corresponding to the feature vector. When the feature vector similarity is greater than the similarity threshold, it is determined that there is no defect in the structural image corresponding to the feature vector. The image segmentation technology adopts grayscale threshold segmentation method; The specific steps of the grayscale threshold segmentation method are: Obtaining the grayscale values of all pixels of the denoised image; Count the frequency of occurrence of different gray values and generate a gray histogram; Obtaining a grayscale threshold by performing a weighted average operation on the grayscale value and the occurrence frequency in the grayscale histogram; Compare all the pixel points with the grayscale threshold, and filter out the pixel points whose grayscale values are higher than the grayscale threshold; Based on the difference between the gray value of the screened pixel point and the gray threshold, dividing the segmented image into different sizes with the screened pixel point as the center; Extract the salient points of all the segmented images, the specific steps are as follows: Calculating the characteristic amplitude and characteristic direction of each pixel point of the segmented image, comparing the characteristic amplitudes of the pixel points with the same characteristic direction, and selecting the pixel point with the largest characteristic amplitude as the salient point; The calculation formula of the characteristic amplitude is: ; Where M is the characteristic amplitude, ~ are the change rates of the pixels in eight directions respectively; the eight directions of the pixels are: horizontal left-right direction, vertical up-down direction, upper left, lower left, upper right and lower right; The calculation formula of the similarity of the feature vector is: ; Among them, T is the similarity, is the target feature vector, is the nth eigenvector adjacent to the target eigenvector.
2. The electronic component defect identification method according to claim 1, characterized in that: The specific method for performing noise reduction processing on the structural image is: performing filtering and noise reduction processing on the structural image through a Gaussian filter; the specific formula for filtering and noise reduction is: ;in , To denoise the image, is the structural image, x is the horizontal coordinate of the pixel in the image, y is the vertical coordinate of the pixel in the image, i is the increment of the horizontal axis, j is the increment of the ordinate, G ( x,y ) is the weight of the Gaussian filter, k is the size of the Gaussian filter.
3. The electronic component defect identification method according to claim 1, characterized in that: The calculation formula of the characteristic direction is: ;in, is the angle value of the feature direction, is the horizontal direction parameter, is the vertical direction parameter.
4. The electronic component defect identification method according to claim 1, characterized in that: The characteristic matrix of all the pixels in the segmented image is calculated respectively, and the specific steps are: the unit vectors in the eight directions of the pixels are combined to obtain the characteristic matrix; the expression of the characteristic matrix is: A=(α1, α2, ...., α8); Among them, A is the characteristic matrix, and α1~α8 are unit vectors in 8 directions respectively.
5. The electronic component defect identification method according to claim 1, characterized in that: The neural network model is a convolutional neural network model.
