SMT component defect detection method, device and equipment based on multi-algorithm fusion

Abstract

The application provides a kind of SMT component defect detection method, device and equipment based on multi-algorithm fusion, the surface optical image of target patch component area is generated solder joint area mask;According to the defect recognition center of solder joint area mask positioning, and with the benchmark of defect recognition center, the multiple identification auxiliary lines when identifying the defect of solder joint are radially outward isometric angle;According to the gray value of each identification auxiliary line corresponding pixel point, determine multiple gray value sequence;According to all gray value sequence, the defect discrimination feature map of solder joint is constructed, the gray consistency index of the preset center highlight area in defect discrimination feature map and the edge gradient index of solder joint contour area are determined;According to the defect discrimination feature vector determined by gray consistency index and edge gradient index, the defect type of solder joint is identified.The technical scheme provided by the application can realize the defect identification of the corresponding solder joint of target patch component under the abnormality of solder joint gray distribution.

Description

Technical Field

[0001] This application relates to the field of component defect detection technology, and more specifically, to a method, apparatus and equipment for SMT component defect detection based on multi-algorithm fusion. Background Technology

[0002] In fields such as electronics manufacturing, automotive parts, and precision instruments, the quality of components directly determines the reliability and safety of end products. Defects (such as cracks, deformation, missing materials, and stains) are the core causes of product failures. With the acceleration of automated production lines and the miniaturization of components (such as chip sizes being reduced to the nanometer level), the industry has placed higher demands on the accuracy (to the micrometer or even nanometer level), efficiency (to match production line speeds of several to tens of pieces per second), and stability of detection technologies. With the rapid development of automated defect detection solutions based on technologies such as machine vision, deep learning, X-ray imaging, and infrared thermal imaging, strong support has been provided for the detection of component defects.

[0003] In existing component defect detection, high-resolution cameras and light source systems are used to acquire surface / appearance images of components. After image preprocessing (denoising and enhancement), features such as grayscale, texture, and contour are extracted. Then, traditional algorithms (such as edge detection and template matching) or deep learning models (such as CNN) are used to compare with standard templates to identify appearance defects such as cracks, missing corners, and stains. However, in the defect detection of target surface mount components, factors such as differences in solder paste printing thickness and uneven heat capacity distribution during reflow soldering can cause nonlinear changes in the grayscale distribution of the center and edge areas of the solder joint. Traditional SMT solder joint defect detection methods rely on a single threshold or global features and are unable to identify the abnormal grayscale distribution caused by uneven solder paste printing and reflow soldering thermal field fluctuations. This leads to missed defects in the corresponding solder joints of the target surface mount components. Therefore, how to identify defects in the corresponding solder joints of target surface mount components under abnormal grayscale distribution has become a challenge for the industry. Summary of the Invention

[0004] This application provides a method, apparatus, and equipment for SMT component defect detection based on multi-algorithm fusion, which can identify defects in the corresponding solder joints of target surface mount components under abnormal solder joint grayscale distribution.

[0005] In a first aspect, this application provides a method for detecting defects in SMT components based on multi-algorithm fusion, comprising the following steps:

[0006] Acquire a surface optical image of the target surface mount component area on the printed circuit board, and generate a solder joint area mask based on the solder joint outline of the corresponding solder joint of the target surface mount component in the surface optical image;

[0007] Based on the solder joint area mask, the defect identification center of the corresponding solder joint of the target patch element is located from the surface optical image, and multiple identification auxiliary lines are radiated outward at equal angles from the defect identification center to identify defects in the corresponding solder joint of the target patch element.

[0008] For each identification auxiliary line, the gray value of the corresponding pixel in the surface optical image is extracted from the defect identification center toward the weld point boundary to obtain multiple gray value sequences that characterize the gray value distribution of a single weld point surface.

[0009] Based on all the grayscale value sequences, a defect discrimination feature map of the solder joint corresponding to the target patch element is constructed. The grayscale consistency index of the central bright area and the edge gradient index of the solder joint contour area are determined in the defect discrimination feature map. The edge gradient index is a set of quantitative parameters that characterize the intensity and direction distribution of grayscale changes in the solder joint contour area.

[0010] Based on the grayscale consistency index and the edge gradient index, the defect discrimination feature vector of the solder joint corresponding to the target patch element is determined, and the defect type of the solder joint corresponding to the target patch element is identified based on the defect discrimination feature vector.

[0011] In some embodiments, generating a solder joint region mask body based on the solder joint contour of the solder joint corresponding to the solder joint of the target patch element in the surface optical image includes:

[0012] Extract the solder joint contour of the solder joint corresponding to the target patch element from the surface optical image;

[0013] Based on the solder joint contour, determine the set of boundary pixels of the solder joint region corresponding to the target patch element;

[0014] A solder joint area mask containing the complete solder joint area is generated using the boundary pixel set.

[0015] In some embodiments, locating the defect identification center of the solder joint corresponding to the target patch element from the surface optical image based on the solder joint area mask specifically includes:

[0016] Perform connectivity analysis on the solder joint area mask to determine the minimum bounding rectangle of the solder joint area corresponding to the target surface mount component;

[0017] The geometric center coordinates of the minimum bounding rectangle are calculated as the defect identification center of the corresponding solder joint of the target patch element.

[0018] In some embodiments, the multiple identification auxiliary lines radiating outward at equal angles from the defect identification center to identify defects in the corresponding solder joints of the target surface mount component specifically include:

[0019] Extract the geometric center coordinates of the defect identification center as the ray starting point;

[0020] The angular interval between adjacent rays is calculated based on the preset number of rays to determine the radiation direction angle of each ray;

[0021] Using the starting point as the origin, generate initial ray paths extending outward from the defect identification center according to each radiation direction angle;

[0022] Based on the solder joint area mask, the initial ray path is truncated at the solder joint outline boundary, thereby obtaining multiple identification auxiliary lines for defect identification of the corresponding solder joint of the target patch element.

[0023] In some embodiments, for each identification auxiliary line, extracting the grayscale value of the corresponding pixel in the surface optical image from the defect identification center toward the weld point boundary to obtain multiple grayscale value sequences characterizing the grayscale distribution of a single weld point surface specifically includes:

[0024] Each identification auxiliary line is sampled for pixel points to obtain the coordinates of all pixel points from the defect identification center to the weld point boundary.

[0025] Extract the grayscale values ​​corresponding to the coordinates of each pixel from the surface optical image to form the original grayscale data;

[0026] The original grayscale data is arranged according to the order of pixel points on the recognition auxiliary line to obtain multiple grayscale value sequences that characterize the grayscale distribution of a single solder joint surface.

[0027] In some embodiments, constructing a defect discrimination feature map of the solder joint corresponding to the target patch element based on all grayscale value sequences specifically includes:

[0028] A polar coordinate system is constructed with the geometric center of the defect identification center as the pole and the radiation direction angle of each identification auxiliary line as the polar angle.

[0029] Map the position of the gray value in each gray value sequence on the corresponding recognition auxiliary line to the polar radius coordinates in the polar coordinate system;

[0030] Based on all polar angle and polar radius coordinates and their corresponding grayscale values ​​in the polar coordinate system, a defect discrimination feature map of the solder joint corresponding to the target patch element is generated.

[0031] In some embodiments, surface optical images of the target surface mount component area on a printed circuit board are acquired using an industrial camera.

[0032] Secondly, this application provides a target surface mount component defect detection device based on multi-algorithm fusion, used to execute an SMT component defect detection method based on multi-algorithm fusion, comprising a surface mount component defect detection unit, the surface mount component defect detection unit comprising:

[0033] The acquisition module is used to acquire surface optical images of the target surface mount component area on the printed circuit board, and generate a solder joint area mask based on the solder joint contour of the corresponding solder joint of the target surface mount component in the surface optical image.

[0034] The processing module is used to locate the defect identification center of the corresponding solder joint of the target patch element from the surface optical image based on the solder joint area mask, and to radiate multiple identification auxiliary lines outward at equal angles from the defect identification center to identify defects in the corresponding solder joint of the target patch element.

[0035] The processing module is also used to extract the gray values ​​of the corresponding pixels in the surface optical image for each identification auxiliary line, starting from the defect identification center and moving towards the boundary direction, to obtain multiple gray value sequences that characterize the gray value distribution of a single solder joint surface.

[0036] The processing module is also used to construct a defect discrimination feature map of the solder joint corresponding to the target patch element based on all gray value sequences, and to determine the gray consistency index of the preset central highlight area and the edge gradient index of the solder joint outline area in the defect discrimination feature map. The edge gradient index is a set of quantitative parameters that characterize the intensity and direction distribution of gray value changes in the solder joint outline area.

[0037] The execution module is used to determine the defect discrimination feature vector of the solder joint corresponding to the target patch element based on the grayscale consistency index and the edge gradient index, and to identify the defect type of the solder joint corresponding to the target patch element based on the defect discrimination feature vector.

[0038] Thirdly, this application provides a computer device, the computer device including a memory and a processor, the memory storing code, and the processor being configured to acquire the code and execute the above-described SMT component defect detection method based on multi-algorithm fusion.

[0039] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described SMT component defect detection method based on multi-algorithm fusion.

[0040] The technical solutions provided by the embodiments disclosed in this application have the following beneficial effects:

[0041] In the SMT component defect detection method, apparatus and equipment based on multi-algorithm fusion provided in this application, firstly...

[0042] Therefore, this application can identify defects in the solder joints of target surface mount components even under abnormal grayscale distribution. First, by acquiring surface optical images and generating a solder joint area mask, the target solder joint area can be accurately isolated from the background, eliminating non-solder joint interference and providing a clean analysis object for subsequent inspection. Second, by locating the defect identification center and radiating equiangular identification auxiliary lines, a detection benchmark is established to ensure analytical symmetry, and uniform coverage ensures no omission of solder joint features, improving the comprehensiveness of feature extraction. Furthermore, grayscale value sequences are extracted along the identification auxiliary lines, and the spatial grayscale distribution is transformed into an ordered data chain, intuitively presenting the grayscale change pattern from the center to the edge, providing a structured data source for defect discrimination feature quantification. Finally, a defect discrimination feature map is constructed, and grayscale consistency and edge gradient indices are extracted. By mapping the solder joint morphology of the target surface mount component to polar coordinates, two types of indicators focus on the core and edge regions of the corresponding solder joint, respectively, achieving precise quantification of defect features. This provides multi-dimensional basis for identifying defects in the corresponding solder joint, avoiding missed defects caused by uneven solder paste printing and abnormal grayscale distribution due to reflow soldering thermal field fluctuations. Finally, the defect discrimination feature vector of the corresponding solder joint is determined based on the grayscale consistency index and the edge gradient index. Based on the defect discrimination feature vector, the defect type of the corresponding solder joint is identified, ultimately achieving accurate detection of defects in the corresponding solder joint. In summary, the technical solution provided in this application can identify defects in the corresponding solder joint of the target surface mount component even under abnormal solder joint grayscale distribution. Attached Figure Description

[0043] Figure 1 This is an exemplary flowchart of a multi-algorithm fusion-based SMT component defect detection method according to some embodiments of this application;

[0044] Figure 2 This is an exemplary flowchart illustrating the determination of a defect identification center according to some embodiments of this application;

[0045] Figure 3 This is a schematic diagram of the structure of a patch element defect detection unit according to some embodiments of this application;

[0046] Figure 4 This is a schematic diagram of the structure of a computer device that implements a multi-algorithm fusion-based SMT component defect detection method according to some embodiments of this application. Detailed Implementation

[0047] To better understand the technical solution of this application, the technical solution of this application will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0048] refer to Figure 1 The figure is an exemplary flowchart of a multi-algorithm fusion-based SMT component defect detection method according to some embodiments of this application. The figure mainly includes the following steps:

[0049] In step S101, a surface optical image of the target surface mount component area on the printed circuit board is acquired, and a solder joint area mask is generated based on the solder joint contour of the corresponding solder joint of the target surface mount component in the surface optical image.

[0050] In practice, an industrial camera can be used to acquire surface optical images of the target surface mount component area on the printed circuit board. Alternatively, other acquisition devices can be used to acquire surface optical images of the target surface mount component area on the printed circuit board. This is not limited here. The target surface mount component area refers to the surface mount component area that needs to be defect-detected.

[0051] It should be noted that the surface optical image in this application refers to an image obtained by taking pictures of the target surface mount component. It is a two-dimensional digital image used to reflect the surface morphology, gloss and color distribution of the solder joints of the target surface mount component on the printed circuit board. The gray value of each pixel in the surface optical image not only contains the material color information of the solder joint itself (such as the bright white of the solder and the dark gray after oxidation), but more importantly, it encodes the gloss changes and brightness differences caused by its three-dimensional geometric features (such as flatness, depression, warping or solder balls). This provides the most fundamental data basis for subsequent identification of defects by analyzing the gray distribution and gradient characteristics of the solder joint through algorithms.

[0052] In some embodiments, generating a solder joint region mask body based on the solder joint contour of the solder joint corresponding to the solder joint of the target patch element in the surface optical image is achieved by the following steps:

[0053] Extract the solder joint contour of the solder joint corresponding to the target patch element from the surface optical image;

[0054] Based on the solder joint contour, determine the set of boundary pixels of the solder joint region corresponding to the target patch element;

[0055] A solder joint area mask containing the complete solder joint area is generated using the boundary pixel set.

[0056] In specific implementation, firstly, considering the pixel grayscale difference between the solder joint and background regions in the surface optical image, the Otsu thresholding algorithm is used to traverse all possible grayscale thresholds in the surface optical image, calculate the inter-class variance between the foreground (solder joint) and background regions under different grayscale thresholds, and select the threshold with the largest inter-class variance as the optimal segmentation threshold. Then, based on the optimal segmentation threshold, the solder joint contour corresponding to the solder joint of the target patch element is separated from the surface optical image. The solder joint contour refers to the closed line formed by the edge pixels of the solder joint corresponding to the target patch element in the surface optical image. Secondly, the coordinates of each edge pixel on the solder joint contour are extracted using the image processing software OpenCV to form the boundary pixel set of the solder joint region corresponding to the target patch element. The boundary pixel set refers to the coordinate data set of all pixels constituting the solder joint contour, used to describe the solder joint region in the image. The spatial boundary position is determined; finally, a seed filling algorithm (such as the four-connected region filling method) is used to select a pixel as a seed point inside the closed region enclosed by the set of boundary pixels. Then, it is determined whether the pixels in the four directions (up, down, left, right) adjacent to the seed point belong to the background pixels and are located in the region enclosed by the set of boundary pixels. If they are satisfied, the corresponding adjacent pixels are marked as foreground pixels and used as new seed points. This process is repeated until all pixels in the region enclosed by the set of boundary pixels are marked as foreground pixels. Finally, a binary image containing only the complete solder joint region (foreground pixels) and the background region (background pixels) is generated. The obtained binary image is used as a solder joint region mask containing the complete solder joint region. The seed filling algorithm is an image processing technique that achieves region filling by starting from a specified seed point and gradually marking all target pixels in the closed region.

[0057] It should be noted that the solder joint area mask in this application refers to a binary image template used to shield the background area of ​​the image and retain only the solder joint area. By determining the solder joint area mask, accurate regional constraints and interference shielding are provided for the detection of solder joint defects of surface mount components. This ensures that subsequent operations such as defect identification center positioning, defect identification ray generation, and grayscale value extraction are carried out only around the solder joint target, reducing redundant data processing and improving defect detection efficiency.

[0058] In step S102, the defect identification center of the corresponding solder joint of the target patch element is located from the surface optical image according to the solder joint area mask, and multiple identification auxiliary lines are radiated outward at equal angles from the defect identification center to identify defects in the corresponding solder joint of the target patch element.

[0059] In some embodiments, reference Figure 2As shown, this figure is an exemplary flowchart illustrating the determination of a defect identification center according to some embodiments of this application. In this embodiment, the defect identification center of the corresponding solder joint of the target patch element can be located from the surface optical image based on the solder joint area mask using the following steps:

[0060] In step S1021, a connectivity analysis is performed on the solder joint area mask to determine the minimum bounding rectangle of the solder joint area corresponding to the target surface mount component.

[0061] In step S1022, the geometric center coordinates of the minimum bounding rectangle are calculated as the defect identification center of the solder joint corresponding to the target patch element.

[0062] In specific implementation, firstly, a connected component labeling algorithm based on neighborhood search (such as the 8-connectivity labeling method) is used to traverse all pixels in the solder joint region mask. Adjacent pixels with the attribute of foreground (solder joint) are labeled as the same connected component, thus obtaining the solder joint region corresponding to the target patch element. The maximum x-coordinate, minimum x-coordinate, maximum y-coordinate, and minimum y-coordinate of the solder joint region in the image coordinate system are extracted. The region obtained by connecting the maximum x-coordinate, minimum x-coordinate, maximum y-coordinate, and minimum y-coordinate is used as the minimum bounding rectangle of the solder joint region corresponding to the target patch element. The minimum bounding rectangle is a rectangle that can completely contain the entire solder joint region. Then, the coordinates of the intersection of the two diagonals of the minimum bounding rectangle are calculated to obtain the geometric center coordinates of the minimum bounding rectangle. The geometric center coordinates are used as the defect identification center of the solder joint corresponding to the target patch element.

[0063] It should be noted that, in this application, the defect identification center refers to the center point used to identify the corresponding solder joint defects of the target patch. The determination of the defect identification center provides an accurate analysis benchmark and feature extraction starting point for the detection of solder joint defects of patch components. The defect identification center can be used as the reference origin of the defect identification ray radiation to ensure that the identification auxiliary line when performing defect detection on the patch component can be uniformly covered from the center of the solder joint to the boundary, avoiding grayscale sequence extraction deviation caused by benchmark offset.

[0064] In some embodiments, the multiple identification auxiliary lines radiating outward at equal angles from the defect identification center to identify defects in the corresponding solder joints of the target surface mount component are specifically implemented using the following steps:

[0065] Extract the geometric center coordinates of the defect identification center as the ray starting point;

[0066] The angular interval between adjacent rays is calculated based on the preset number of rays to determine the radiation direction angle of each ray;

[0067] Using the starting point as the origin, generate initial ray paths extending outward from the defect identification center according to each radiation direction angle;

[0068] Based on the solder joint area mask, the initial ray path is truncated at the solder joint outline boundary, thereby obtaining multiple identification auxiliary lines for defect identification of the corresponding solder joint of the target patch element.

[0069] In specific implementation, firstly, the geometric center coordinates of the defect identification center are extracted as the ray starting point. The ray starting point refers to the common starting point of all identification auxiliary lines. The position of the ray starting point directly determines the coverage and distribution symmetry of the ray within the weld joint area. Secondly, based on the preset total number of rays according to the detection requirements (for example, it can be set to 24 lines, or other numbers can be set according to actual requirements, which is not limited here), the angle interval between two adjacent rays (i.e., the ratio of 360 degrees to the total number of rays) is calculated using the angle equalization algorithm. Then, with the horizontal rightward direction of the image coordinate system as the 0-degree reference, the angle between each ray and the 0-degree reference is determined sequentially according to the angle interval. This angle is the radiation direction angle of the corresponding ray. The radiation direction angle refers to the angle between the identification auxiliary line and the reference direction in the image coordinate system. Angle is a key parameter for determining the direction of ray extension. Then, with the starting point as the origin, an initial ray path extending outward from the defect identification center is generated according to each radiation direction angle. The initial ray path refers to the ray extension trajectory generated by the starting point and the radiation direction angle. Finally, each pixel on the initial ray path is traversed to determine whether the pixel belongs to the background area of ​​the solder joint area mask (i.e., beyond the solder joint outline boundary). When the first background pixel is detected, the previous pixel of the background pixel is taken as the termination point of the ray, and the initial ray path is truncated. The final pixel trajectory from the starting point to the termination point is the identification auxiliary line when performing defect identification on the corresponding solder joint of the target patch element, thus obtaining multiple identification auxiliary lines when performing defect identification on the corresponding solder joint of the target patch element.

[0070] It should be noted that the identification auxiliary lines in this application refer to auxiliary lines used for defect detection of corresponding solder joints of target surface mount components. Existing technologies often use fixed directions (such as horizontal or vertical) or random directions to generate rays, which are prone to missing grayscale features in areas such as solder joint edges and corners due to uneven directional coverage. Moreover, it often requires additional processing of redundant data where rays extend beyond the solder joint boundary. In contrast, this solution uses the geometric center of the defect identification center as the starting point and determines the radiation direction angle based on the preset number of rays, ensuring that the rays uniformly cover the entire solder joint area within a 360° range. This avoids the problem of missing features in local areas. At the same time, it directly performs boundary boundary processing on the initial ray path based on the solder joint area mask. The truncation method, which eliminates the need for additional data filtering steps, significantly improves the efficiency and accuracy of ray generation. It innovatively combines "uniform distribution at equal angles" with "adaptive truncation of mask boundaries." The equal-angle design ensures the comprehensiveness and symmetry of grayscale feature sampling on the solder joint surface, while the mask truncation achieves precise matching between the rays and the actual contour of the solder joint. This allows each ray to fully reflect the grayscale change process from the center to the edge of the solder joint, providing an ordered and non-redundant feature data source for the subsequent construction of a two-dimensional grayscale distribution map. This effectively solves the pain points of uneven ray coverage and redundant data interference in existing technologies, and improves the accuracy and reliability of subsequent defect identification.

[0071] In step S103, for each identification auxiliary line, the gray value of the corresponding pixel in the surface optical image is extracted from the defect identification center toward the weld point boundary to obtain multiple gray value sequences characterizing the gray value distribution of a single weld point surface.

[0072] In some embodiments, for each identification auxiliary line, the grayscale value of the corresponding pixel in the surface optical image is extracted from the defect identification center toward the weld point boundary to obtain multiple grayscale value sequences characterizing the grayscale distribution of a single weld point surface. This is specifically achieved through the following steps:

[0073] Each identification auxiliary line is sampled for pixel points to obtain the coordinates of all pixel points from the defect identification center to the weld point boundary.

[0074] Extract the grayscale values ​​corresponding to the coordinates of each pixel from the surface optical image to form the original grayscale data;

[0075] The original grayscale data is arranged according to the order of pixel points on the recognition auxiliary line to obtain multiple grayscale value sequences that characterize the grayscale distribution of a single solder joint surface.

[0076] In specific implementation, firstly, for each identification auxiliary line, starting from the starting point of the identification auxiliary line (i.e., the geometric center of the defect identification center), the horizontal and vertical coordinates of each pixel in the surface optical image are recorded sequentially along the extension direction of the identification auxiliary line until the ending point of the identification auxiliary line (i.e., the boundary of the solder joint contour). The coordinates of all the recorded coordinates together constitute the coordinates of all pixels from the defect identification center to the boundary. The pixel coordinates refer to the position parameters of the pixels in the image coordinate system. Secondly, the gray values ​​corresponding to the coordinates of each pixel are extracted from the surface optical image to form the original gray value data. The original gray value data refers to the set of pixel gray values ​​extracted from the surface optical image, which is the basic data reflecting the gray value characteristics of the solder joint surface. Finally, the original gray value data is arranged according to the arrangement order of the pixels on the identification auxiliary line to obtain the gray value sequence corresponding to each identification auxiliary line, and thus obtain multiple gray value sequences representing the gray value distribution of a single solder joint surface.

[0077] It should be noted that, in this application, the gray value sequence refers to the set of gray values ​​arranged in order of the position of the pixels on the identification guideline. The determination of the gray value sequence is the core link connecting the identification guideline and the gray feature analysis of the solder joint. By extracting and serializing the pixel gray values ​​in order from the center to the boundary of the identification guideline, the gray information of the area covered by each ray is transformed into ordered one-dimensional data, which intuitively presents the gray change pattern of the solder joint from the core to the edge, and provides accurate raw data support for the subsequent construction of defect discrimination feature map.

[0078] In step S104, a defect discrimination feature map of the solder joint corresponding to the target patch element is constructed based on all gray value sequences. The gray consistency index of the central highlight area and the edge gradient index of the solder joint outline area are determined in the defect discrimination feature map. The edge gradient index is a set of quantitative parameters that characterize the intensity and direction distribution of gray value changes in the solder joint outline area.

[0079] In some embodiments, constructing a defect discrimination feature map of the solder joint corresponding to the target patch element based on all grayscale value sequences is specifically achieved through the following steps:

[0080] A polar coordinate system is constructed with the geometric center of the defect identification center as the pole and the radiation direction angle of each identification auxiliary line as the polar angle.

[0081] Map the position of the gray value in each gray value sequence on the corresponding recognition auxiliary line to the polar radius coordinates in the polar coordinate system;

[0082] Based on all polar angle and polar radius coordinates and their corresponding grayscale values ​​in the polar coordinate system, a defect discrimination feature map of the solder joint corresponding to the target patch element is generated.

[0083] In specific implementation, firstly, based on the conversion rules between the image coordinate system and the polar coordinate system, the geometric center of the defect recognition center is set as the pole (i.e., the origin) of the polar coordinate system. Then, the radiation direction angle of each recognition auxiliary line (i.e., the angle with the horizontal reference direction of the image) is used as the corresponding polar angle in the polar coordinate system. The polar angle is used to define the orientation of each gray value sequence in the polar coordinate system, thereby constructing a polar coordinate system that matches the spatial shape of the solder joint area. The polar coordinate system refers to the framework that transforms the direction and position information of the recognition auxiliary lines into unified spatial coordinates. Secondly, for each gray value... The grayscale value sequence is used to convert the position of each grayscale value on the recognition auxiliary line (i.e., the distance from the starting point to the pixel) into polar radius coordinates in the polar coordinate system, so that each grayscale value corresponds to a unique set of (polar angle, polar radius) coordinates. The polar radius coordinates refer to the straight-line distance from the pole to the target point in the polar coordinate system. Finally, a two-dimensional grid is constructed with the polar angle as the horizontal dimension and the polar radius as the vertical dimension. The grayscale value corresponding to each (polar angle, polar radius) coordinate is used as the pixel value of the network node in the two-dimensional network to generate a defect discrimination feature map of the solder joint corresponding to the target patch element.

[0084] It should be noted that the defect discrimination feature map in this application refers to the image that presents the gray-scale distribution characteristics of the solder joint in polar coordinates when performing defect discrimination on the corresponding solder joint of the target surface mount component. Existing technologies mostly use a rectangular coordinate system to construct the gray-scale feature map of the solder joint, which is prone to distortion of the coordinate mapping of the edge area due to the irregular shape of the solder joint, and it is difficult to intuitively reflect the directional differences in gray-scale distribution, thus affecting the accurate capture of defect features. This solution innovatively constructs a polar coordinate system with the geometric center of the solder joint defect identification center as the pole and the radiation direction angle of the identification auxiliary ray as the polar angle. The gray-scale value sequence is mapped according to the "polar angle-polar radius" correspondence, which not only fits the actual shape of the solder joint, which is mostly circular or near-circular, but also avoids the shape adaptation deviation under the rectangular coordinate system. It can also directly associate the gray-scale distribution with the ray direction through the polar angle dimension, clearly presenting the gray-scale change differences in different directions (such as the gray-scale distribution in a certain polar angle direction). The abrupt change in gray level can directly correspond to the edge defect of the weld point in the corresponding direction. The generated feature map can intuitively reflect the symmetry, uniformity and local anomalies of the gray level of the entire weld point. For example, the gray level consistency of the central bright area can be directly observed through the gray level fluctuation of different polar angles under the same polar diameter. The gray level gradient of the contour area can be clearly presented through the gray level change rate in the direction of increasing polar diameter. It innovatively integrates "the direction and position information of the auxiliary ray" with "the spatial representation advantage of the polar coordinate system", so that the feature map is not only a visualization of gray level data, but also a carrier for the correlation between defect direction, position and gray level anomaly. It solves the problem of the disconnect between gray level distribution and the spatial shape and directional features of the weld point in the existing technology. It provides a feature carrier that is more in line with the defect detection needs for the accurate calculation of subsequent gray level consistency index and gray level gradient distribution index, and greatly improves the pertinence and accuracy of defect feature extraction.

[0085] In some embodiments, determining the grayscale consistency index of the preset central highlight region and the edge gradient index of the solder joint contour region in the defect discrimination feature map is specifically achieved through the following steps:

[0086] In the defect discrimination feature map, a central bright area with a preset radius is delineated, and the standard deviation and coefficient of variation of all pixel gray values ​​within the central bright area are calculated as gray-scale consistency indicators.

[0087] Extract the pixel grayscale values ​​of the solder joint outline region in the defect discrimination feature map, and calculate the gradient magnitude and gradient direction of each pixel in the solder joint outline region using the Sobel operator;

[0088] Calculate the mean gradient magnitude, variance of gradient magnitude, and gradient direction distribution entropy of the solder joint contour region, and combine the mean gradient magnitude, variance of gradient magnitude, and gradient direction distribution entropy into an edge gradient index of the solder joint contour region.

[0089] In specific implementation, firstly, in the polar coordinate system of the defect discrimination feature map, based on a preset radius range (i.e., the average size of the corresponding solder joints of the same type of patch element as the radius range), a central highlight area is delineated in the defect discrimination feature map with the origin of the polar coordinate system corresponding to the defect discrimination feature map as the center and the radius range. Then, the standard deviation (reflecting the degree of deviation of gray values ​​from the mean) and coefficient of variation (i.e., the ratio of the standard deviation of pixel gray values ​​to the mean of pixel gray values) of all pixel gray values ​​in the central highlight area are calculated. The two calculated quantization parameters are used as gray-level consistency indicators. The gray-level consistency indicators refer to the set of quantization parameters used to characterize the uniformity of pixel gray-level distribution in the central highlight area. Secondly, the solder joint contour area (i.e., the pixel area of ​​the surface optical image with the polar radius within the maximum radius range of the solder joint) is located in the defect discrimination feature map, and the gray values ​​of all pixels in the solder joint contour area are extracted. Then, the Sobel operator is used to construct convolution kernels in the horizontal and vertical directions, and the convolution kernels are... The kernel performs a convolution operation on the grayscale image corresponding to the solder joint contour region, calculates the gradient components of each pixel in the horizontal and vertical directions, obtains the gradient magnitude of each pixel through square root operation, and obtains the gradient direction of each pixel through arctangent operation. The gradient magnitude and gradient direction refer to the quantized values ​​that represent the intensity and direction of grayscale change of the pixel, respectively. Finally, the mean and variance of the gradient magnitude of the solder joint contour region are calculated. At the same time, the gradient direction is divided into multiple intervals through the information entropy calculation method (specifically, the gradient direction is divided into 12 intervals with equal intervals of 30° in the full angular range of 0°~360°. In addition, it can be set according to actual needs, which is not limited here), and the pixel ratio of each interval is counted. The pixel ratio of each interval is then used as the input variable to the information entropy. The result of the information entropy is used as the gradient direction distribution entropy. The mean gradient magnitude, the variance of the gradient magnitude, and the gradient direction distribution entropy are combined to form the edge gradient index of the solder joint contour region.

[0090] It should be noted that the edge gradient index in this application refers to a set of quantitative parameters used to characterize the intensity and direction distribution of grayscale changes in the weld point contour area. It can reflect the integrity and regularity of the contour edge. By determining the grayscale consistency index of the central bright area in the defect discrimination feature map and the edge gradient index of the weld point contour area, the defect discrimination feature map can be transformed into the core link of defect discrimination quantification. The two work together to provide accurate feature support for subsequent defect identification.

[0091] In step S105, the defect discrimination feature vector of the solder joint corresponding to the target patch element is determined according to the grayscale consistency index and the edge gradient index, and the defect type of the solder joint corresponding to the target patch element is identified based on the defect discrimination feature vector.

[0092] In some embodiments, determining the defect discrimination feature vector of the solder joint corresponding to the target patch element based on the grayscale consistency index and the edge gradient index is specifically achieved through the following steps:

[0093] The grayscale consistency index and the edge gradient index are normalized to obtain normalized feature parameters;

[0094] Principal component analysis is used to extract key feature components from the normalized feature parameters for defect detection of solder joints corresponding to target patch components.

[0095] The key feature components are weighted and fused according to preset weights to form a defect discrimination feature vector of the solder joint corresponding to the target patch element.

[0096] In specific implementation, firstly, minimum-maximum normalization is used to map the values ​​of the gray-scale consistency index (standard deviation, coefficient of variation) and the edge gradient index (mean gradient magnitude, variance, directional distribution entropy) to the [0,1] interval, respectively. This is achieved by calculating "(original index value - minimum index value) / (maximum index value - minimum index value)" to eliminate the influence of differences in dimensions and numerical ranges among different indicators. The normalized gray-scale consistency index and the edge gradient index are then combined into normalized feature parameters, which refer to a set of feature indicators within a unified numerical range. Secondly, principal component analysis is used to construct a covariance matrix for the normalized feature parameters. The solution to the covariance matrix is ​​then performed... The eigenvalues ​​and eigenvectors are used to select the two eigenvectors with the largest eigenvalues ​​as principal components. Then, normalized feature parameters are projected onto the principal components to obtain the key feature components for defect detection of the solder joints corresponding to the target surface mount component. The key feature components refer to feature parameters that can centrally reflect the core information of the original indicators and play a dominant role in defect discrimination. Finally, corresponding weights (set between 0 and 1) are set according to the importance of different key feature components in defect identification of the solder joints corresponding to the target surface mount component. The weights can be set according to actual needs and are not limited here. A weighted summation algorithm is used to multiply each key feature component with its corresponding weight and then sum them to form the defect discrimination feature vector of the solder joints corresponding to the target surface mount component.

[0097] It should be noted that the defect discrimination feature vector in this application refers to the feature used to identify the defect type of the corresponding solder joint of the target patch component. The determination of the defect discrimination feature vector is the key step in transforming the multi-dimensional quality features of the solder joint into a unified discrimination criterion. The defect discrimination feature vector can serve as a standardized input for the defect identification model, and can transform complex image features into quantitative data that the model can directly process, realizing the transformation from multi-source features to a unified discrimination criterion, and providing an efficient and comprehensive feature carrier for accurately identifying various defects of solder joints.

[0098] In some embodiments, identifying the defect type of the solder joint corresponding to the target patch element based on the defect discrimination feature vector is specifically achieved through the following steps:

[0099] The defect discrimination feature vector is input into a pre-trained multi-class support vector machine model to obtain the discrimination probability of each defect type.

[0100] The defect category with the highest discrimination probability is selected as the defect type of the corresponding solder joint of the target surface mount component.

[0101] In specific implementation, firstly, the defect discrimination feature vector is input into a pre-trained multi-class support vector machine model (this multi-class support vector machine model is trained by using the defect discrimination feature vectors of historical defect samples and the corresponding defect type labels, and uses a kernel function to map low-dimensional features to high-dimensional space to construct the optimal classification hyperplane). The multi-class support vector machine model calculates the distance between the defect discrimination feature vector and each classification hyperplane, and outputs the discrimination probability of the target solder joint belonging to each preset defect type (such as cold solder joint, insufficient solder, bridging, etc.). The discrimination probability refers to the original probability value that characterizes the solder joint corresponding to the target surface mount component belonging to each type of defect. Then, the defect category with the highest discrimination probability is selected as the defect type of the solder joint corresponding to the target surface mount component.

[0102] It should be noted that the defect type in this application refers to the defect type of the solder joint corresponding to the target surface mount component.

[0103] Furthermore, in another aspect of this application, in some embodiments, this application provides a target patch element defect detection device based on multi-algorithm fusion, the device including a patch element defect detection unit, referencing... Figure 3 The figure is a schematic diagram of the structure of a surface mount component defect detection unit according to some embodiments of this application. The surface mount component defect detection unit includes: a data acquisition module 201, a processing module 202, and an execution module 203, which are described below:

[0104] The acquisition module 201 in this application is mainly used to acquire the surface optical image of the target surface mount component area on the printed circuit board, and generate a solder joint area mask based on the solder joint outline of the corresponding solder joint of the target surface mount component in the surface optical image.

[0105] Processing module 202, in this application, is mainly used to locate the defect identification center of the corresponding solder joint of the target patch element from the surface optical image according to the solder joint area mask, and to radiate multiple identification auxiliary lines outward at equal angles from the defect identification center to identify defects in the corresponding solder joint of the target patch element.

[0106] The processing module 202 is also used to extract the gray values ​​of the corresponding pixels in the surface optical image from the defect identification center toward the boundary direction for each identification auxiliary line, so as to obtain multiple gray value sequences that characterize the gray value distribution of a single solder joint surface.

[0107] In addition, the processing module 202 is also used to construct a defect discrimination feature map of the solder joint corresponding to the target patch element based on all gray value sequences, and determine the gray consistency index of the preset central highlight area and the edge gradient index of the solder joint outline area in the defect discrimination feature map. The edge gradient index is a set of quantitative parameters that characterize the intensity and direction distribution of gray change in the solder joint outline area.

[0108] The execution module 203 in this application is mainly used to determine the defect discrimination feature vector of the solder joint corresponding to the target patch element based on the grayscale consistency index and the edge gradient index, and to identify the defect type of the solder joint corresponding to the target patch element based on the defect discrimination feature vector.

[0109] In addition, this application also provides a computer device, the computer device including a memory and a processor, the memory storing code, and the processor being configured to acquire the code and execute the above-described SMT component defect detection method based on multi-algorithm fusion.

[0110] In some embodiments, reference Figure 4 The figure is a schematic diagram of the structure of a computer device implementing a multi-algorithm fusion-based SMT component defect detection method according to some embodiments of this application. The multi-algorithm fusion-based SMT component defect detection method in the above embodiments can... Figure 4 The computer device shown is used to implement this, and the computer device includes at least one processor 301, a communication bus 302, a memory 303, and at least one communication interface 304.

[0111] The processor 301 can be a general-purpose central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more devices used to control the execution of the SMT component defect detection method based on multi-algorithm fusion in this application.

[0112] The communication bus 302 can be used to transmit information between the aforementioned components.

[0113] The memory 303 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disks or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 303 may exist independently and be connected to the processor 301 via the communication bus 302. The memory 303 may also be integrated with the processor 301.

[0114] The memory 303 stores program code for executing the solution of this application, and its execution is controlled by the processor 301. The processor 301 executes the program code stored in the memory 303. The program code may include one or more software modules. In the above embodiments, the determination of the SMT component defect detection method based on multi-algorithm fusion can be achieved by the processor 301 and one or more software modules in the program code in the memory 303.

[0115] Communication interface 304 uses any transceiver-like device for communicating with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area networks (WLAN), etc.

[0116] In a specific implementation, as one example, a computer device may include multiple processors, each of which may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. Here, a processor may refer to one or more devices, circuits, and / or processing cores used to process data (e.g., computer program instructions).

[0117] The aforementioned computer device can be a general-purpose computer device or a special-purpose computer device. In specific implementations, the computer device can be a desktop computer, a portable computer, a network server, a handheld digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device, or an embedded device. This application does not limit the type of computer device.

[0118] In addition, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described SMT component defect detection method based on multi-algorithm fusion.

[0119] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0120] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for detecting defects in SMT components based on multi-algorithm fusion, characterized in that, Includes the following steps: Acquire a surface optical image of the target surface mount component area on the printed circuit board, and generate a solder joint area mask based on the solder joint outline of the corresponding solder joint of the target surface mount component in the surface optical image; Based on the solder joint area mask, the defect identification center of the corresponding solder joint of the target patch element is located from the surface optical image, and multiple identification auxiliary lines are radiated outward at equal angles from the defect identification center to identify defects in the corresponding solder joint of the target patch element. For each identification auxiliary line, the gray value of the corresponding pixel in the surface optical image is extracted from the defect identification center toward the weld point boundary to obtain multiple gray value sequences that characterize the gray value distribution of a single weld point surface. Based on all grayscale value sequences, a defect discrimination feature map of the solder joint corresponding to the target patch element is constructed. The grayscale consistency index of the central highlighted area and the edge gradient index of the solder joint contour area are determined within the defect discrimination feature map. The edge gradient index is a set of quantitative parameters characterizing the intensity and directional distribution of grayscale changes in the solder joint contour area. Specifically, constructing the defect discrimination feature map of the solder joint corresponding to the target patch element based on all grayscale value sequences includes: constructing a polar coordinate system with the geometric center of the defect identification center as the pole and the radiation direction angle of each identification auxiliary line as the polar angle; mapping the position of the grayscale value in each grayscale value sequence on the corresponding identification auxiliary line to the polar radius coordinates in the polar coordinate system; and generating the defect discrimination feature map of the solder joint corresponding to the target patch element based on all polar angles, polar radius coordinates, and their corresponding grayscale values ​​in the polar coordinate system. Based on the grayscale consistency index and the edge gradient index, the defect discrimination feature vector of the solder joint corresponding to the target patch element is determined, and the defect type of the solder joint corresponding to the target patch element is identified based on the defect discrimination feature vector.

2. The method as described in claim 1, characterized in that, The method for generating a solder joint region mask based on the solder joint contour of the corresponding solder joint of the target patch element in the surface optical image includes: Extract the solder joint contour of the solder joint corresponding to the target patch element from the surface optical image; Based on the solder joint contour, determine the set of boundary pixels of the solder joint region corresponding to the target patch element; A solder joint area mask containing the complete solder joint area is generated using the boundary pixel set.

3. The method as described in claim 1, characterized in that, Locating the defect identification center of the corresponding solder joint of the target patch element from the surface optical image based on the solder joint area mask specifically includes: Perform connectivity analysis on the solder joint area mask to determine the minimum bounding rectangle of the solder joint area corresponding to the target surface mount component; The geometric center coordinates of the minimum bounding rectangle are calculated as the defect identification center of the corresponding solder joint of the target patch element.

4. The method as described in claim 1, characterized in that, Multiple auxiliary identification lines radiating outward at equal angles from the defect identification center to identify defects in the corresponding solder joints of the target surface mount component specifically include: Extract the geometric center coordinates of the defect identification center as the ray starting point; The angular interval between adjacent rays is calculated based on the preset number of rays to determine the radiation direction angle of each ray; Using the starting point as the origin, generate initial ray paths extending outward from the defect identification center according to each radiation direction angle; Based on the solder joint area mask, the initial ray path is truncated at the solder joint outline boundary, thereby obtaining multiple identification auxiliary lines for defect identification of the corresponding solder joint of the target patch element.

5. The method as described in claim 1, characterized in that, For each identification auxiliary line, the grayscale value of the corresponding pixel in the surface optical image is extracted from the defect identification center towards the weld point boundary, resulting in multiple grayscale value sequences characterizing the grayscale distribution of a single weld point surface. Specifically, these sequences include: Each identification auxiliary line is sampled for pixel points to obtain the coordinates of all pixel points from the defect identification center to the weld point boundary. Extract the grayscale values ​​corresponding to the coordinates of each pixel from the surface optical image to form the original grayscale data; The original grayscale data is arranged according to the order of pixel points on the recognition auxiliary line to obtain multiple grayscale value sequences that characterize the grayscale distribution of a single solder joint surface.

6. The method as described in claim 1, characterized in that, The surface optical image of the target surface mount component area on the printed circuit board is acquired using an industrial camera.

7. A target surface mount component defect detection device based on multi-algorithm fusion, used to execute the SMT component defect detection method based on multi-algorithm fusion as described in any one of claims 1 to 6, wherein the target surface mount component defect detection device based on multi-algorithm fusion includes a surface mount component defect detection unit, characterized in that, The patch element defect detection unit includes: The acquisition module is used to acquire surface optical images of the target surface mount component area on the printed circuit board, and generate a solder joint area mask based on the solder joint contour of the corresponding solder joint of the target surface mount component in the surface optical image. The processing module is used to locate the defect identification center of the corresponding solder joint of the target patch element from the surface optical image based on the solder joint area mask, and to radiate multiple identification auxiliary lines outward at equal angles from the defect identification center to identify defects in the corresponding solder joint of the target patch element. The processing module is also used to extract the gray values ​​of the corresponding pixels in the surface optical image for each identification auxiliary line, starting from the defect identification center and moving towards the boundary direction, to obtain multiple gray value sequences that characterize the gray value distribution of a single solder joint surface. The processing module is further configured to construct a defect discrimination feature map of the solder joint corresponding to the target patch element based on all grayscale value sequences, and determine the grayscale consistency index of the preset central highlight area and the edge gradient index of the solder joint contour area in the defect discrimination feature map. The edge gradient index is a set of quantitative parameters characterizing the intensity and directional distribution of grayscale changes in the solder joint contour area. Specifically, constructing the defect discrimination feature map of the solder joint corresponding to the target patch element based on all grayscale value sequences includes: constructing a polar coordinate system with the geometric center of the defect identification center as the pole and the radiation direction angle of each identification auxiliary line as the polar angle; mapping the position of the grayscale value in each grayscale value sequence on the corresponding identification auxiliary line to the polar radius coordinates in the polar coordinate system; and generating the defect discrimination feature map of the solder joint corresponding to the target patch element based on all polar angles, polar radius coordinates and their corresponding grayscale values ​​in the polar coordinate system. The execution module is used to determine the defect discrimination feature vector of the solder joint corresponding to the target patch element based on the grayscale consistency index and the edge gradient index, and to identify the defect type of the solder joint corresponding to the target patch element based on the defect discrimination feature vector.

8. A computer device, characterized in that, The computer device includes a memory and a processor, the memory storing code, and the processor being configured to acquire the code and execute the SMT component defect detection method based on multi-algorithm fusion as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the SMT component defect detection method based on multi-algorithm fusion as described in any one of claims 1 to 6.

Images