Defect visual identification detection method for dense welding spots

Abstract

The invention provides a defect visual identification detection method for dense welding spots, which comprises the following steps: collecting a high-density welding spot array image, carrying out hierarchical processing on the image by adopting wavelet transform, separating out feature information containing an edge overlapping region and shadow distribution, and obtaining a preliminary welding spot region distribution diagram; according to the preliminary welding spot area distribution diagram, performing feature enhancement processing on edge overlapping areas in the distance between adjacent welding spots by adopting a Laplace operator to obtain an enhanced welding spot boundary image; for the enhanced welding spot boundary image, performing fuzzy processing on shadow boundaries in a distance range of adjacent welding spots, separating contour information of a single welding spot through an adaptive threshold segmentation method, and determining an individual region range of the welding spots; and extracting image features of surface texture, edge integrity and shape regularity of the welding spots from the determined individual region range, and performing normalization processing on gray distribution of each welding spot region by applying a histogram equalization method to obtain corrected welding spot feature data.

Description

Technical Field

[0001] This invention relates to the field of information technology, and in particular to a visual recognition and detection method for defects in dense solder joints. Background Technology

[0002] In modern manufacturing, solder joint quality inspection is a crucial area directly related to product safety and reliability. This is especially true in high-density electronic components, where the stability and integrity of solder joints have a decisive impact on overall performance. The dense layout of solder joint arrays makes defect detection a core task, and its accuracy and efficiency are irreplaceable for production quality control. However, existing inspection methods often fall short in complex scenarios, particularly in high-density solder joint arrays, where traditional image processing methods struggle to cope with dynamically changing environments. Many methods are susceptible to interference from adjacent solder joints when dealing with small solder joint spacing, leading to blurred identification boundaries and even misjudging defect locations. This limitation stems not merely from the inherent limitations of the technology itself, but from a lack of in-depth understanding of the complex relationship between the physical gaps between solder joints and visual perception, as well as insufficient adaptability to dynamically changing scenarios. Focusing on specific challenges, minute differences in solder joint spacing become the primary factor affecting detection accuracy. When adjacent solder joints are too close together, the outlines of the solder joint edges may overlap. This overlap not only alters the visual characteristics of individual solder joints but also triggers a redistribution of shadow areas, causing previously clear boundaries in the image to become blurred. This boundary ambiguity further exacerbates the difficulty of visual segmentation, making it hard for algorithms to accurately distinguish individual features when dividing solder joint regions, thus affecting the precise location of defects. The correlation between solder joint spacing and visual segmentation accuracy has become a pressing technical challenge. Therefore, how to dynamically adjust the visual segmentation strategy to adapt to the effects of edge overlap and shadow changes under complex conditions such as small solder joint spacing and deformation or displacement, while ensuring the accuracy of locating individual solder joint defects, has become the key issue of this research. Summary of the Invention

[0003] This invention provides a visual recognition and detection method for defects in densely packed solder joints, mainly comprising:

[0004] High-density solder joint array images were acquired, and wavelet transform was used to perform layered processing on the images to separate feature information including edge overlap areas and shadow distribution, thereby obtaining a preliminary solder joint area distribution map.

[0005] Based on the preliminary solder joint area distribution map, the Laplacian operator is used to perform feature enhancement processing on the edge overlapping area within the distance between adjacent solder joints to obtain the enhanced solder joint boundary image;

[0006] For the enhanced solder joint boundary image, the shadow boundary within the distance between adjacent solder joints is blurred, and the contour information of a single solder joint is separated by an adaptive threshold segmentation method to determine the individual region range of the solder joint.

[0007] Image features of surface texture, edge integrity, and shape regularity of solder joints are extracted from a defined individual region. Histogram equalization is applied to normalize the grayscale distribution of each solder joint region to obtain corrected solder joint feature data.

[0008] Based on the corrected solder joint feature data, a comparative analysis is performed using a pre-established defect feature library. If the gray value fluctuation or edge curvature change in the solder joint area deviates from the preset threshold, it is marked as a potential defect area, and a preliminary defect distribution result is obtained. If the gray value fluctuation and edge curvature change do not deviate from the preset threshold, it is marked as a qualified solder joint area.

[0009] Based on the preliminary defect distribution results, the spatial context analysis method based on neighboring pixels is applied to verify the correlation between potential defect areas and adjacent solder joint areas. Combined with the geometric constraints of solder joint arrangement, false detections caused by uneven lighting or imaging angle are eliminated, and the corrected defect location information is output.

[0010] By using the corrected defect location information, each defect area is classified, and features of edge curvature change and shadow gradient distribution are extracted to determine the defect type and location data, and generate weld quality analysis results.

[0011] Furthermore, the acquisition of high-density solder joint array images, followed by wavelet transform for layered image processing to separate feature information including overlapping edge regions and shadow distribution, yields a preliminary solder joint area distribution map, including:

[0012] High-density solder joint array image data is acquired. Based on the periodicity of the solder joint arrangement, the acquired grayscale image is contrast-enhanced to obtain an enhanced solder joint array image. The grayscale value distribution of each pixel in the enhanced solder joint array image is determined. For the enhanced solder joint array image, a two-dimensional decomposition method is used to decompose the image into multiple layers, separating the low-frequency component containing the main outline of the solder joints and the high-frequency component containing overlapping edge regions and shadow distribution. Based on the low-frequency component and the high-frequency component, the coefficients in the high-frequency component that exceed a preset threshold are suppressed, while the outline information in the low-frequency component is preserved. The components are reconstructed through inverse transformation to generate a reconstructed image. Based on the grayscale difference between the reconstructed image and the background, threshold segmentation is performed to obtain a binary image containing the solder joint positions and outline boundaries, thus obtaining the preliminary solder joint region distribution map.

[0013] Furthermore, based on the preliminary solder joint area distribution map, the Laplacian operator is used to perform feature enhancement processing on the edge overlap region within the spacing between adjacent solder joints to obtain an enhanced solder joint boundary image, including:

[0014] Based on the preliminary solder joint area distribution map, the centroid coordinates of each solder joint are calculated, and the solder joint spacing value is determined by the distance between adjacent centroids. Boundary contact areas with spacing less than a preset threshold are extracted as edge overlap areas to be processed. Convolution operation is performed on the edge overlap areas to be processed to obtain the second derivative response values ​​of the pixels and form an edge feature response map. An enhanced solder joint boundary image is generated based on the edge feature response map.

[0015] Furthermore, the step of blurring the shadow boundaries within the spacing range of adjacent solder joints in the enhanced solder joint boundary image includes:

[0016] For the enhanced solder joint boundary image, the grayscale difference of each pixel in the horizontal and vertical directions is calculated to obtain the gradient magnitude. A weight value is assigned according to the gradient magnitude to form a gradient weight distribution map. According to the gradient weight distribution map, continuous pixel areas with weight values ​​lower than a preset threshold are identified as shadow boundary areas. The grayscale values ​​of pixels in the shadow boundary areas are filtered to obtain an intermediate image with blurred shadows.

[0017] Furthermore, the extraction of image features such as solder joint surface texture, edge integrity, and shape regularity from the defined individual region range includes:

[0018] From a defined individual region, the number of times gray-level combinations of pixel pairs at a fixed pixel distance occur in different directions is calculated to construct a gray-level co-occurrence matrix. Statistical characteristic parameters are extracted as surface texture features. Based on the surface texture features, contour pixels are encoded in a clockwise direction, and the directional change angle of adjacent contour points is recorded. The proportion of discontinuous points is statistically analyzed as an edge integrity measure. The ratio of the area enclosed by the contour to the area of ​​the smallest circle is calculated as shape regularity.

[0019] Furthermore, based on the corrected solder joint feature data, a comparative analysis is performed using a pre-established defect feature library. If fluctuations in the grayscale value or changes in the edge curvature of the solder joint area deviate from a preset threshold, it is marked as a potential defect area, including:

[0020] Reference value ranges for various defects are extracted from a pre-established defect feature library to construct a defect judgment reference table. For the corrected solder joint feature data, the variance of the gray level difference between adjacent pixels is calculated as the gray level fluctuation value, and the angle formed by each point on the contour with its two adjacent points is calculated as the curvature. The maximum curvature value is taken as the edge curvature feature value. According to the defect judgment reference table, solder joint areas where the gray level fluctuation value or edge curvature feature value exceeds the reference range are marked as potential defect areas.

[0021] Furthermore, based on the preliminary defect distribution results, a spatial context analysis method based on neighboring pixels is applied to verify the correlation between potential defect areas and adjacent solder joint areas. Combined with the geometric constraints of the solder joint arrangement, false detections caused by uneven illumination or imaging angles are eliminated, and corrected defect location information is output, including:

[0022] Extract the grayscale values ​​of the center pixel of the potential defect region and its eight surrounding adjacent pixels. Calculate the difference between the mean grayscale value of the potential defect region and the mean grayscale value of the corresponding position in the adjacent solder joint region to obtain the grayscale difference value between regions. Based on the grayscale difference value between regions, calculate the grayscale gradient direction of each pixel on the boundary of the potential defect region. Determine the direction consistency by the dot product of the gradient direction and the gradient direction of the corresponding pixel on the boundary of the adjacent solder joint. Correct the defect location information based on the direction consistency and output the corrected defect location information.

[0023] Furthermore, the process involves classifying each defect region using the corrected defect location information, extracting features of edge curvature changes and shadow gradient distribution, and determining the defect type and location data, including:

[0024] Using the corrected defect location information, the image data of each defect region is input into a neural network. Through multi-layer convolution operations, the curvature change sequence of edge pixels and the gradient magnitude distribution of shadow areas are extracted to generate a feature map set. Based on the feature map set, the peak value, mean value, and standard deviation are extracted and combined with the corrected solder joint feature data to form a feature vector. This vector is then input into a fully connected layer for classification operations, and the defect type determination result is output.

[0025] The technical solutions provided by the embodiments of the present invention may include the following beneficial effects:

[0026] This invention discloses a visual defect recognition and detection method for dense weld points, solving the problems of edge overlap, shadow interference, and false defect judgment caused by excessively close weld point spacing, uneven illumination, and imaging angle in traditional detection methods. The method uses wavelet transform multi-resolution decomposition to separate low-frequency and high-frequency features to obtain a preliminary weld point distribution; it utilizes the Laplacian operator to enhance edge features and suppress gray-level gradient anomalies; it employs gradient-weighted region segmentation and adaptive threshold segmentation to accurately extract the contours of individual weld points; it combines histogram equalization to correct illumination and normalize feature data; it eliminates false detections and corrects defect locations through defect feature library comparison and neighborhood pixel spatial context analysis; and finally, it applies a convolutional neural network to classify defect types and generate a heatmap visualization quality analysis report. This invention significantly improves the accuracy and robustness of defect detection in high-density weld point arrays and is suitable for complex industrial scenarios. Attached Figure Description

[0027] Figure 1 This is a flowchart of a defect visual recognition and detection method for dense weld joints according to the present invention. Detailed Implementation

[0028] To further understand the content of this invention, a detailed description of the invention is provided in conjunction with the accompanying drawings and embodiments. The specific embodiments described herein are for illustrative purposes only and are not intended to limit the invention. It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.

[0029] like Figure 1 This embodiment of a defect visual recognition and detection method for dense solder joints may specifically include:

[0030] Step S101: Acquire a high-density solder joint array image, perform layered processing on the image using wavelet transform, separate feature information including edge overlap areas and shadow distribution, and obtain a preliminary solder joint area distribution map.

[0031] By acquiring raw image data of a high-density solder joint array, and based on the periodicity of the solder joint arrangement and the reflectivity of the metal surface, histogram equalization is performed on the acquired grayscale image to obtain a contrast-enhanced solder joint array image, thus determining the grayscale value distribution of each pixel in the image. For the solder joint array image with a known grayscale value distribution, a two-dimensional discrete wavelet transform is used to decompose the image. A three-level decomposition is performed using the db4 wavelet basis function to separate the low-frequency approximate component containing the main outline of the solder joints and the high-frequency detail component containing overlapping edge regions and shadow distribution. Based on the decomposed low-frequency approximate component and high-frequency detail component, the solder joint outline information of the low-frequency component is preserved, while coefficients exceeding a preset threshold in the high-frequency component are suppressed to reduce edge overlap and shadow interference. The processed components are then reconstructed using inverse wavelet transform. For the reconstructed image, threshold segmentation is performed based on the grayscale difference between the solder joints and the background to obtain a binary image containing the solder joint positions and outline boundaries, resulting in a preliminary solder joint region distribution map.

[0032] Specifically, during the image acquisition process of high-density solder joint arrays, the strong reflective properties of the metal material on the solder joint surface often result in uneven brightness distribution in the acquired original images. Histogram equalization can redistribute the gray levels of the image, expanding pixel values ​​that were originally concentrated in a certain gray level range to the entire gray level range.

[0033] Specifically, for dark images with gray values ​​concentrated in the range of 50-100, by calculating the cumulative distribution function and performing gray-level mapping, it can be extended to the full range of 0-255, thereby enhancing the contrast between the solder joints and the background.

[0034] In one possible implementation, the decomposition process of the two-dimensional discrete wavelet transform is achieved by selecting appropriate wavelet basis functions. The db4 wavelet basis function has fourth-order vanishing moments, which can effectively preserve image edge information while achieving frequency domain decomposition. The three-level decomposition means performing three recursive decomposition operations on the image. The first level decomposes the original image into a low-frequency approximate component LL1 and three high-frequency detail components LH1, HL1, and HH1; the second level further decomposes LL1 to obtain LL2 and the corresponding high-frequency components; the third level decomposes LL2 again. This multi-level decomposition concentrates the main contour information of the solder joint in the low-frequency components, while edge overlap and shadow interference are mainly distributed in the high-frequency components.

[0035] It should be noted that the threshold suppression of high-frequency components is based on the statistical characteristics of wavelet coefficients. High-frequency coefficients generated by overlapping edges and shadows typically have small amplitudes and are relatively dispersed, while high-frequency coefficients generated by genuine solder joint edges are relatively concentrated and have larger amplitudes. By setting an adaptive threshold based on the standard deviation of the coefficients, such as twice the standard deviation, those small-amplitude coefficients caused by interference can be effectively suppressed while retaining important edge information. The inverse wavelet transform reconstructs the processed components according to the reverse process of decomposition, resulting in an image free of edge overlaps and shadow interference.

[0036] For example, the threshold segmentation of the reconstructed image uses Otsu's method to automatically determine the optimal segmentation threshold. This method calculates the inter-class variance of the foreground and background under different thresholds and finds the threshold that maximizes the inter-class variance as the optimal segmentation point. For solder joint images, the grayscale value of the solder joint region is usually higher than that of the background region. By using this threshold, the image is converted into a binary image, with the solder joint region marked as a white pixel value of 255 and the background region marked as a black pixel value of 0. The resulting binary image clearly shows the location and contour boundary of each solder joint, forming a preliminary solder joint region distribution map, which provides a foundation for subsequent solder joint quality inspection and localization analysis.

[0037] Step S102: Based on the preliminary solder joint area distribution map, the Laplacian operator is used to perform feature enhancement processing on the edge overlapping area within the distance between adjacent solder joints to obtain the enhanced solder joint boundary image.

[0038] Based on the preliminary solder joint area distribution map, the centroid coordinates of each solder joint in the image are calculated. The spacing between adjacent centroids is determined by the Euclidean distance. For adjacent solder joint pairs with a spacing less than a preset threshold, their boundary contact area is extracted as the edge overlap region to be processed. For the edge overlap region to be processed, a Laplacian operator is used to perform convolution operations on this region. The Laplacian operator kernel matrix has a center value of 8 and eight surrounding positions with values ​​of -1. The second derivative response value of each pixel is obtained through convolution, forming an edge feature response map. Based on the distribution of the second derivative response values ​​in the edge feature response map, pixels with response values ​​exceeding a preset multiple of the neighborhood mean are identified as edge enhancement points. The original grayscale value of the edge enhancement points is preserved, while the grayscale values ​​of the remaining pixels are decreased according to their distance from the edge enhancement points, resulting in an intermediate image after feature enhancement. For the intermediate image after feature enhancement, starting from the edge enhancement points, pixels are connected along the opposite direction of grayscale value decrease to complete the boundary breaks caused by abnormal grayscale gradients, restoring the continuity of the solder joint contour and obtaining the enhanced solder joint boundary image.

[0039] In one possible implementation, the centroid coordinates of the solder joints are calculated by performing moment calculations on the binarized solder joint region. For each independent solder joint region, the centroid position is obtained by calculating the average of all its pixel coordinates. When the centroid distance between two adjacent solder joints is less than 1.5 times the average diameter of the solder joints, it indicates that the two solder joints may be in boundary contact. The boundary contact region is extracted through morphological dilation operations. The two solder joint regions are dilated separately, and the overlapping part after dilation is the edge overlap region to be processed. The Laplacian operator, as a second-order differential operator, is very sensitive to abrupt changes in grayscale in the image.

[0040] Specifically, the 3×3 kernel matrix of this operator calculates the second derivative by multiplying the center pixel value by 8 and then subtracting the sum of the values ​​of the surrounding 8 pixels. This calculation method results in an output value close to zero in regions with uniform grayscale variation, while at edges, due to abrupt grayscale changes, a larger positive or negative response is generated. For overlapping areas at solder joint edges, the true solder joint boundary produces a strong response, while the pseudo-edge response caused by uneven lighting or surface reflection is relatively weak.

[0041] It should be noted that the identification of edge enhancement points is based on local statistical characteristics. The mean of the second derivative response of all pixels is calculated within a 5×5 neighborhood window. When the response value of the center pixel exceeds a certain multiple of this mean, it is identified as an edge enhancement point. These points typically correspond to the actual contour location of the solder joint. For the grayscale value reduction processing of non-edge enhancement points, a distance-based weighting function is used. Pixels farther away from the nearest edge enhancement point experience greater grayscale value decay, creating a gradual transition effect from the edge to the interior.

[0042] In one embodiment, the pixel joining process begins at each edge enhancement point, examining the distribution of pixel grayscale values ​​within its 8-neighborhood. Tracing is performed along the direction of the slowest decrease in grayscale value, which typically corresponds to the direction of the solder joint boundary. When a breakpoint with a sudden drop in grayscale value is encountered, interpolation is used to fill in the intermediate pixel values, maintaining boundary continuity. This method is particularly suitable for handling boundary blurring caused by adjacent solder joints being too close together. By enhancing the true edges and suppressing interference, the resulting enhanced solder joint boundary image clearly distinguishes the independent contour of each solder joint, maintaining good boundary separation even when solder joints are densely packed.

[0043] Step S103: For the enhanced solder joint boundary image, the shadow boundary within the distance between adjacent solder joints is blurred, and the contour information of a single solder joint is separated by an adaptive threshold segmentation method to determine the individual region range of the solder joint.

[0044] For the enhanced solder joint boundary image, the gray-level difference of each pixel in the image is calculated in the horizontal and vertical directions. The square root of the sum of the squares of the differences in the two directions is used to obtain the gradient magnitude. A corresponding weight value is assigned according to the gradient magnitude, forming a gradient weight distribution map. Based on the gradient weight distribution map, continuous pixel regions with weight values ​​below a preset threshold are identified as shadow boundary regions. Gaussian filtering is applied to the gray-level values ​​of pixels within the shadow boundary regions, with the filter kernel size adaptively adjusted according to the width of the region, resulting in a shadow-blurred intermediate image. For the shadow-blurred intermediate image, the Otsu method is used to calculate the global segmentation threshold. For each local region in the image, the mean and standard deviation of the gray-level value of that region are calculated. The local segmentation threshold is adjusted to the sum of the global threshold and the standard deviation multiplied by a preset coefficient. Binary segmentation is performed using the adjusted threshold. The candidate solder joint regions obtained through binary segmentation are then labeled with connected components to obtain the pixel set of each independent solder joint. The outer contour coordinate sequence of the pixel set is extracted, and the minimum bounding rectangle is calculated based on the contour coordinates to determine the individual region range of each solder joint.

[0045] In one possible implementation, the gradient magnitude is calculated based on the gray-level difference between adjacent pixels. For a given pixel in the image, the horizontal gray-level difference is obtained by subtracting the gray-level value of the pixel to the left from the gray-level value of the pixel to the right, and the vertical gray-level difference is obtained by subtracting the gray-level value of the pixel above from the gray-level value of the pixel below. The gradient magnitude is obtained by squareding these differences in each direction, summing them, and then taking the square root. Gradient weights are assigned using a linear mapping method: pixels with a gradient magnitude of 0 are assigned a weight of 0, pixels with a gradient magnitude reaching its maximum value are assigned a weight of 1, and intermediate values ​​are weighted proportionally. The identification of shadow boundary regions depends on the spatial distribution characteristics of the gradient weights.

[0046] Specifically, at the edges of solder joints, due to drastic grayscale changes, the gradient weight values ​​are high, typically exceeding 0.7; while in shadow areas, due to the gradual grayscale changes caused by gradual illumination variations, the gradient weight values ​​are generally below 0.3. By setting a weight threshold of 0.3, these low-gradient areas can be effectively identified. The Gaussian filter kernel size is dynamically adjusted according to the width of the identified shadow area. When the area width is 5 pixels, a 5×5 filter kernel is used; when the width is 7 pixels, a 7×7 filter kernel is used. This adaptive adjustment ensures that shadow areas of different sizes receive appropriate blurring.

[0047] It should be noted that Otsu's method, as a classic automatic thresholding method, determines the optimal segmentation threshold by maximizing the inter-class variance between the foreground and background. In solder joint images, due to differences in lighting conditions in different areas, a single global threshold often cannot adapt to all areas. Local adaptive adjustment is achieved by calculating the gray-level statistical characteristics within each 32×32 pixel window. The standard deviation reflects the dispersion of gray-level values ​​within this area; a larger standard deviation indicates higher contrast in this area, requiring a higher segmentation threshold to avoid oversegmentation. The preset coefficient is typically set between 0.5 and 1.5 and adjusted according to the overall image contrast.

[0048] In one embodiment, connected component labeling employs the eight-neighborhood connectivity criterion, meaning that a pixel is considered to belong to the same connected component if it is connected to at least one of its eight surrounding pixels. After labeling, each connected component represents a potential solder joint region. Outer contour extraction is achieved by tracing the boundary pixels of the connected components, starting from the top-left boundary point and recording the coordinates of the boundary pixels sequentially in a counter-clockwise direction. The calculation of the minimum bounding rectangle is implemented using a rotating caliper algorithm. This algorithm finds the smallest rectangle that can enclose all contour points by rotating the rectangle at different angles. The coordinates of the four vertices of this rectangle determine the precise range of the individual solder joint region, providing an accurate basis for subsequent solder joint quality assessment and defect detection.

[0049] Step S104: Extract image features of solder joint surface texture, edge integrity, and shape regularity from the determined individual region range, and apply histogram equalization to normalize the grayscale distribution of each solder joint region to obtain the corrected solder joint feature data.

[0050] Within a defined individual region, the frequency of gray-level combinations in different directions for pixel pairs within the solder joint region that are at fixed pixel distances is calculated to construct a gray-level co-occurrence matrix. Energy, contrast, and correlation parameters are calculated using the statistical properties of the matrix elements as surface texture features. Based on the contrast parameter in the surface texture features, the solder joint contour pixels are encoded clockwise, and the directional change angle of adjacent contour points is recorded. The accumulated angle change values ​​are used, and points exceeding a preset threshold are considered discontinuous. The proportion of discontinuous points to the total number of contour points is used as an edge integrity measure. For solder joints with determined edge integrity, the ratio of the area enclosed by the contour to the area of ​​the smallest circle containing the contour is calculated as the shape regularity. Simultaneously, the frequency of each gray level within the solder joint region is counted to construct an original gray-level histogram. A cumulative distribution function is calculated using the original gray-level histogram. Based on the cumulative distribution function value, the original gray levels are mapped to uniformly distributed new gray levels. Gray-level mapping transformation is performed on all pixels within the solder joint region. The texture features, edge integrity measure, and shape regularity of the transformed image are combined to obtain corrected solder joint feature data.

[0051] Specifically, the construction process of the gray-level co-occurrence matrix is ​​based on the statistical analysis of the spatial relationships between pixel pairs. In the solder joint image, pixel pairs separated by one pixel are selected, and statistics are performed in four directions: 0 degrees, 45 degrees, 90 degrees, and 135 degrees. When two pixels with gray values ​​i and j appear adjacent to each other at a specified distance and direction, the count at the i-th row and j-th column of the matrix is ​​incremented by 1. After traversing the entire solder joint area, each element in the matrix reflects the frequency of occurrence of a specific gray-level combination. The energy parameter is obtained by calculating the sum of squares of all elements in the matrix, reflecting the uniformity of the image texture; the contrast is obtained by calculating the weighted sum of different gray-level differences, reflecting the clarity of the texture; and the correlation parameter measures the degree of linear correlation between pixel gray-level values.

[0052] In one possible implementation, contour encoding employs an eight-direction chain code representation. Starting from the beginning of the solder joint contour, the direction from each contour pixel to the next adjacent pixel is represented by numbers from 0 to 7, where 0 represents horizontal to the right, increasing by 45 degrees each time. The difference between two adjacent direction codes represents the angle of change in direction. When this angle exceeds 90 degrees, it indicates a sharp turn in the contour. By accumulating the number of these large angle changes, the degree of contour irregularity can be identified. The contrast parameter acts as a threshold adjustment in this process; high-contrast solder joint edges are more likely to produce clear contours, thereby reducing misjudgments of discontinuities.

[0053] It should be noted that the calculation of shape regularity involves a comparison of two key geometric quantities. The area of ​​the region enclosed by the contour is calculated using Green's formula, which is obtained by summing the cross coordinates of the contour points. The determination of the minimum circumcircle is achieved through an iterative algorithm. Initially, the centroid of the contour is used as the center, and the position of the center and the size of the radius are gradually adjusted until all contour points are located inside the circle and at least three points are on the circumference. The area ratio of a regular circular weld joint is close to 0.785, while this ratio will be significantly lower for elliptical or irregularly shaped weld joints.

[0054] For example, the calculation and application of the cumulative distribution function embodies the core idea of ​​histogram equalization.

[0055] The original grayscale histogram shows the pixel distribution at each grayscale level. The cumulative distribution function is obtained by summing these values; this function monotonically increases from 0 to 1. Mapping the original grayscale level i to a new grayscale level involves multiplying the value of the cumulative distribution function at i by the maximum grayscale level to obtain the new grayscale value.

[0056]

[0057] CDF(r) k ) represents the cumulative distribution function at gray level r k The value at n jThe formula represents the number of pixels at gray level j, and N represents the total number of pixels in the image. This formula obtains a cumulative distribution function that monotonically increases from 0 to 1 by summing the probabilities of each gray level.

[0058] s k = (L-1)×CDF(r) k )

[0059] s k This represents the new grayscale value after mapping, where L represents the total number of grayscale levels, and CDF(r) represents the new grayscale value after mapping. k ) represents the original gray level r k The corresponding cumulative distribution function value, this formula realizes the transformation process of mapping the original gray level to the new gray level through the cumulative distribution function. This mapping makes the pixels originally concentrated in a certain gray range uniformly distributed to the entire gray range, thereby eliminating the phenomenon of local over-brightness or under-brightness caused by uneven lighting. After this normalization process, the solder joint images taken under different lighting conditions have similar gray-level distribution characteristics, making the extracted texture features, edge integrity measures, and shape regularity values ​​comparable.

[0060] Step S105: Based on the corrected solder joint feature data, a comparative analysis is performed using a pre-established defect feature library. If the gray value fluctuation or edge curvature change in the solder joint area deviates from the preset threshold, it is marked as a potential defect area, and a preliminary defect distribution result is obtained. If the gray value fluctuation and edge curvature change do not deviate from the preset threshold, it is marked as a qualified solder joint area.

[0061] Based on the corrected solder joint feature data, reference value ranges for various defects are extracted from a pre-established defect feature library. Texture feature values ​​below the lower limit indicate cold solder joints, edge integrity metrics below the threshold indicate missing solder joints, and shape regularity exceeding the upper limit indicates excessive solder. A defect judgment reference table is constructed. For this table, the variance of the grayscale difference between adjacent pixels within the current solder joint area is calculated as the grayscale fluctuation value. Curvature is calculated using the angle formed by each point on the solder joint outline and its two adjacent points, and the maximum curvature value is taken as the edge curvature feature value. If the grayscale fluctuation value exceeds the cold solder joint grayscale fluctuation threshold in the defect judgment reference table, or the edge curvature feature value exceeds the missing solder joint edge curvature threshold, the solder joint is marked as a potential defect area, and the specific value exceeding the threshold and the corresponding defect type are recorded. If both the grayscale fluctuation value and the edge curvature feature value are within the normal range of the defect judgment reference table, the solder joint is marked as a qualified solder joint area. The detection marks and defect type information of all solder joints are summarized to obtain preliminary defect distribution results.

[0062] In one possible implementation, the defect feature library is built based on statistical analysis of a large number of historical solder joint samples. A typical characteristic of a cold solder joint is a surface texture value below 0.3, because the solder fails to melt sufficiently during a cold solder joint, resulting in a rough, granular texture. A missing solder joint is characterized by an edge integrity metric below 0.6, with obvious gaps or discontinuities in the solder joint outline. With excessive solder, the shape regularity exceeds 1.2, resulting in an overly full spherical shape. These reference value ranges are derived through feature extraction and statistical analysis of thousands of known defect types of solder joints, forming a reliable judgment benchmark. The calculation of grayscale fluctuation values ​​reflects the uniformity of the solder joint surface.

[0063] Specifically, within a 5x5 pixel window, the grayscale difference between the center pixel and the surrounding 24 pixels is calculated, and then the variance of these differences is calculated. Normal solder joints have a relatively smooth surface, with gradual grayscale changes, and the variance is typically between 10 and 30. In contrast, poorly soldered joints have a rough, uneven surface, large local differences in reflectivity, and drastic grayscale fluctuations, often with a variance exceeding 50. This fluctuation characteristic can effectively distinguish between good and bad soldering quality.

[0064] It should be noted that the edge curvature is calculated using a discrete curvature estimation method. For each point on the contour, five points are selected: the point itself and two points before and after it. An arc is fitted using the least squares method, and the reciprocal of the arc radius is the curvature value at that point. Normal weld joints exhibit smooth, arc-shaped edges with uniform curvature values, and the maximum curvature generally does not exceed 0.1. In contrast, incomplete weld defects create sharp turns at the weld joint edges, leading to a sharp increase in local curvature, potentially exceeding 0.3. By setting a curvature threshold of 0.15, edge anomalies can be effectively identified.

[0065] For example, the defect determination process employs a multi-feature comprehensive evaluation method. When a grayscale fluctuation value of 65 is detected for a solder joint, exceeding the cold solder joint threshold of 50, the system further checks whether its texture feature value is also below 0.3. If both conditions are met simultaneously, it is confirmed as a cold solder joint defect, and the specific value of the fluctuation value exceeding the threshold (15) is recorded. This multi-verification mechanism avoids misjudgments that may be caused by a single feature. For edge curvature detection, when the maximum curvature value reaches 0.22, the system marks that there may be a missing solder joint at that location and records the degree of deviation from the normal range as 0.07. By summarizing the detection results of all solder joints, the resulting defect distribution map not only indicates the defect location but also provides information on the defect type and severity.

[0066] Step S106: Based on the preliminary defect distribution results, the spatial context analysis method based on neighboring pixels is applied to verify the correlation between the potential defect area and the adjacent solder joint area. Combined with the geometric constraints of the solder joint arrangement, false detections caused by uneven lighting or imaging angle are eliminated, and the corrected defect location information is output.

[0067] For potential defect areas in the preliminary defect distribution results, the grayscale values ​​of the central pixel and its eight surrounding pixels are extracted. The difference between the average grayscale value of these nine pixels and the average grayscale value of the nine pixels at the corresponding positions in adjacent solder joint areas is calculated to obtain the grayscale difference value between regions. Based on the grayscale difference value between regions, if the difference value is less than a preset threshold, the grayscale gradient direction of each pixel on the boundary of the potential defect area is calculated. The consistency of direction is determined by the dot product of the gradient direction and the gradient direction of the corresponding pixel on the boundary of the adjacent solder joint. If the consistency exceeds a preset value, it is determined that there is a contour continuity relationship. For regions with contour continuity relationships, the distance from the centroid of the potential defect area to the centroids of the four nearest normal solder joints is calculated. Based on the distance value, the row and column positions of the solder joints that this position should belong to are estimated. If the estimated position deviates from the actual detection position by more than half of the average diameter of the solder joints, it is determined to be a positional anomaly. By combining the determination results of contour continuity relationship and positional anomaly, regions that simultaneously satisfy contour continuity and positional anomaly are marked as false detections and removed, while the remaining potential defect areas are retained as real defects, and the corrected defect position information is output.

[0068] In one possible implementation, the selection of eight adjacent pixels follows the standard eight-neighborhood definition. Taking the center pixel of the potential defect region as a reference, its eight adjacent pixels are located in the top, bottom, left, right, and four diagonal directions. These nine pixels form a 3×3 local window, which can fully reflect the local grayscale characteristics of the region. When calculating the grayscale mean, if the grayscale mean of the nine pixels in the potential defect region is 120, while the mean at the corresponding position of the adjacent normal solder joint is 125, the difference is only 5, far less than the preset threshold of 20. This indicates that the two regions are similar in grayscale characteristics, and it is very likely that the same solder joint has been incorrectly segmented into two parts. The calculation of gradient direction and consistency determination are key to identifying contour continuity relationships.

[0069] Specifically, for each pixel on the boundary, the gradient vector can be obtained by calculating its gray-level difference in the horizontal and vertical directions. The gradient direction is the angle of this vector, usually calculated using the arctangent function. When the gradient directions of corresponding pixels on the boundaries of two adjacent regions are multiplied by a dot product, if the result is close to 1, it means that the two gradient directions are almost parallel, indicating that the two boundaries are actually a continuation of the same contour.

[0070] For example, if the gradient direction of a point on the boundary of a potential defect area is 45 degrees, and the gradient direction of the corresponding point on the boundary of an adjacent weld point is 43 degrees, the two are almost identical, and the dot product value reaches 0.99, exceeding the preset value of 0.9, thus confirming the existence of a contour continuity relationship.

[0071] It should be noted that the estimation of solder joint arrangement is based on the regularity of the array. In a standard solder joint array, solder joints are arranged with fixed row and column spacing. By measuring the coordinates of the centroids of the four nearest normal solder joints, the average row and column spacing can be calculated. Assuming a row spacing of 2.5 mm and a column spacing of 2.0 mm, when the distance from the centroid of a potential defect area to the nearest normal solder joint is 3.2 mm, simple geometric calculations can infer that this position should be located at the next solder joint in the diagonal direction. If the deviation between the actual detected position and the estimated position reaches 1.5 mm, while the average diameter of the solder joint is 2 mm, the deviation exceeds half the diameter. This positional anomaly strongly suggests that the area may be a false defect caused by lighting shadows or imaging angle.

[0072] For example, the comprehensive judgment process embodies the idea of ​​multiple verifications. When a region simultaneously meets both the conditions of contour continuation and positional anomaly, it can be basically determined to be a false detection. This situation is common in the high-reflectivity areas at the edges of solder joints, where strong reflections cause local overexposure, leading to misidentification as independent regions during image processing. Through spatial context analysis, the system can identify that these areas are actually part of a normal solder joint, thus avoiding incorrectly labeling them as defects. After this correction process, the output defect location information is more accurate and reliable, significantly reducing the false detection rate.

[0073] Step S107: Using the corrected defect location information, classify each defect region, extract the features of edge curvature change and shadow gradient distribution, determine the defect type and location data, generate a solder joint quality inspection report, visualize the defect location in the high-density array, and obtain the solder joint quality analysis results.

[0074] Using the corrected defect location information, the image data of each defect region is input into a convolutional neural network. The network extracts the curvature change sequence of edge pixels and the gradient magnitude distribution of shadow areas through multi-layer convolution operations, generating a feature map set containing spatial feature information. Based on the numerical distribution in the feature map set, peak value, mean, and standard deviation are extracted as refined features. These are combined with the acquired texture feature values, shape regularity, grayscale fluctuation values, and edge integrity measures to form a feature vector, which is input into the fully connected layer of the network for classification operations, outputting the judgment results for three defect types: cold solder joint, missing solder joint, and excessive solder. For the judgment result of each defect type, the defect severity index is calculated based on the degree to which each component in the feature vector deviates from the normal value. The defect location coordinates, type identifier, and severity index are arranged in the order of the solder joint array to generate a solder joint quality inspection report containing complete defect information. Using the data in the solder joint quality inspection report, the corresponding positions in the solder joint array are mapped to different shades of color values ​​according to the severity index. Bilinear interpolation is used to generate a smooth transition between adjacent solder joints, constructing a heat map visualization result reflecting the spatial distribution and severity of defects, and obtaining a complete solder joint quality analysis result.

[0075] Specifically, convolutional neural networks extract features layer by layer through multiple convolutional operations. The first convolutional layer primarily captures edge information of defective regions. A 3×3 convolutional kernel slides across the image, and the convolution result at each location reflects the edge strength of that local region. For solder joint defects, the edges of weak solder joints often exhibit irregular jagged edges, resulting in drastic changes in edge curvature; while the edges of normal solder joints are relatively smooth. By using convolutional kernels in multiple directions, edge features in various directions can be extracted, forming a multi-channel feature map. The second and third convolutional layers further extract higher-level features, such as the gradation patterns of shadows and the periodic changes in texture.

[0076] In one possible implementation, refined feature extraction is based on statistical analysis of the feature map. Taking an edge curvature change sequence as an example, a series of curvature values ​​can be obtained by sampling along the detected edge trajectory on the feature map. The peak values ​​of these values ​​reflect the maximum curvature of the edge, the mean reflects the overall curvature trend, and the standard deviation characterizes the severity of the curvature change. The analysis of shadow gradient distribution is similar; by calculating the distribution characteristics of gradient amplitude within the shadow region, different types of defects can be distinguished. Missing solder defects typically form a deep shadow where the solder joint should be, with gradient values ​​concentrated in the higher range; while excessive solder will form a gradual shadow around the solder joint, with a more dispersed gradient value distribution.

[0077] It should be noted that the defect severity index calculation comprehensively considers the deviation of multiple feature dimensions. Assuming the texture feature value range of a normal solder joint is 0.4 to 0.6, when a solder joint is detected with a texture value of 0.25, its deviation is 0.15. Similarly, the deviations of shape regularity, grayscale fluctuation value, and edge integrity measure are calculated, and then a weighted sum is used to obtain the comprehensive severity index. The weights are set based on the contribution of each feature to defect judgment; typically, edge integrity has the highest weight because it most directly reflects welding quality. The severity index is usually normalized to between 0 and 1, where 0 represents completely normal and 1 represents a severe defect.

[0078] For example, the heatmap generation process fully leverages human visual sensitivity to color. The array of solder joints is mapped onto a two-dimensional matrix, with each solder joint location assigned a numerical value based on its quality status: 0 for normal solder joints, 0.3 to 0.5 for minor defects, 0.5 to 0.8 for moderate defects, and 0.8 to 1 for severe defects. Bilinear interpolation creates a smooth transition between adjacent solder joints, avoiding discrete color block effects. The color mapping typically uses a gradient from blue to red, with blue representing normal areas, yellow representing areas of concern, and red highlighting severe defects. This visualization allows quality engineers to quickly locate problem areas, especially when dealing with high-density arrays containing hundreds of solder joints; the intuitiveness of the heatmap significantly improves inspection efficiency.

[0079] The above embodiments are merely one of the preferred embodiments of the present invention and should not be used to limit the scope of protection of the present invention. Any modifications or refinements made to the main design concept and spirit of the present invention that are not of substantial significance, but solve the same technical problem as the present invention, should be included within the scope of protection of the present invention.

Claims

1. A visual recognition and detection method for defects in densely packed weld joints, characterized in that, The method includes: High-density solder joint array images were acquired, and wavelet transform was used to perform layered processing on the images to separate feature information including edge overlap areas and shadow distribution, thereby obtaining a preliminary solder joint area distribution map. Based on the preliminary solder joint area distribution map, the Laplacian operator is used to perform feature enhancement processing on the edge overlapping area within the distance between adjacent solder joints to obtain the enhanced solder joint boundary image; For the enhanced solder joint boundary image, the shadow boundary within the distance between adjacent solder joints is blurred, and the contour information of a single solder joint is separated by an adaptive threshold segmentation method to determine the individual region range of the solder joint. Image features of surface texture, edge integrity, and shape regularity of solder joints are extracted from a defined individual region. Histogram equalization is applied to normalize the grayscale distribution of each solder joint region to obtain corrected solder joint feature data. Based on the corrected solder joint feature data, a comparative analysis is performed using a pre-established defect feature library. If the gray value fluctuation or edge curvature change in the solder joint area deviates from the preset threshold, it is marked as a potential defect area, and a preliminary defect distribution result is obtained. If the gray value fluctuation and edge curvature change do not deviate from the preset threshold, it is marked as a qualified solder joint area. Based on the preliminary defect distribution results, the spatial context analysis method based on neighboring pixels is applied to verify the correlation between potential defect areas and adjacent solder joint areas. Combined with the geometric constraints of solder joint arrangement, false detections caused by uneven lighting or imaging angle are eliminated, and the corrected defect location information is output. By using the corrected defect location information, each defect area is classified, and features of edge curvature change and shadow gradient distribution are extracted to determine the defect type and location data, and generate weld quality analysis results.

2. The defect visual recognition and detection method for dense weld joints according to claim 1, characterized in that, The process involves acquiring high-density solder joint array images, performing layered processing on the images using wavelet transform, separating feature information including overlapping edge regions and shadow distribution, and obtaining a preliminary solder joint area distribution map, including: High-density solder joint array image data is acquired. Based on the periodicity of the solder joint arrangement, the acquired grayscale image is contrast-enhanced to obtain an enhanced solder joint array image. The grayscale value distribution of each pixel in the enhanced solder joint array image is determined. For the enhanced solder joint array image, a two-dimensional decomposition method is used to decompose the image into multiple layers, separating the low-frequency component containing the main outline of the solder joints and the high-frequency component containing overlapping edge regions and shadow distribution. Based on the low-frequency component and the high-frequency component, the coefficients in the high-frequency component that exceed a preset threshold are suppressed, while the outline information in the low-frequency component is preserved. The components are reconstructed through inverse transformation to generate a reconstructed image. Based on the grayscale difference between the reconstructed image and the background, threshold segmentation is performed to obtain a binary image containing the solder joint positions and outline boundaries, thus obtaining the preliminary solder joint region distribution map.

3. The defect visual recognition and detection method for dense weld joints according to claim 1, characterized in that, Based on the preliminary solder joint area distribution map, the Laplacian operator is used to perform feature enhancement processing on the overlapping edge regions within the spacing between adjacent solder joints, resulting in an enhanced solder joint boundary image, including: Based on the preliminary solder joint area distribution map, the centroid coordinates of each solder joint are calculated, and the solder joint spacing value is determined by the distance between adjacent centroids. Boundary contact areas with spacing less than a preset threshold are extracted as edge overlap areas to be processed. Convolution operation is performed on the edge overlap areas to be processed to obtain the second derivative response values ​​of the pixels and form an edge feature response map. An enhanced solder joint boundary image is generated based on the edge feature response map.

4. The defect visual recognition and detection method for dense weld joints according to claim 1, characterized in that, The process of blurring the shadow boundaries within the spacing range of adjacent solder joints in the enhanced solder joint boundary image includes: For the enhanced solder joint boundary image, the grayscale difference of each pixel in the horizontal and vertical directions is calculated to obtain the gradient magnitude. A weight value is assigned according to the gradient magnitude to form a gradient weight distribution map. According to the gradient weight distribution map, continuous pixel areas with weight values ​​lower than a preset threshold are identified as shadow boundary areas. The grayscale values ​​of pixels in the shadow boundary areas are filtered to obtain an intermediate image with blurred shadows.

5. The defect visual recognition and detection method for dense weld joints according to claim 1, characterized in that, The extraction of image features, including surface texture, edge integrity, and shape regularity of solder joints from a defined individual region, includes: From a defined individual region, the number of times gray-level combinations of pixel pairs at a fixed pixel distance occur in different directions is calculated to construct a gray-level co-occurrence matrix. Statistical characteristic parameters are extracted as surface texture features. Based on the surface texture features, contour pixels are encoded in a clockwise direction, and the directional change angle of adjacent contour points is recorded. The proportion of discontinuous points is statistically analyzed as an edge integrity measure. The ratio of the area enclosed by the contour to the area of ​​the smallest circle is calculated as shape regularity.

6. The defect visual recognition and detection method for dense weld joints according to claim 1, characterized in that, The process involves comparing and analyzing the corrected solder joint feature data using a pre-established defect feature library. If fluctuations in the grayscale value or changes in the edge curvature of the solder joint area deviate from a preset threshold, it is marked as a potential defect area, including: Reference value ranges for various defects are extracted from a pre-established defect feature library to construct a defect judgment reference table. For the corrected solder joint feature data, the variance of the gray level difference between adjacent pixels is calculated as the gray level fluctuation value, and the angle formed by each point on the contour with its two adjacent points is calculated as the curvature. The maximum curvature value is taken as the edge curvature feature value. According to the defect judgment reference table, solder joint areas where the gray level fluctuation value or edge curvature feature value exceeds the reference range are marked as potential defect areas.

7. The defect visual recognition and detection method for dense weld joints according to claim 1, characterized in that, Based on the preliminary defect distribution results, a spatial context analysis method based on neighboring pixels is applied to verify the correlation between potential defect areas and adjacent solder joint areas. Combined with the geometric constraints of the solder joint arrangement, false detections caused by uneven illumination or imaging angles are eliminated, and corrected defect location information is output, including: Extract the grayscale values ​​of the center pixel of the potential defect region and its eight surrounding adjacent pixels. Calculate the difference between the mean grayscale value of the potential defect region and the mean grayscale value of the corresponding position in the adjacent solder joint region to obtain the grayscale difference value between regions. Based on the grayscale difference value between regions, calculate the grayscale gradient direction of each pixel on the boundary of the potential defect region. Determine the direction consistency by the dot product of the gradient direction and the gradient direction of the corresponding pixel on the boundary of the adjacent solder joint. Correct the defect location information based on the direction consistency and output the corrected defect location information.

8. The defect visual recognition and detection method for dense weld joints according to claim 1, characterized in that, The process involves classifying each defect region using the corrected defect location information, extracting features such as edge curvature changes and shadow gradient distribution, and determining the defect type and location data, including: Using the corrected defect location information, the image data of each defect region is input into a neural network. Through multi-layer convolution operations, the curvature change sequence of edge pixels and the gradient magnitude distribution of shadow areas are extracted to generate a feature map set. Based on the feature map set, the peak value, mean value, and standard deviation are extracted and combined with the corrected solder joint feature data to form a feature vector. This vector is then input into a fully connected layer for classification operations, and the defect type determination result is output.

Images