SMT patch defect identification method and system

Abstract

The invention relates to an SMT patch defect identification method and system. The method comprises the following steps: acquiring a first image of a welding spot in a first illumination mode, and judging whether the first image has abnormal optical characteristics or not according to the brightness distribution characteristics of the first image; if the optical characteristics of the first image are abnormal, adaptively switching to a second illumination mode, and acquiring a second image of the welding spot in the second illumination mode; acquiring rough position information of a welding spot area in the first image; acquiring fine edge information of a welding spot area in the second image; screening and fusing the fine edge information according to the rough position information to reconstruct a welding spot contour; and on the basis of the reconstructed welding spot contour, welding spot geometric information is calculated, and defect judgment is carried out, so that the misjudgment probability under the condition that the optical characteristics of the welding spots are abnormal is effectively reduced, and the SMT patch defect identification accuracy is remarkably improved.

Description

Technical Field

[0001] This application relates to the field of SMT (Surface Mount Technology) defect identification technology, and more specifically, to an SMT defect identification method and system. Background Technology

[0002] In modern electronic product manufacturing, surface mount technology (SMT) has become the mainstream circuit board assembly method. To ensure product quality, an automated optical inspection (AOI) step is usually set up after the reflow soldering process to identify defects in solder joints using image processing technology. This identification method typically relies on a high-resolution camera to capture images of solder joints and uses a series of image processing steps, such as image enhancement, feature extraction, and pattern recognition, to determine whether there are problems such as too much solder, too little solder, solder bridging, or cold solder joints.

[0003] However, in actual production, when solder paste with different flux residue characteristics is introduced during the production process, a semi-transparent film that is difficult to detect with the naked eye may form on the surface of the solder joint. This film changes the light reflection characteristics of the solder joint, changing it from the expected specular reflection to diffuse reflection, thus significantly reducing the image contrast between the solder joint and the circuit board background. This significant reduction in brightness difference poses a serious challenge to the image processing software in automated optical inspection systems. The program steps in the system software responsible for accurately extracting solder joints from complex image backgrounds typically work by automatically determining a brightness dividing line or region based on the assumption that there is a clear "bright area" (representing the solder joint) and a "dark area" (representing the background) in the image brightness distribution. However, in new image data, because the brightness difference between the solder joint and the background is too small, the boundary between this bright and dark area becomes blurred, and the original image segmentation method struggles to find a suitable dividing line that can accurately distinguish the solder joint from the background.

[0004] Because image processing programs cannot accurately depict the true outline of solder joints, all subsequent calculations relying on solder joint geometry, such as solder joint size (e.g., diameter, area), solder amount (estimated through height or volume), and key geometric information (e.g., roundness, symmetry), become significantly inaccurate. This distorted information, when fed into the final judgment program and compared to the stored acceptable product standard range, leads to numerous misjudgments. For example, many morphologically normal solder joints are judged as defective simply because their calculated dimensions deviate from the standard range (e.g., incorrectly calculated as "too much solder" or "too little solder"). This misjudgment is not random but rather systematic, as all affected solder joints exhibit similar changes in optical properties.

[0005] Ultimately, without fundamental changes to the production line hardware (such as cameras and light sources), core software logic, or main production materials (the metal composition of solder paste), a seemingly minor material management error—introducing a solder paste with different flux residue characteristics—brought the entire automated optical inspection system to a near standstill. The system continuously generated numerous false alarms that were difficult to resolve by simply adjusting camera brightness, exposure time, or judging the program's acceptable range. The root cause of the problem lay in a microscopic, unforeseen semi-transparent film on the solder joint surface, which undermined the very physical foundation upon which the optical inspection system relied—the inherent assumptions about the light reflection characteristics of the solder joint surface. In this situation, traditional parameter adjustments or simple software corrections failed to address the core issue because the system was unable to detect this fundamental optical change.

[0006] To address the aforementioned issues, existing technologies urgently need improvement. Summary of the Invention

[0007] This application discloses an SMT (Surface Mount Technology) defect identification method and system, which aims to solve the problem that when solder paste with different flux residue characteristics is introduced during the production process, a semi-transparent film is formed on the surface of the solder joint, thereby changing the light reflection characteristics of the solder joint, significantly reducing the image contrast between the solder joint and the circuit board background, and thus causing misjudgment by the automatic optical inspection system.

[0008] The technical solution of this application is as follows: In a first aspect, this application discloses a method for identifying SMT (Surface Mount Technology) surface mount defects, including: Acquire the first image of the solder joint under the first illumination mode, and determine whether there are any optical anomalies in the first image based on the brightness distribution characteristics of the first image; If the first image has optical anomalies, the system will adaptively switch to the second illumination mode and acquire a second image of the solder joint under the second illumination mode. Obtain the approximate location information of the solder joint area in the first image; Obtain fine edge information of the solder joint area in the second image; Based on the rough location information, the fine edge information is filtered and fused to reconstruct the weld point outline; Based on the reconstructed weld point contour, the geometric information of the weld point is calculated and defect judgment is performed.

[0009] Furthermore, the steps for obtaining the approximate location information of the solder joint area in the first image include: Perform grayscale morphological processing on the first image to obtain candidate regions containing solder joints; The steps for filtering and fusing the fine edge information based on the coarse location information to reconstruct the solder joint outline include: The candidate region is converted into a binary mask image, where the pixel value of the candidate region in the first image is 1, and the pixel value of other regions outside the candidate region is 0. The edge pixels with a value of 1 at the corresponding position in the binary mask image are obtained from the fine edge information, and the edge pixels are connected and closed to reconstruct the solder joint contour.

[0010] In some preferred embodiments, the step of obtaining rough location information of the solder joint area in the first image includes: generating a guided confidence map based on the texture features of the first image, wherein the value of each pixel in the guided confidence map represents the probability that it belongs to the solder joint; The steps for filtering and fusing the fine edge information based on the coarse location information to reconstruct the solder joint outline include: The edge intensity of each edge pixel in the fine edge information is weighted by the confidence value of the edge pixel at the corresponding position in the guided confidence map to obtain the weighted edge intensity; The weighted edge intensity is subjected to a second thresholding process to obtain edge pixels that are higher than the weighted intensity threshold. These edge pixels that are higher than the weighted intensity threshold are then connected and closed to reconstruct the solder joint contour.

[0011] Further, the step of generating a guided confidence map based on the texture features of the first image, wherein the value of each pixel in the guided confidence map represents the probability that it belongs to a solder joint, includes: Obtain the texture feature baseline of the background region of the first image; Calculate the local texture features of each pixel in the first image; The local texture feature is compared with the texture feature baseline, and the local texture feature is corrected according to the comparison result to suppress the contribution of the local texture feature of the background region to the guiding confidence map. The guided confidence map is generated based on the corrected local texture features.

[0012] Furthermore, the step of generating the guided confidence map based on the corrected local texture features includes: The corrected local texture features are smoothed to eliminate minor fluctuations in feature values; The smoothed local texture features are then filled to ensure the continuity of feature values. Based on the local texture features after filling, the segmentation boundary of the feature value mapping is determined to distinguish between solder joint and non-solder joint areas; Based on this segmentation boundary, a preliminary guided confidence map is generated; The preliminary guidance confidence map was morphologically processed to eliminate isolated noise points and small holes, and the boundaries were smoothed to obtain the guidance confidence map.

[0013] Based on the above, the preliminary guidance confidence map is subjected to morphological processing to eliminate isolated noise points and small holes, and to smooth the boundaries. The steps to obtain the guidance confidence map include: Based on the local features of this preliminary guided confidence map, the size of the structural elements for morphological operations is determined; Based on the curvature or connectivity of the local boundaries of the solder joint region in the preliminary guided confidence map, structural elements with a size smaller than the first size threshold are selected for morphological processing. Based on the isolation or size of the background noise region in the preliminary guided confidence map, structural elements with a size greater than the second size threshold are selected for morphological processing. Based on the morphological processing results, the guidance confidence map was obtained.

[0014] As an optional approach, the step of generating a guided confidence map based on the texture features of the first image, wherein the value of each pixel in the guided confidence map represents the probability that it belongs to a solder joint, includes: Multi-scale feature extraction was performed on the first image to obtain local texture features at different scales; The local texture features at different scales are fused to obtain the fused features; Based on this fusion feature, a nonlinear mapping is used to convert the fusion feature into a confidence value, thereby generating the guided confidence map.

[0015] Furthermore, the steps of performing multi-scale feature extraction on the first image to obtain local texture features at different scales include: The first image is preprocessed at multiple scales to obtain multiple preprocessed images at different scales. The multi-scale preprocessing includes denoising and illumination correction of the preprocessed images at different scales. Calculate the local initial texture features of the preprocessed images at multiple different scales; Spatial consistency verification is performed on the local initial texture features at multiple different scales to obtain the verification results. This spatial consistency verification is used to identify false texture features caused by noise, uneven lighting, or background interference. Based on the verification result, the false texture feature is suppressed; Inter-scale feature enhancement is performed on the suppressed local initial texture features at multiple different scales to highlight the true optical properties of the solder joints at different scales, resulting in enhanced local texture features at different scales.

[0016] Furthermore, spatial consistency verification is performed on the local texture features at multiple different scales to obtain the verification results. The steps of this spatial consistency verification for identifying false texture features caused by noise, uneven illumination, or background interference include: Calculate the nonlinear correlation measure of local initial texture features at different scales for each spatial location; Based on this nonlinear correlation metric and a preset threshold, the nonlinear changes of real texture features are distinguished from false texture features caused by noise, uneven lighting, or background interference. The verification result is generated based on the differentiation results.

[0017] Secondly, this application also discloses an SMT (Surface Mount Technology) component defect identification system for identifying SMT component solder joint defects. The system includes: The first image acquisition module is used to acquire the first image of the solder joint under the first lighting mode; The optical characteristic judgment module is used to determine whether there are any optical characteristic abnormalities in the first image based on the brightness distribution characteristics of the first image. The illumination mode switching module is used to adaptively switch to the second illumination mode when the first image has optical anomalies; The second image acquisition module is used to acquire a second image of the solder joint under the second lighting mode; The coarse location extraction module is used to obtain coarse location information of the solder joint area in the first image. The fine edge extraction module is used to obtain fine edge information of the solder joint area in the second image; The contour reconstruction module is used to filter and fuse the fine edge information based on the coarse position information in order to reconstruct the weld point contour. The defect judgment module is used to calculate the geometric information of the solder joint and judge the defects based on the reconstructed solder joint contour.

[0018] Beneficial effects The SMT (Surface Mount Technology) defect identification method disclosed in this application first acquires a first image of the solder joint under a first illumination mode. Based on the brightness distribution characteristics of the first image, it determines whether there are any optical anomalies. If an optical anomaly is detected in the first image, the system adaptively switches to a second illumination mode and acquires a second image of the solder joint under the second illumination mode. This adaptive illumination mode switching mechanism can select the most suitable illumination conditions to acquire high-quality images based on the actual optical characteristics of the solder joint, effectively overcoming the problem of insufficient image contrast under a single illumination mode and ensuring the effectiveness of subsequent image processing. Then, it acquires the approximate location information of the solder joint area in the first image and the second image... The fine edge information of the solder joint area can be fully utilized by combining image information acquired under different lighting modes: the first image may provide more stable coarse position information in some cases, while the second image provides clearer fine edge information under optimized lighting; finally, the fine edge information is filtered and fused according to the coarse position information to reconstruct the solder joint contour, and the geometric information of the solder joint is calculated and defect judgment is made based on the reconstructed solder joint contour. Through the above methods, this application effectively solves the problem of misjudgment in the case of abnormal optical characteristics of solder joints in existing AOI systems by adaptive lighting mode switching, multi-source information fusion and accurate contour reconstruction, and significantly improves the accuracy of SMT component defect identification. Attached Figure Description

[0019] Figure 1 A flowchart illustrating a method for identifying SMT (Surface Mount Technology) defects provided in this application; Figure 2 for Figure 1 A flowchart of step S105 is shown below; Figure 3 for Figure 1 Another flowchart of step S105 shown; Figure 4 A flowchart illustrating the step of generating a guided confidence map based on the texture features of a first image, as provided in this application; Figure 5 A flowchart illustrating the steps for generating a guided confidence map based on the modified local texture features provided in this application; Figure 6 for Figure 5 The flowchart of step S505 is shown below; Figure 7 This is another flowchart illustrating the step of generating a guided confidence map based on the texture features of the first image, as provided in this application. Figure 8 for Figure 7 The flowchart of step S701 is shown below; Figure 9 This is a schematic diagram of the structure of an SMT (Surface Mount Technology) defect identification system provided in this application. Detailed Implementation

[0020] The technical solutions of this application will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments. The components of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0021] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0022] participate Figure 1 This application proposes a method for identifying SMT (Surface Mount Technology) surface mount defects, including: Step S101: Obtain the first image of the solder joint under the first illumination mode, and determine whether there is any optical characteristic abnormality in the first image based on the brightness distribution characteristics of the first image.

[0023] In one implementation, an image of the solder joint under a first illumination mode can be captured by a camera in a machine vision system. For example, a high-brightness LED ring light source can be used as the first illumination mode, making the solder joint surface appear as a bright area while the surrounding substrate area appears as a low-brightness area, thus creating a clear brightness contrast. Subsequently, brightness distribution analysis is performed on the acquired first image, such as calculating the image's grayscale histogram, local contrast, or brightness mean and variance. By comparing these brightness distribution features with a preset normal solder joint brightness distribution model, it can be determined whether there are any optical anomalies in the current first image. For example, if the brightness mean of the solder joint area is significantly lower than the normal range, or the contrast between the solder joint and the background is lower than a certain threshold, it can be determined that there are optical anomalies.

[0024] Step S102: If the first image has optical anomalies, adaptively switch to the second illumination mode and acquire the second image of the solder joint under the second illumination mode.

[0025] When the system determines that the first image has optical anomalies, such as diffuse reflection caused by a semi-transparent film on the solder joint surface, the traditional first illumination mode may not provide a clear outline of the solder joint. To address this, the system automatically switches to a second illumination mode. For example, it can switch to a low-angle illumination mode, in which the light source illuminates the solder joint from the side, creating shadows or highlights at the edges of the solder joint, thus highlighting its outline. Subsequently, a second image of the solder joint is captured by the camera under the second illumination mode.

[0026] Step S103: Obtain the approximate location information of the solder joint area in the first image.

[0027] Step S104: Obtain fine edge information of the solder joint area in the second image.

[0028] In one implementation, an edge detection algorithm, such as Canny edge detection, the Sobel operator, or the Laplacian operator, can be applied to the second image to extract areas of dramatic brightness variation in the image, which typically correspond to the fine edges of solder joints. Since the second illumination mode is designed to highlight edge details, more accurate edge information can be obtained from the second image.

[0029] Step S105: Filter and fuse fine edge information based on coarse position information to reconstruct the weld point outline.

[0030] Step S106: Based on the reconstructed solder joint contour, calculate the solder joint geometric information and determine defects.

[0031] In one implementation, once the solder joint profile is reconstructed, a series of geometric parameters can be calculated based on that profile, such as the solder joint's area, perimeter, diameter, roundness, and centroid position. This geometric information is then compared with preset acceptable solder joint standards. For example, if the calculated solder joint area exceeds the allowable range, or the roundness is below a certain threshold, the solder joint can be determined to have a defect, such as too much solder, too little solder, or an abnormal shape.

[0032] This application automatically identifies various common solder joint defects, such as excessive solder, insufficient solder, bridging, cold solder joints, and tombstoning, by analyzing solder joint images. The "first illumination mode" and "second illumination mode" refer to different lighting conditions or configurations used to illuminate the solder joints to obtain images. For example, the first illumination mode can be commonly used coaxial lighting or ring lighting, designed to provide high-contrast solder joint images; the second illumination mode can be low-angle lighting, diffuse lighting, or polarized lighting, designed to highlight the edges or surface details of the solder joints, especially when the contrast is insufficient under the first illumination mode.

[0033] This application introduces anomaly detection in the optical characteristics of the first illumination mode to promptly identify optical changes. Once an anomaly is detected, the system adaptively switches to a second illumination mode designed to highlight solder joint edge details, acquiring effective edge information even in low-contrast conditions. Simultaneously, the first image is used to obtain coarse location information of the solder joint region, providing a general location even with low contrast. Subsequently, the fine edge information obtained from the second image is filtered and fused with the coarse location information from the first image. This strategy effectively utilizes the advantages of both illumination modes: the coarse location information provides a global, approximate solder joint region, while the fine edge information provides local, accurate contour details. Through this fusion, even when complete and accurate information cannot be obtained under a single illumination mode, a precise solder joint contour can be reconstructed. Therefore, based on the reconstructed accurate solder joint contour, reliable solder joint geometry information can be calculated, significantly reducing the false positive rate and improving the accuracy and robustness of defect identification. Compared to existing technologies that rely solely on a single illumination mode and a fixed image processing algorithm, the method of this application can better adapt to changes in optical properties that may occur during the production process, ensuring that the AOI system can still operate stably and efficiently in complex and ever-changing production environments, and avoiding the problem of the entire detection system being paralyzed due to minor changes in material properties.

[0034] In one embodiment, step S103 may specifically include: performing grayscale morphological processing on the first image to obtain candidate regions containing solder joints, thereby obtaining coarse location information of the solder joint regions. Specifically, performing grayscale morphological processing on the first image can be understood as using structuring elements to perform operations such as dilation, erosion, opening, or closing operations on the image. Thus, by changing the grayscale distribution of the image, the features of the solder joint regions are highlighted, while background noise is suppressed, thereby effectively obtaining candidate regions containing solder joints. For example, grayscale opening operations can be used to eliminate small bright spots and fine connections in the image, or grayscale closing operations can be used to fill small holes and areas of broken connections, making the solder joint regions more complete and prominent in the image.

[0035] See Figure 2 The above step S105 may specifically include the following steps: Step S201: Convert the candidate region into a binary mask image, wherein the pixel value of the candidate region in the first image is 1, and the pixel value of other regions outside the candidate region is 0.

[0036] Converting candidate regions into binary mask images involves setting the pixel values ​​of candidate regions obtained through grayscale morphological processing to a specific value (e.g., 1), while setting the pixel values ​​of non-candidate regions in the image to another specific value (e.g., 0). This creates a clear region indicator for subsequent precise filtering of fine edge information. In practical applications, this binary mask image can be considered a region of interest (ROI), defining the effective range of fine edge information.

[0037] Step S202: Obtain the edge pixels with a value of 1 at the corresponding position in the binary mask image from the fine edge information, and connect and close the edge pixels to reconstruct the weld point contour.

[0038] Furthermore, obtaining the edge pixels with a value of 1 at the corresponding position in the binary mask image from the fine edge information means using the generated binary mask image as a filter to retain only those fine edge pixels located within the candidate solder joint region. This eliminates irrelevant edge information in the background, preventing it from interfering with the solder joint contour reconstruction. Subsequently, these selected edge pixels are connected and closed, for example, through edge tracking algorithms or morphological closure operations, to form a complete and continuous solder joint contour.

[0039] The above technical solution employs grayscale morphological processing to obtain coarse positional information, effectively suppressing image noise and background interference, and improving the recognition accuracy of solder joint candidate regions. Furthermore, by converting the candidate region into a binary mask image and using it to filter fine edge information, the effective edge range can be precisely defined, significantly reducing interference from non-solder joint edges, allowing the subsequent contour reconstruction process to focus more on the solder joint itself.

[0040] In some preferred embodiments, a specific example is given below. Suppose that defect identification of a solder joint is required on an SMT assembly line. First, a first image of the solder joint is acquired using a first illumination mode. This image may have some background texture or uneven illumination. To obtain coarse location information of the solder joint area, grayscale morphological processing is performed on the first image. For example, grayscale opening can be performed on the image to eliminate small noise points and to perform preliminary smoothing and connection of the solder joint area, thereby obtaining a relatively complete candidate region containing the solder joint. Subsequently, a second image of the solder joint is acquired under a second illumination mode, and fine edge information is extracted from it. To use the coarse location information to filter and fuse these fine edges, the previously obtained candidate region is first converted into a binary mask image. In this mask image, the pixel values ​​within the solder joint candidate region are set to 1, while the pixel values ​​in other regions are set to 0. Then, all the fine edge pixels extracted from the second image are compared point-by-point with the binary mask image. Only edge pixels with a value of 1 at their corresponding positions in the binary mask image are retained. This means that all fine edges unrelated to the solder joint (such as edges from other components on the PCB or background textures) located outside the candidate solder joint area are effectively filtered out. Finally, these filtered edge pixels are connected and closed, for example using an edge-connection algorithm after Canny edge detection, to reconstruct an accurate and complete solder joint contour. In this way, even against complex backgrounds, the true boundaries of solder joints can be accurately identified, providing reliable geometric information for subsequent defect assessment.

[0041] In other embodiments of this application, the step of obtaining the rough location information of the solder joint area in the first image includes: generating a guided confidence map based on the texture features of the first image, wherein the value of each pixel in the guided confidence map represents the probability that it belongs to the solder joint.

[0042] See Figure 3 The step S105 above, which involves filtering and fusing fine edge information based on coarse position information to reconstruct the weld point contour, includes: Step S301: Weight the edge intensity of each edge pixel in the fine edge information with the confidence value of the edge pixel at the corresponding position in the guiding confidence map to obtain the weighted edge intensity; Step S302: Perform secondary thresholding on the weighted edge intensity to obtain edge pixels with higher than the weighted intensity threshold, and perform connection and closure processing on the edge pixels with higher than the weighted intensity threshold to reconstruct the solder joint contour.

[0043] Specifically, when obtaining the approximate location information of the solder joint region in the first image, the system no longer relies solely on simple grayscale or brightness features. Instead, it generates a guided confidence map by analyzing the texture features of the first image. Texture features can be understood as the grayscale variation patterns of pixels and their surrounding neighborhoods in the image, such as Local Binary Pattern (LBP), Gabor filter response, and Gray-Level Co-occurrence Matrix (GLCM). These texture features can more effectively capture the structural differences between the solder joint region and the background region. The guided confidence map is a two-dimensional matrix of the same size as the first image, where the value of each pixel represents the probability or likelihood that the pixel belongs to the solder joint region. This map can be generated by training a classifier (such as a support vector machine or neural network) based on the extracted texture features, or by using a pre-defined texture pattern matching algorithm, thereby providing a more accurate and robust preliminary estimate of the solder joint region's location.

[0044] Furthermore, when filtering and fusing fine edge information based on coarse location information to reconstruct the solder joint contour, this application employs a weighted fusion strategy. The fine edge information is typically extracted from the second image using edge detection algorithms (such as Canny, Sobel, etc.), containing detailed boundary information of the solder joint region, but may also include background noise or edges of non-solder joint structures. Edge strength refers to the gradient magnitude of grayscale changes at an edge pixel, reflecting the saliency of the edge. The confidence value originates from the generated guided confidence map, representing the probability that the edge pixel is located in the solder joint region. Weighted edge strength is obtained by weighting the edge strength of each edge pixel with its corresponding confidence value in the guided confidence map. This weighting operation aims to enhance the weight of edges located in high-confidence regions (i.e., more likely to be solder joint regions) while suppressing or weakening the weight of edges located in low-confidence regions (i.e., more likely to be background or noise). For example, multiplicative weighting or linear weighting can be used.

[0045] Subsequently, a secondary thresholding process is applied to the weighted edge strength. This "secondary thresholding" refers to applying one or more thresholds again after the weighting operation to further filter edge pixels. For example, a weighted strength threshold can be set; only edge pixels with a weighted edge strength higher than this threshold are considered valid solder joint edge pixels. This step aims to remove weak edges or noise that still exist after weighting, ensuring high reliability of the final selected edge pixels. Finally, these edge pixels with strengths higher than the weighted strength threshold are connected and closed. Connecting involves linking spatially adjacent valid edge pixels to form continuous edge segments; closing involves connecting the beginning and end of these edge segments to form a complete, closed solder joint contour. This can be achieved using morphological closure operations, contour tracking algorithms, or graph theory methods to recover smooth, complete solder joint boundaries from discrete edge pixels.

[0046] This application's solution significantly improves the accuracy and robustness of obtaining coarse solder joint location information by introducing a guided confidence map based on texture features. Traditional coarse positioning methods may perform poorly under complex backgrounds or lighting variations, while texture features can more effectively distinguish between solder joints and non-solder joint areas, allowing the generated guided confidence map to more accurately indicate the likelihood of solder joint presence. It is precisely this high-precision coarse location guidance that enables the subsequent screening and fusion process of fine edge information to focus more on the actual solder joint area. By weighting the intensity of fine edges with the confidence value of the guided confidence map, this application can effectively utilize coarse location information to "guide" the selection of fine edges, thereby preserving the true edge details of the solder joint while suppressing background noise and interfering edges. This weighted fusion mechanism ensures that only pixels with both high edge intensity and high confidence regions are retained, thus improving the reliability of edge information. Finally, through secondary thresholding and connection closure processing, a complete and accurate solder joint contour can be precisely reconstructed from these highly reliable edge pixels.

[0047] In some preferred embodiments, this application is implemented as follows: Suppose an SMT (Surface Mount Technology) defect identification system needs to inspect solder joints on a PCB board. First, the system acquires a first image of the solder joint area under a first illumination mode. To obtain coarse location information of the solder joint area, the system extracts texture features from the first image. For example, it can calculate the Local Binary Pattern (LBP) features of the neighborhood around each pixel, or use a Gabor filter bank to extract multi-scale, multi-directional texture information. Then, using these texture features, a pre-trained classification model (such as a random forest or convolutional neural network) is used to predict the probability that each pixel belongs to a solder joint, thereby generating a guided confidence map. In this map, the pixel values ​​of the solder joint area are higher, while the pixel values ​​of the non-solder joint area are lower.

[0048] Subsequently, the system switches to the second illumination mode, acquires a second image of the solder joint area, and performs Canny edge detection on this second image to obtain fine edge information of the solder joint area. This edge information includes the position of each edge pixel and its corresponding edge intensity. Next, for each fine edge pixel, the system queries its confidence value at the corresponding position in the previously generated guided confidence map. Then, the edge intensity of the edge pixel is weighted with the queried confidence value, for example, through a simple multiplication operation (weighted edge intensity = edge intensity × confidence value).

[0049] Finally, the system performs a secondary thresholding process on these weighted edge intensities. For example, a weighted intensity threshold is set, and only edge pixels with a weighted edge intensity exceeding this threshold are retained. These retained edge pixels are considered high-confidence solder joint edges. To form a complete solder joint contour, the system connects and closes these high-confidence edge pixels. For example, a graph search-based contour tracking algorithm can be used, or morphological operations such as dilation and erosion can be performed first to connect broken edges and close the contour. In this way, the complete contour of the solder joint can be accurately reconstructed, effectively avoiding false positives and false negatives even in the case of complex backgrounds or uneven lighting in the first image.

[0050] Further, see Figure 4 The steps described above, which generate a guided confidence map based on the texture features of the first image, where the value of each pixel in the guided confidence map represents the probability that it belongs to a solder joint, include: Step S401: Obtain the texture feature baseline of the background region of the first image; Step S402: Calculate the local texture features of each pixel in the first image; Step S403: Compare the local texture features with the texture feature baseline, and correct the local texture features according to the comparison results to suppress the contribution of the local texture features of the background region to the guiding confidence map; Step S404: Generate a guided confidence map based on the corrected local texture features.

[0051] Specifically, obtaining the texture feature baseline of the background region in the first image refers to extracting texture features from a pre-determined background region in the first image, or from a region identified through image analysis, to establish a standard reference representing the pure background texture. This texture feature baseline can be one or a set of numerical values ​​describing the background texture characteristics. For example, methods such as Gray-Level Co-occurrence Matrix (GLCM), Local Binary Pattern (LBP), or Gabor filters can be used to extract the texture features of the background region, and their statistical average or distribution can be used as the baseline, thus providing a reliable background reference for subsequent comparison of local texture features.

[0052] Calculating the local texture features of each pixel in the first image can be understood as analyzing each pixel and its surrounding neighborhood to quantify the texture information of the local region containing that pixel. This is typically achieved by sliding a window of a preset size across the image and calculating the texture descriptor within each window. The method for calculating local texture features should be consistent with the method for calculating the texture feature baseline to ensure comparability between the two.

[0053] In practical applications, local texture features are compared with a texture feature baseline. Based on the comparison results, the local texture features are corrected to suppress the contribution of local texture features from the background region to the guided confidence map. This comparison process aims to evaluate the similarity between the texture of each local region and the background texture. When a local texture feature shows a high similarity to the texture feature baseline, the local region is considered highly likely to belong to the background. In this case, the local texture feature should be corrected, for example, by reducing its eigenvalue, setting it to zero, or applying a decay function, thereby weakening its influence in the subsequent generation of the guided confidence map. This correction mechanism effectively filters out background noise, enabling the guided confidence map to more accurately reflect the true probability of the solder joint area.

[0054] Generating a guided confidence map based on corrected local texture features refers to constructing a confidence map using corrected, cleaner local texture features after effectively suppressing background texture contributions. This process typically involves converting the corrected texture features into confidence values ​​between 0 and 1 using a predefined mapping function (e.g., a non-linear function or a lookup table), where higher values ​​indicate a greater likelihood that the pixel belongs to a solder joint. In this way, texture information can be transformed into an intuitive representation of solder joint probabilities.

[0055] In some preferred embodiments, a specific example is given below. Suppose that during SMT (Surface Mount Technology) defect identification, it is necessary to generate a guided confidence map of solder joints. First, several regions known to be pure background, such as exposed portions of a PCB board, can be selected from the first image. Gray-Level Co-occurrence Matrix (GLCM) features are extracted from these regions, and their contrast, energy, homogeneity, and other statistics are calculated. The average or distribution of these statistics is used as the texture feature baseline. Next, a pixel-by-pixel sliding window process is performed on the first image. GLCM features are also calculated within a small window centered on each pixel to obtain the local texture features of each pixel. Then, each local texture feature is compared with the preset texture feature baseline using Euclidean distance or cosine similarity. If the similarity between a local texture feature and the texture feature baseline is higher than a certain threshold, the local area is considered to belong to the background, and its corresponding local texture feature value is corrected, for example, by multiplying its feature value by a decay factor less than 1, or by setting it to zero. Finally, the corrected local texture features are converted into confidence values ​​between 0 and 1 using a sigmoid function or a linear mapping function, thereby generating the final guided confidence map. For example, the higher the corrected feature value and the closer the confidence value is to 1, the greater the probability that it belongs to a solder joint.

[0056] Further, see Figure 5 The steps for generating the guided confidence map based on the corrected local texture features include: Step S501: Smooth the corrected local texture features to eliminate minor fluctuations in feature values; Step S502: Fill the smoothed local texture features to ensure the continuity of feature values; Step S503: Based on the local texture features after filling, determine the segmentation boundary of the feature value mapping to distinguish between solder joints and non-solder joint areas; Step S504: Generate a preliminary guided confidence map based on the segmentation boundaries; Step S505: Perform morphological processing on the preliminary guidance confidence map to eliminate isolated noise points and small holes, and smooth the boundaries to obtain the guidance confidence map.

[0057] The smoothing process for the corrected local texture features can be understood as applying specific filtering algorithms, such as Gaussian filtering, mean filtering, or median filtering, to reduce high-frequency noise in the feature map, thereby eliminating the small fluctuations introduced during image acquisition or feature calculation. This makes the local texture features more stable and reliable, providing a cleaner data foundation for subsequent processing.

[0058] Furthermore, filling the smoothed local texture features refers to filling in the tiny holes or broken areas that may exist in the feature map through interpolation, morphological closing operations or other connection algorithms, thereby ensuring the spatial continuity of feature values ​​and avoiding the incorrect segmentation or omission of solder joint areas due to local deficiencies.

[0059] Based on this, determining the segmentation boundary of the feature value mapping according to the local texture features after filling refers to dividing the feature value space into two parts representing the solder joint area and the non-solder joint area by setting one or more thresholds or using methods such as cluster analysis and edge detection. This can clearly distinguish the solder joint from the background and lay the foundation for generating a preliminary guided confidence map.

[0060] Therefore, based on the segmentation boundary, a preliminary guided confidence map is generated, which can be understood as directly mapping the segmentation result to confidence values. For example, pixels belonging to the solder joint area are assigned a high confidence value (such as 1), and pixels in the non-solder joint area are assigned a low confidence value (such as 0), thus initially constructing the confidence distribution of the solder joint area.

[0061] Finally, morphological processing is performed on the preliminary guidance confidence map. This involves refining the preliminary map through morphological operations such as erosion, dilation, opening, and closing to eliminate isolated noise points that may have been generated during the initial generation process, fill in small holes, and smooth the boundaries of the solder joint area, thereby obtaining a more accurate, complete, and clearly defined guidance confidence map.

[0062] The proposed solution effectively addresses the noise, discontinuities, and blurred boundaries that may exist in the original features by performing multi-stage refinement processing on the corrected local texture features. Smoothing effectively suppresses minor fluctuations in feature values, ensuring feature stability; filling compensates for local feature deficiencies, guaranteeing feature continuity; defining segmentation boundaries clearly distinguishes solder joints from non-solder joints, providing a clear basis for subsequent confidence map generation; morphological processing further optimizes the initial map, eliminating noise and holes, and smoothing boundaries, making the final guided confidence map more accurate and robust.

[0063] Further, see Figure 6 The steps described above for performing morphological processing on the preliminary guidance confidence map to eliminate isolated noise points and small holes, and to smooth the boundaries, to obtain the guidance confidence map, include: Step S601: Determine the size of the structural elements for morphological operations based on the local features of the preliminary guided confidence map; Step S602: Based on the curvature or connectivity of the local boundary of the solder joint region in the preliminary guided confidence map, select structuring elements with a size smaller than the first size threshold for morphological processing. Step S603: Based on the isolation or size of the background noise region in the preliminary guided confidence map, select structuring elements with a size greater than the second size threshold for morphological processing; Step S604: Based on the morphological processing results, obtain the guided confidence map.

[0064] Specifically, when performing morphological processing on the initial guided confidence map, the appropriate structuring element size must first be determined based on the map's local features. These local features can include pixel grayscale values, gradient information, texture information, or connectivity within their neighborhoods. For example, features can be evaluated by analyzing the pixel value distribution, edge density, or the magnitude of connected components in a local region.

[0065] Furthermore, for the solder joint region in the initial guided confidence map, its local boundaries typically exhibit high curvature or good connectivity, indicating that this region is a crucial component of the solder joint and requires meticulous processing to preserve its details. Therefore, structuring elements with dimensions smaller than a first size threshold are selected for morphological operations. This first size threshold can be set based on the typical size of the solder joint and the desired level of detail. For example, smaller structuring elements of 3x3 or 5x5 can be selected to perform operations such as erosion, dilation, opening, or closing, thereby smoothing the boundaries while maximizing the preservation of the solder joint's fine contour features.

[0066] Meanwhile, background noise regions in the initial guided confidence profile are typically characterized by isolated pixels, small patches, or irregular holes, exhibiting isolation or small size. To effectively eliminate this noise, structuring elements with a size larger than a second size threshold are selected for morphological processing. This second size threshold is usually larger than the first size threshold; for example, a 7x7 or larger structuring element can be selected. Through operations such as opening or closing, these isolated noise points and small holes can be effectively removed without significantly affecting the main solder joint area.

[0067] Therefore, by using structural elements of different sizes to perform morphological processing on different regions, optimized morphological processing results can be obtained, and a more accurate and robust guided confidence map can be generated.

[0068] See Figure 7 In another embodiment of this application, the step of generating a guided confidence map based on the texture features of the first image, wherein the value of each pixel in the guided confidence map represents the probability that it belongs to a solder joint, includes: Step S701: Perform multi-scale feature extraction on the first image to obtain local texture features at different scales; Step S702: Fuse local texture features at different scales to obtain fused features; Step S703: Based on the fusion features, convert the fusion features into confidence values ​​through nonlinear mapping to generate a guided confidence map.

[0069] Specifically, multi-scale feature extraction of the first image refers to extracting texture features from images with different spatial resolutions or different filter kernel sizes. For example, methods such as Gaussian pyramids, Laplacian pyramids, or wavelet transforms can be used to decompose the first image into multiple scales. Then, texture features such as Local Binary Pattern (LBP), Gray-Level Co-occurrence Matrix (GLCM), or Gabor filter responses can be calculated at each scale. This allows the texture information of the solder joints at different scales. For example, smaller scales can capture the microscopic texture details of the solder joint surface, while larger scales can capture the overall shape and macroscopic texture features of the solder joint, thus providing a more comprehensive texture description.

[0070] The fusion of local texture features at different scales can be understood as combining texture features extracted from different scales to form a more representative and robust feature vector. In practical applications, fusion methods can include feature vector concatenation, weighted averaging, principal component analysis (PCA) dimensionality reduction fusion, or learning-based fusion through neural network layers. This application comprehensively utilizes information from different scales to overcome the limitations of single-scale features, enhances the ability of features to represent the true texture of solder joints, and suppresses noise and background interference.

[0071] Furthermore, based on the fused features, a nonlinear mapping is used to convert the fused features into confidence values, generating a guided confidence map. The nonlinear mapping can employ the Sigmoid function, ReLU function, Softmax function, or other custom nonlinear transformation functions. For example, the fused features can be input into a pre-trained classifier (such as a Support Vector Machine (SVM), Logistic Regression, or a small neural network). After the output layer of this classifier is processed by a nonlinear activation function, the probability or confidence value of each pixel belonging to the solder joint can be obtained. By converting abstract texture features into intuitive and interpretable confidence values, the value of each pixel directly reflects its probability of belonging to the solder joint, providing reliable guidance for subsequent edge filtering and fusion.

[0072] The proposed solution extracts features from the first image at multiple scales, capturing texture information of solder joints at different granularities, thus effectively addressing the complexity and diversity of solder joint size, shape, and surface texture. For example, smaller solder joints or subtle defect textures can be captured through smaller-scale feature extraction, while larger solder joints or overall texture features can be obtained through larger-scale feature extraction. Subsequently, by fusing these local texture features at different scales, the advantages of each scale can be comprehensively utilized to form a more comprehensive and robust texture description. This fusion mechanism helps eliminate the limitations that may exist in single-scale features, such as sensitivity to noise or preference for specific texture patterns. Finally, the fused features are converted into confidence values ​​through a nonlinear mapping, assigning each pixel a quantified value representing the probability that it belongs to a solder joint. This nonlinear transformation can better fit the complex relationship between features and confidence, thereby generating a more accurate and reliable guided confidence map.

[0073] In some preferred embodiments, a specific example is given below. Suppose we need to identify solder joint defects of different sizes and surface gloss levels. First, the acquired first image is subjected to multi-scale Gaussian filtering to generate a series of images with different degrees of blur, representing different scales. Next, Local Binary Pattern (LBP) features are calculated on each scale image to capture local texture information. Then, these LBP feature vectors from different scales are concatenated and fused to form a high-dimensional fused feature vector. To convert this fused feature into confidence values, a simple fully connected neural network can be constructed. Its input layer receives the fused feature vector, the intermediate layers perform nonlinear transformations (e.g., using the ReLU activation function), and the output layer uses the Sigmoid activation function to map the output value between 0 and 1, representing the confidence that each pixel belongs to a solder joint. The resulting guided confidence map can more accurately reflect the approximate location of the solder joint area, providing reliable guided information even under complex solder joint textures or uneven lighting, thus effectively supporting subsequent solder joint contour reconstruction and defect judgment.

[0074] participate Figure 8 The steps described above for multi-scale feature extraction of the first image to obtain local texture features at different scales include: Step S801: Perform multi-scale preprocessing on the first image to obtain multiple preprocessed images at different scales. Multi-scale preprocessing includes denoising and illumination correction on the preprocessed images at different scales.

[0075] Specifically, multi-scale preprocessing refers to performing a series of operations on the original first image to generate multiple versions representing the image at different resolutions or levels of detail. These operations can include image scaling, Gaussian filtering, median filtering, etc. Denoising aims to eliminate random noise in the image caused by the sensor, transmission, or environment, for example, by applying algorithms such as Gaussian smoothing, bilateral filtering, or nonlocal mean denoising. Illumination correction aims to compensate for uneven lighting conditions in the image, ensuring a more consistent brightness distribution in the solder joint area, for example, through adaptive histogram equalization, gamma correction, or the Retinex algorithm, to provide a higher-quality and more stable image foundation for subsequent feature extraction.

[0076] Step S802: Calculate the local initial texture features of multiple preprocessed images at different scales.

[0077] Local initial texture features can be understood as quantified information describing the texture patterns of local regions in an image. These features can be calculated using methods such as Local Binary Pattern (LBP), Gabor filter response, Gray-Level Co-occurrence Matrix (GLCM), or wavelet transform. These features can capture the regularity of pixel intensity changes in an image, reflecting the unique texture structure of the solder joint surface.

[0078] Step S803: Perform spatial consistency verification on local initial texture features at multiple different scales to obtain the verification results. Spatial consistency verification is used to identify false texture features caused by noise, uneven lighting, or background interference.

[0079] In practical applications, spatial consistency verification refers to evaluating the consistency or correlation between local texture features at different scales or between adjacent regions within the same scale. For example, it can be done by comparing the similarity of feature vectors at the same spatial location at different scales, or by analyzing whether the statistical distribution of features within a local region conforms to the expected pattern, in order to distinguish between genuine solder joint texture features and false features caused by occasional noise, abnormal local lighting, or cluttered background textures.

[0080] Step S804: Based on the verification results, suppress false texture features; Specifically, suppressing false texture features refers to weakening or removing features identified as false based on the spatial consistency check results. For example, feature values ​​that do not meet the spatial consistency standard can be weighted and attenuated, directly set to zero, or excluded through masking operations, thereby reducing interference from false features in subsequent processing and improving feature purity.

[0081] Step S805: Perform inter-scale feature enhancement on the suppressed local initial texture features at multiple different scales to highlight the true optical properties of the solder joints at different scales, and obtain the enhanced local texture features at different scales.

[0082] Inter-scale feature enhancement can be understood as fusing or combining suppressed local texture features from different scales to strengthen the true solder joint features that are stable across multiple scales, while further weakening or eliminating noise or interference that appears at a single scale and lacks cross-scale consistency. For example, weighted averaging, maximum value fusion, or attention-based fusion methods can be used. This ensures that the final local texture features can more comprehensively and robustly reflect the true optical properties of the solder joint, providing more reliable input for subsequent guided confidence map generation.

[0083] In some preferred embodiments, a specific example is given below. Suppose that defect identification of solder joints on an SMT circuit board is required. First, a first image of the solder joints is acquired by a first image acquisition module under a first illumination mode. This first image may contain some random noise and localized uneven illumination.

[0084] To obtain local texture features at different scales, the first image is first preprocessed at multiple scales. For example, three preprocessed images at different scales can be generated: the original size, half the size, and a quarter the size. For each scale image, Gaussian filtering is applied to remove noise, and adaptive histogram equalization is performed to correct for uneven illumination.

[0085] Next, local initial texture features are calculated for each of the three preprocessed images at different scales. For example, a Local Binary Pattern (LBP) descriptor can be used to capture texture information around each pixel.

[0086] Then, spatial consistency checks are performed on these initial local texture features. Specifically, for each pixel location in the image, the similarity of its LBP feature vectors calculated at three different scales is compared. If the LBP feature of a pixel location at one scale differs significantly from the LBP features at other scales, or is inconsistent with the LBP features of surrounding pixels, then the feature is considered to be caused by noise or background interference and is marked as a false texture feature.

[0087] Based on the verification results, these fake texture features can be suppressed. For example, the weights of LBP feature vectors marked as fake can be reduced, or they can be set to zero.

[0088] Finally, inter-scale feature enhancement is performed on the suppressed initial local texture features. For example, a weighted average can be used to fuse the suppressed LBP features at three scales, where the weights are determined based on the confidence level of the spatial consistency check, i.e., features with high consistency are given higher weights. In this way, enhanced and more robust local texture features at different scales can be obtained. These features can more accurately reflect the true optical properties of the solder joints, providing high-quality input for the subsequent generation of the guiding confidence map.

[0089] Furthermore, the spatial consistency verification of local texture features at multiple different scales is performed to obtain the verification results. The steps of spatial consistency verification to identify false texture features caused by noise, uneven illumination, or background interference include: Calculate the nonlinear correlation measure of local initial texture features at different scales for each spatial location; Based on nonlinear correlation measurement and preset threshold, the nonlinear changes of real texture features are distinguished from false texture features caused by noise, uneven lighting or background interference. Based on the differentiation results, a verification result is generated.

[0090] Specifically, calculating the nonlinear correlation measure of local initial texture features at different scales for each spatial location refers to analyzing the degree of correlation between the local initial texture features extracted at different scales for each pixel in an image. This correlation is not a simple linear relationship, but rather captures more complex, nonlinear dependencies. For example, methods such as mutual information, kernel canonical correlation analysis (KCCA), or feature similarity measures based on deep learning can be used to quantify this nonlinear correlation. Mutual information can measure the degree of interdependence between two random variables and can effectively reflect the nonlinear statistical relationship between features at different scales.

[0091] The distinction between nonlinear variations in genuine texture features and false texture features caused by noise, uneven lighting, or background interference, based on nonlinear correlation metrics and preset thresholds, involves classifying these metrics by setting one or more thresholds after obtaining them. Genuine texture features, especially those in solder joint areas, typically exhibit a consistent or predictable nonlinear variation pattern at different scales, and their nonlinear correlation metrics are likely to be high and stable. Conversely, false texture features caused by noise, uneven lighting, or background interference may exhibit more random and inconsistent behavior at different scales, or their nonlinear correlation metrics may be low and fluctuate significantly. By comparing the metrics with preset thresholds, these two types of features can be effectively distinguished. For example, when the nonlinear correlation metric is higher than a certain threshold, it can be identified as a genuine texture feature; when it is lower than that threshold, it is identified as a false texture feature.

[0092] The step of generating a verification result based on the discrimination results means using the output of the discrimination process described above as the verification result. This verification result can be a binary mask image marking regions identified as false texture features, or a confidence map representing the probability that each pixel belongs to a false texture feature. This verification result will be used in subsequent steps to suppress false texture features, ensuring that only true and reliable texture features are used for subsequent inter-scale feature enhancement.

[0093] In some preferred embodiments, a specific example is given below. Suppose that during multi-scale feature extraction of the first image, multiple preprocessed images at different scales have been obtained, and the local initial texture features of these images have been calculated. For spatial consistency verification, the mutual information value between the local initial texture feature vectors at two adjacent scales (e.g., scale A and scale B) can be calculated for each spatial location. A higher mutual information value indicates that the features at these two scales are more correlated and more likely to be genuine texture features; conversely, a lower mutual information value indicates that they are more likely to be false texture features.

[0094] Specifically, a mutual information threshold T can be set. For each pixel in the image, if the mutual information value of its local initial texture feature vector at scale A and scale B is greater than T, the texture feature at that pixel is considered to have good spatial consistency and is judged as a real texture feature. If the mutual information value is less than or equal to T, the texture feature at that pixel is considered to be affected by noise, uneven illumination, or background interference and is judged as a false texture feature. Based on this distinction, a binary verification result map can be generated, which marks all pixels judged as false texture features. This verification result map will then be used to guide the suppression of false texture features, for example, by setting the texture feature values ​​of these false feature regions to zero or by weighted attenuation, so that in subsequent inter-scale feature enhancement, only the true optical properties of the solder joints at different scales are highlighted. This verification method based on nonlinear correlation can effectively improve the ability to identify false texture features under complex backgrounds and variable illumination conditions.

[0095] See Figure 9 This application also discloses an SMT (Surface Mount Technology) component defect identification system 900, used to perform the steps in the above-described SMT component defect identification method. The system includes: a first image acquisition module 91, an optical characteristic judgment module 92, an illumination mode switching module 93, a second image acquisition module 94, a coarse position extraction module 95, a fine edge extraction module 96, a contour reconstruction module 97, and a defect judgment module 98.

[0096] The first image acquisition module 91 is used to acquire a first image of the solder joint under a first illumination mode. For example, this module may include a high-resolution digital camera, which is used in conjunction with a ring light source or a coaxial light source to capture images of the solder joint under normal illumination conditions, thereby providing preliminary image data for optical characteristic assessment.

[0097] The optical characteristic judgment module 92 is used to determine whether the first image has optical characteristic anomalies based on the brightness distribution characteristics of the first image. This module can be an image processing unit, programmed to analyze features such as the grayscale histogram, local contrast, or brightness mean and variance of the input first image. For example, when the brightness mean of the solder joint area is detected to be significantly lower than a preset threshold, or the contrast between the solder joint and the background is significantly reduced, this module can output a judgment result of optical characteristic anomalies.

[0098] The illumination mode switching module 93 is used to adaptively switch to a second illumination mode when the first image shows an optical anomaly. This module can be a controller that can control the switching of different light sources or illumination configurations based on the output signal of the optical characteristic judgment module. For example, when an anomaly is detected, the module can instruct the system to switch from ring illumination to low-angle illumination or diffuse illumination to adapt to changes in the optical characteristics of the solder joint surface.

[0099] The second image acquisition module 94 is used to acquire a second image of the solder joint under the second illumination mode. This module may also include a high-resolution digital camera, but it works in conjunction with the light source in the second illumination mode. For example, after switching to low-angle illumination, the second image captured by this module is designed to highlight the edge information of the solder joint, providing clear edge details even when the overall contrast is not high.

[0100] The coarse location extraction module 95 is used to obtain coarse location information of the solder joint region in the first image. This module can be an image processing unit programmed to perform basic image segmentation operations on the first image, such as based on global thresholding or region growing algorithms, to identify potential solder joint regions with high brightness in the image. Thus, a general region containing the solder joint can be obtained, such as a rectangular box or a coarse binary mask. The above embodiments have already described obtaining coarse location information by performing grayscale morphological processing on the first image or generating a guided confidence map based on texture features, and will not be repeated here.

[0101] The fine edge extraction module 96 is used to acquire fine edge information of the solder joint region in the second image. This module can be an image processing unit programmed to apply standard edge detection algorithms, such as Canny, Sobel, or Laplacian operators, to the second image to extract pixels with drastic changes in brightness gradients. Since the second illumination mode is designed to highlight edges, this module can obtain more accurate and complete solder joint edge information from the second image than from the first image.

[0102] The contour reconstruction module 97 is used to filter and fuse fine edge information based on coarse location information to reconstruct the solder joint contour. This module can be an image processing unit programmed to treat the coarse location information as a region of interest and retain only the fine edge pixels falling within that region. For example, background noise and irrelevant edges can be effectively removed by performing a logical AND operation between the fine edge information and a mask formed by the coarse location information. Subsequently, the module performs connection and closure processing on the filtered edge pixels, such as using morphological closure operations or neighborhood analysis-based connection algorithms, to form a complete and accurate solder joint contour.

[0103] The defect judgment module 98 is used to calculate the geometric information of the solder joint and judge defects based on the reconstructed solder joint contour. This module can be a data analysis unit, programmed to extract various geometric parameters from the reconstructed solder joint contour, such as area, perimeter, diameter, roundness, and centroid. Subsequently, this calculated geometric information is compared with preset acceptance criteria. For example, if the area or roundness of the solder joint exceeds the allowable range, the module will determine that the solder joint has a defect and output the corresponding defect type and location information.

[0104] This application's SMT (Surface Mount Technology) defect recognition system effectively solves the problem of traditional automated optical inspection systems failing to accurately extract solder joint contours when faced with abnormal optical properties on the solder joint surface (such as diffuse reflection due to flux residue), due to significantly reduced image contrast. This system integrates multi-mode image acquisition, intelligent optical characteristic judgment and adaptive illumination switching, and multi-source information fusion strategies. Furthermore, this application can flexibly adapt to complex image acquisition environments, ensuring the acquisition of crucial information for accurate contour reconstruction under various optical conditions, thereby significantly improving the accuracy and robustness of defect recognition.

[0105] The above are merely embodiments of this application and are not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for identifying surface mount defects (SMT), characterized in that, include: Acquire the first image of the solder joint under the first illumination mode, and determine whether there are any optical anomalies in the first image based on the brightness distribution characteristics of the first image; If the first image has abnormal optical properties, the system will adaptively switch to the second illumination mode and acquire a second image of the solder joint under the second illumination mode. Obtain the approximate location information of the solder joint area in the first image; Obtain fine edge information of the solder joint area in the second image; The fine edge information is filtered and fused based on the coarse position information to reconstruct the weld point outline; Based on the reconstructed weld point contour, the weld point geometric information is calculated and defect judgment is performed.

2. The SMT chip defect identification method according to claim 1, characterized in that, The step of obtaining the approximate location information of the solder joint area in the first image includes: In the grayscale morphological processing of the first image, candidate regions containing solder joints are obtained; The step of filtering and fusing the fine edge information based on the coarse position information to reconstruct the solder joint outline includes: The candidate region is converted into a binary mask image, wherein the pixel value of the candidate region in the first image is 1, and the pixel value of other regions outside the candidate region is 0; The edge pixels with a value of 1 at the corresponding position in the binary mask image are obtained from the fine edge information, and the edge pixels are connected and closed to reconstruct the solder joint contour.

3. The SMT (Surface Mount Technology) defect identification method according to claim 1, characterized in that, The step of obtaining the rough location information of the solder joint area in the first image includes: generating a guided confidence map based on the texture features of the first image, wherein the value of each pixel in the guided confidence map represents the probability that it belongs to the solder joint; The step of filtering and fusing the fine edge information based on the coarse position information to reconstruct the solder joint outline includes: The edge intensity of each edge pixel in the fine edge information is weighted with the confidence value of the edge pixel at the corresponding position in the guiding confidence map to obtain the weighted edge intensity; The weighted edge intensity is subjected to a secondary thresholding process to obtain edge pixels with a value higher than the weighted intensity threshold. The edge pixels with a value higher than the weighted intensity threshold are then connected and closed to reconstruct the solder joint contour.

4. The SMT chip defect identification method according to claim 3, characterized in that, The step of generating a guided confidence map based on the texture features of the first image, wherein the value of each pixel in the guided confidence map represents the probability that it belongs to a solder joint, includes: Obtain the texture feature baseline of the background region of the first image; Calculate the local texture features of each pixel in the first image; The local texture features are compared with the texture feature baseline, and the local texture features are corrected according to the comparison results to suppress the contribution of local texture features in the background region to the guided confidence map. The guided confidence map is generated based on the corrected local texture features.

5. The SMT chip defect identification method according to claim 4, characterized in that, The step of generating the guided confidence map based on the corrected local texture features includes: The corrected local texture features are smoothed to eliminate minor fluctuations in feature values; The smoothed local texture features are filled to ensure the continuity of feature values; Based on the local texture features after filling, the segmentation boundary of the feature value mapping is determined to distinguish between solder joint and non-solder joint areas; Based on the segmentation boundaries, a preliminary guided confidence map is generated; The preliminary guidance confidence map is subjected to morphological processing to eliminate isolated noise points and small holes, and to smooth the boundaries, thus obtaining the guidance confidence map.

6. The SMT chip defect identification method according to claim 5, characterized in that, The step of performing morphological processing on the preliminary guidance confidence map to eliminate isolated noise points and small holes, and smooth the boundaries to obtain the guidance confidence map includes: Based on the local features of the preliminary guided confidence map, the size of the structural element for morphological operations is determined; Based on the curvature or connectivity of the local boundaries of the solder joint region in the preliminary guided confidence map, structural elements with a size smaller than the first size threshold are selected for morphological processing. Based on the isolation or size of the background noise region in the preliminary guided confidence map, structural elements with a size greater than the second size threshold are selected for morphological processing. Based on the morphological processing results, the guidance confidence map is obtained.

7. The SMT (Surface Mount Technology) defect identification method according to claim 3, characterized in that, The step of generating a guided confidence map based on the texture features of the first image, wherein the value of each pixel in the guided confidence map represents the probability that it belongs to a solder joint, includes: Multi-scale feature extraction is performed on the first image to obtain local texture features at different scales; The local texture features at different scales are fused to obtain fused features; Based on the fusion features, the fusion features are converted into confidence values ​​through nonlinear mapping to generate the guided confidence map.

8. The SMT chip defect identification method according to claim 7, characterized in that, The step of performing multi-scale feature extraction on the first image to obtain local texture features at different scales includes: The first image is subjected to multi-scale preprocessing to obtain multiple preprocessed images of different scales. The multi-scale preprocessing includes denoising and illumination correction of the preprocessed images of different scales. Calculate the local initial texture features of the multiple preprocessed images at different scales; Spatial consistency verification is performed on the local initial texture features at multiple different scales to obtain the verification results. The spatial consistency verification is used to identify false texture features caused by noise, uneven lighting, or background interference. Based on the verification results, the false texture features are suppressed; Inter-scale feature enhancement is performed on the suppressed local initial texture features at multiple different scales to highlight the true optical properties of the solder joints at different scales, resulting in enhanced local texture features at different scales.

9. The SMT (Surface Mount Technology) defect identification method according to claim 8, characterized in that, The step of performing spatial consistency verification on the local texture features at multiple different scales to obtain verification results, wherein the spatial consistency verification is used to identify false texture features caused by noise, uneven illumination, or background interference, includes: Calculate the nonlinear correlation measure of local initial texture features at different scales for each spatial location; Based on the nonlinear correlation metric and preset threshold, the nonlinear changes of real texture features are distinguished from false texture features caused by noise, uneven lighting, or background interference. The verification result is generated based on the differentiation result.

10. A surface mount defect identification system (SMT) for identifying SMT solder joint defects, characterized in that, The system includes: The first image acquisition module is used to acquire the first image of the solder joint under the first lighting mode; The optical characteristic judgment module is used to determine whether there are any optical characteristic abnormalities in the first image based on the brightness distribution characteristics of the first image. The illumination mode switching module is used to adaptively switch to the second illumination mode when the first image has optical anomalies; The second image acquisition module is used to acquire a second image of the solder joint under the second lighting mode; The coarse location extraction module is used to obtain coarse location information of the solder joint area in the first image; The fine edge extraction module is used to obtain fine edge information of the solder joint area in the second image; The contour reconstruction module is used to filter and fuse the fine edge information based on the coarse position information in order to reconstruct the weld point contour. The defect judgment module is used to calculate the geometric information of the solder joint and judge the defects based on the reconstructed solder joint contour.

Images