A method for extracting a bga solder joint profile and an electronic device

Abstract

This invention relates to a method and electronic device for extracting BGA solder joint contours. The method includes: preprocessing an original image to obtain a preprocessed image, the preprocessed image including multiple initial connected components containing solder joint contours; eliminating background foreign object adhesion in the preprocessed image based on solder joint features to separate the initial connected components, obtaining a secondary processed image; calculating the fitted diameter of each connected component in the secondary processed image, and obtaining a reference diameter of the solder joint using the fitted diameter; eliminating foreign object interference in the secondary processed image based on the reference diameter to obtain a tertiary processed image; iteratively updating the reference diameter in the tertiary processed image, removing connected components outside a preset diameter deviation range, and obtaining the solder joint contour. This invention can eliminate background foreign object adhesion in the image, eliminate foreign object interference on the solder joint contour, obtain the reference diameter of the solder joint, does not require a priori actual solder joint diameter, removes connected components that do not meet the diameter requirements, and obtains accurate solder joint contours.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method for extracting BGA solder joint contours and an electronic device. Background Technology

[0002] Solder joint contour extraction typically employs the following methods:

[0003] First, a preliminary solder joint outline is extracted based on a global threshold method. Then, the outline is filtered by combining preset feature parameters such as solder joint roundness and diameter, thereby completing the solder joint extraction.

[0004] 2. Based on edge detection and circle detection, the contour of the solder joint is initially extracted. Then, the contour is filtered by pre-set feature parameters such as the roundness and diameter of the solder joint to complete the extraction of the solder joint.

[0005] Third, the solder joint contour is initially extracted based on the watershed algorithm, and then the contour is filtered by combining preset feature parameters such as solder joint roundness and diameter to complete the solder joint extraction.

[0006] Fourth, the solder joint contour is initially extracted based on the region growing algorithm, and then the contour is filtered by pre-set feature parameters such as solder joint roundness and diameter to complete the solder joint extraction.

[0007] However, when extracting the weld joint contour using the above methods, the global threshold-based method is difficult to solve the problem of weld joints adhering to interfering foreign objects in complex situations; the edge detection-based and watershed algorithm-based methods are difficult to solve the problems of edge breakage and foreign object edge interference in complex situations; the region growth-based method is difficult to determine stable growth stopping parameters and is easily affected by background interference. Furthermore, all four methods rely on prior parameters such as the actual weld joint diameter, requiring operators to manually input the actual weld joint diameter, which is time-consuming, labor-intensive, and inflexible. Summary of the Invention

[0008] The purpose of this invention is to provide a BGA solder joint contour extraction method and electronic device that can eliminate interference from foreign objects in the background without requiring prior parameters such as the actual solder joint diameter, thereby accurately extracting the solder joint contour of the BGA.

[0009] In a first aspect, embodiments of the present invention provide a method for extracting BGA solder joint contours, the method comprising:

[0010] The original image is preprocessed to obtain a preprocessed image, which includes multiple initial connected regions containing solder joint outlines.

[0011] Based on the characteristics of the solder joints, background foreign matter adhesion in the preprocessed image is eliminated to separate the initial connected regions and obtain a secondary processed image;

[0012] Calculate the fitted diameter of each connected region in the secondary processed image, and obtain the reference diameter of the solder joint through the fitted diameter;

[0013] The foreign object interference in the secondary processed image is eliminated according to the reference diameter to obtain the tertiary processed image;

[0014] The reference diameter in the three-level processed image is updated cyclically, and the connected components other than the preset diameter deviation amplitude are removed to obtain the weld point contour.

[0015] In some embodiments, the preprocessing of the original image to obtain a preprocessed image includes a plurality of initial connected regions containing solder joint contours, including:

[0016] The original image is subjected to noise reduction processing to obtain a first intermediate image;

[0017] The first intermediate image is subjected to threshold segmentation to obtain the preprocessed image, which includes multiple initial connected regions containing solder joint contours.

[0018] In some embodiments, the first intermediate image is thresholded to obtain the preprocessed image, the preprocessed image comprising a plurality of initial connected components containing solder joint contours, including:

[0019] Calculate the grayscale values ​​of all pixels in the first intermediate image;

[0020] The grayscale values ​​of all pixels in the first intermediate image are divided into foreground pixel values ​​and background pixel values ​​to determine the segmentation threshold;

[0021] Based on the segmentation threshold, the first intermediate image is segmented to obtain the preprocessed image.

[0022] In some embodiments, dividing the grayscale values ​​of all pixels in the first intermediate image into foreground pixel values ​​and background pixel values ​​to determine a segmentation threshold includes:

[0023] The grayscale value with the largest variance between the foreground pixel value and the background pixel value is used as the segmentation threshold.

[0024] In some embodiments, the step of eliminating background foreign matter adhesion in the preprocessed image based on solder joint features to separate the initial connected components and obtain a secondary processed image includes:

[0025] In the first intermediate image, extract the region of interest corresponding to the first intermediate connected region other than the solder joint features, where the first intermediate connected region is one or more of the plurality of initial connected regions;

[0026] Based on an adaptive threshold, threshold segmentation is performed on the region of interest in the preprocessed image to separate the background foreign objects adhering to the preprocessed image.

[0027] In some embodiments, the step of performing threshold segmentation on the region of interest in the preprocessed image based on an adaptive threshold to separate background foreign object adhesions in the preprocessed image includes:

[0028] If the pixel value at the coordinates of the region of interest is greater than the adaptive threshold, then the pixel value at the coordinates of the region of interest that is greater than the adaptive threshold is set to 0, so as to eliminate the background foreign object adhesion in the preprocessed image and obtain the second intermediate image.

[0029] If the pixel value at the coordinates in the region of interest is less than the adaptive threshold, then the pixel value at the coordinates in the region of interest that is less than the adaptive threshold is set to a preset constant to obtain a third intermediate image, wherein the third intermediate image contains a plurality of second intermediate connected regions that conform to the solder joint features.

[0030] The second intermediate image and the third intermediate image are merged to obtain the secondary processed image;

[0031] The adaptive threshold is determined based on the mean value of the neighborhood centered at the coordinates in the region of interest, with a radius equal to the length of the shorter side of the region of interest.

[0032] In some embodiments, calculating the fitted diameter of each connected component in the secondary processed image and obtaining the reference diameter of the solder joint using the fitted diameter includes:

[0033] Calculate the fitted diameter of all connected components in the secondary processed image;

[0034] Calculate the mean of the fitted diameters to obtain the reference diameter of the weld joint.

[0035] In some embodiments, obtaining a tertiary processed image by eliminating foreign object interference in the secondary processed image based on the reference diameter includes:

[0036] Based on the reference diameter, morphological opening operations are performed on the connected components in the secondary processed image to eliminate foreign object interference in the secondary processed image and obtain the tertiary processed image.

[0037] In some embodiments, the step of cyclically updating the reference diameter in the third-level processed image, removing connected components other than the preset diameter deviation magnitude, and obtaining the solder joint contour includes:

[0038] The fitted diameter of all connected components in the three-level processed image is calculated iteratively to update the reference diameter in the three-level processed image as the final reference diameter;

[0039] The preset diameter deviation amplitude is determined based on the final reference diameter, the lower limit of diameter deviation, and the upper limit of diameter deviation.

[0040] Remove the connected regions that do not conform to the preset diameter deviation range to obtain the weld point profile.

[0041] In some embodiments, after obtaining the solder joint profile, the method further includes:

[0042] Combine the original image and the solder joint outline.

[0043] In a second aspect, embodiments of the present invention provide an electronic device, the electronic device comprising:

[0044] At least one processor, and

[0045] A memory, communicatively connected to the at least one processor, stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to perform the method described above.

[0046] Thirdly, embodiments of the present invention provide a non-volatile computer-readable storage medium storing computer-executable instructions, which, when executed by an electronic device, cause the electronic device to perform the method described above.

[0047] The BGA solder joint contour extraction method and electronic device of this invention preprocesses the original image to eliminate random noise interference and threshold segmentation, obtaining a preprocessed image of an initial connected region including multiple solder joint contours. Then, based on solder joint features, background foreign object adhesion in the preprocessed image is eliminated to separate the initial connected region, obtaining a secondary processed image, thereby eliminating background foreign object adhesion. The fitted diameter of each connected region in the secondary processed image is calculated, and the reference diameter of the solder joint is obtained through the fitted diameter, without requiring the prior actual solder joint diameter. Based on the reference diameter, foreign object interference in the secondary processed image is eliminated to obtain a tertiary processed image. The reference diameter in the tertiary processed image is cyclically updated, and connected regions outside the preset diameter deviation amplitude are removed to obtain the solder joint contour. Connected regions outside the reference diameter requirement are removed to obtain an accurate solder joint contour. Attached Figure Description

[0048] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0049] Figure 1 This is a flowchart illustrating an embodiment of the BGA solder joint contour extraction method of the present invention;

[0050] Figure 2 This is a schematic diagram of the original solder joint image of an embodiment of the BGA solder joint contour extraction method of the present invention;

[0051] Figure 3 This is a schematic diagram of the first intermediate image of an embodiment of the BGA solder joint contour extraction method of the present invention;

[0052] Figure 4 This is a schematic diagram of the initial connected components of an embodiment of the BGA solder joint contour extraction method of the present invention;

[0053] Figure 5 This is an intermediate connected region of an embodiment of the BGA solder joint contour extraction method of the present invention;

[0054] Figure 6 This is a schematic diagram of the opening operation result of an embodiment of the BGA solder joint contour extraction method of the present invention;

[0055] Figure 7 This is a schematic diagram showing the result of removing connected regions that do not conform to the preset deviation diameter in one embodiment of the BGA solder joint contour extraction method of the present invention;

[0056] Figure 8 This is a schematic diagram of the solder joint contour of an embodiment of the BGA solder joint contour extraction method of the present invention;

[0057] Figure 9 This is a schematic diagram of a structure of an embodiment of the BGA solder joint contour extraction device of the present invention;

[0058] Figure 10 This is a schematic diagram of the hardware structure of the controller in one embodiment of the electronic device of the present invention. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0060] The BGA solder joint contour extraction method and apparatus provided in this invention can be applied to electronic devices.

[0061] It is understandable that electronic devices are equipped with a controller, which acts as the main control center. After acquiring the original image of the solder joint, the controller preprocesses the image, identifies and separates connected components of foreign objects, and eliminates interference from foreign objects to determine the reference diameter of the solder joint. Based on the reference diameter, unqualified connected components are removed to obtain the solder joint outline.

[0062] The solder joint contour extraction in this application can be for BGA solder joint contour extraction. BGA (Ball Grid Array) is a high-density surface mount packaging technology. On the bottom of the package, the leads are all spherical and arranged in a grid-like pattern. Therefore, the shape of the BGA solder joint contour is usually circular.

[0063] Please see Figure 1 , Figure 1 This is a flowchart illustrating the BGA solder joint contour extraction method provided in an embodiment of the present invention. The method can be executed by a controller in an electronic device, such as... Figure 1 As shown, the method includes:

[0064] S101: Preprocess the original image to obtain a preprocessed image, the preprocessed image including multiple initial connected regions containing solder joint contours.

[0065] Original image as Figure 2 As shown, in Figure 2 As can be seen, the original image includes images of other foreign objects besides the solder joints. These could be excess solder or other foreign objects that have fallen onto the PCB board. The original image can be a grayscale image.

[0066] Since electronic devices cannot directly observe the solder joint outline like the human eye, they need to perform a series of image processing steps to accurately extract the solder joint outline.

[0067] In some implementations, step S101 may include:

[0068] The original image is denoised to obtain the first intermediate image;

[0069] The first intermediate image is thresholded to obtain a preprocessed image, which includes multiple initial connected regions containing solder joint contours.

[0070] Specifically, the original image is first denoised by filtering it to eliminate random noise interference. Filters such as Gaussian filters and mean filters can be used. Taking a Gaussian filter as an example, the filtering process is performed using Formula 1, as follows:

[0071] g = I * k (Formula 1)

[0072] Where I represents the original image, k represents the filter kernel, g represents the filtered result, which is the first intermediate image, and * represents convolution.

[0073] In the Figure 2 After filtering the original image of the solder joints, the following image was obtained: Figure 3 The first intermediate image shown.

[0074] After obtaining the first intermediate image, threshold segmentation is performed on the first intermediate image to obtain a preprocessed image, which includes multiple initial connected regions containing solder joint contours.

[0075] In some embodiments, a threshold segmentation is performed on the first intermediate image to obtain a preprocessed image. The preprocessed image includes multiple initial connected regions containing solder joint contours, and may include:

[0076] Calculate the grayscale values ​​of all pixels in the first intermediate image;

[0077] The grayscale values ​​of all pixels in the first intermediate image are divided into foreground pixel values ​​and background pixel values ​​to determine the segmentation threshold;

[0078] Based on the segmentation threshold, the first intermediate image is segmented to obtain the initial connected components.

[0079] Specifically, thresholding can be performed on the first intermediate image using a global thresholding method, such as the Otsu algorithm, also known as the maximum inter-class variance method. This algorithm determines the threshold by segmenting the foreground and background using a threshold. The optimal threshold is determined by maximizing the inter-class variance. Based on the grayscale characteristics of the image, the image is divided into background and foreground. Segmentation that maximizes the inter-class variance means minimizing the probability of misclassification.

[0080] Therefore, we can first calculate the grayscale values ​​of all pixels in the first intermediate image. Then, in order to classify all pixels into foreground and background, we use the Otsu algorithm to iterate through the grayscale values ​​of all pixels in the first intermediate image to maximize the variance between the foreground and background, thereby obtaining the segmentation threshold. The inter-class variance is calculated using Formula 2, as follows:

[0081] σ 2 =p1p2(m1-m2) 2Formula 2;

[0082] Where σ represents the inter-class variance, p1 and p2 represent the probabilities of foreground and background pixels appearing in the first intermediate image, respectively, and m1 and m2 represent the gray mean values ​​of foreground and background pixels, respectively.

[0083] Furthermore, p1 and p2, m1 and m2 are calculated using the following formulas 3-6, which are shown below:

[0084]

[0085] p2=1-p1 Formula 4;

[0086]

[0087]

[0088] Where, p i The probability that a pixel with a gray value of i appears in the first intermediate image is represented by k, k represents the gray level being traversed, and L represents all possible gray levels. For an 8-bit grayscale image, L = 256; for a 16-bit grayscale image, L = 65536.

[0089] Using formulas 2-6, let k iterate through all possible gray levels from 0 to L-1 to maximize the variance σ between foreground and background pixel values. The resulting gray value is then determined as the segmentation threshold. For example, taking an 8-bit grayscale image as an example, the determined segmentation threshold is 100.

[0090] Based on a segmentation threshold, image segmentation is performed on the first intermediate image to obtain a preprocessed image. For example, the grayscale value of pixels greater than the segmentation threshold is set to 255, and the grayscale value of pixels less than the segmentation threshold is set to 0, thus completing image segmentation. Figure 3 After thresholding, the preprocessed image is as follows: Figure 4 As shown, it is obvious that in Figure 4 The initial connected region includes multiple solder joint profiles.

[0091] By preprocessing the original image, a preprocessed image containing the initial connected components of the solder joint contour is obtained, eliminating random noise interference in the original image and allowing for the preliminary determination of the connected components of the solder joint contour.

[0092] S102: Eliminate background foreign matter adhesion in the preprocessed image based on the characteristics of the solder joints to separate the initial connected regions and obtain the secondary processed image.

[0093] Since the solder joint outline is typically circular, the roundness value of each initial connected component in the preprocessed image can be calculated. The roundness value of each initial connected component is calculated using the following formula 7:

[0094]

[0095] Where Roundness represents the roundness value, S represents the area of ​​the connected region, and C represents the perimeter of the connected region.

[0096] Based on the unique circular shape of solder joints and the diameter of the circle, workers can set a preset roundness threshold range as a characteristic of the solder joint, based on practical experience. For example, the preset roundness threshold range is 40% to 80%.

[0097] It is obvious that the first intermediate connected region that does not meet the roundness threshold is likely to be foreign objects adhering to the background. Therefore, when extracting the solder joint contour, it is necessary to solve the problem of solder joints adhering to interfering foreign objects.

[0098] To filter out foreign objects that may be adhering to the background, the roundness values ​​of each initial connected region can be compared with a preset roundness threshold range for preliminary screening. This allows for the selection of connected regions that meet the preset roundness threshold range and those that do not, thereby determining the connected regions that meet the solder joint characteristics and those that do not.

[0099] Specifically, during the screening process, each initial connected component is compared with a preset roundness threshold range. The initial connected components corresponding to roundness values ​​within the preset roundness threshold range are retained, while the initial connected components corresponding to roundness values ​​outside the roundness threshold range are extracted. Thus, in the first intermediate image, the first intermediate connected components that do not meet the solder joint features and the second intermediate connected components that meet the solder joint features can be screened out.

[0100] In some implementations, step S102 may include:

[0101] In the first intermediate image, the region of interest corresponding to the first intermediate connected region other than the solder joint features is extracted. The first intermediate connected region is one or more of a plurality of initial connected regions.

[0102] Based on adaptive thresholding, threshold segmentation is performed on the region of interest in the preprocessed image to separate the background foreign objects adhering to the preprocessed image.

[0103] Specifically, after determining the first intermediate connected region other than the solder joint features, the region of interest corresponding to the first intermediate connected region is extracted in the first intermediate image. Then, an adaptive threshold is obtained. Based on the adaptive threshold, the region of interest is segmented in the preprocessed image to separate the background foreign object adhesion in the preprocessed image.

[0104] During threshold segmentation, if the pixel value at the coordinates of the region of interest is greater than the adaptive threshold, the pixel value at the coordinates of the region of interest that is greater than the adaptive threshold is set to 0 to eliminate the adhesion of background foreign objects in the preprocessed image and obtain the second intermediate image.

[0105] If the pixel value at the coordinates in the region of interest is less than the adaptive threshold, then the pixel value at the coordinates in the region of interest that is less than the adaptive threshold is set to a preset constant to obtain a third intermediate image. The third intermediate image contains multiple second connected regions that conform to the solder joint features.

[0106] Furthermore, the separation of adhering foreign objects in the background can be achieved using Formula 8:

[0107]

[0108] In this context, ROI(i,j) represents the pixel value at coordinates (i,j) within the region of interest (ROI). MAX_VALUE is a preset constant, typically 255 for 8-bit grayscale images and 65535 for 16-bit grayscale images, used to represent the foreground grayscale value. T represents the adaptive threshold, determined by the mean of the neighborhood centered at coordinates (i,j) within the ROI, with a radius equal to the shorter side of the ROI. For example, it can be equal to the mean of the neighborhood centered at coordinates (i,j) with a radius of kr. This mean can be determined based on the operator's experience, being approximately 1 / 4 of the shorter side of the ROI. However, the value of the mean is not limited to 1 / 4 of the shorter side of the ROI and is not specified here, thus determining the corresponding adaptive threshold. ROI_Bin(i,j) represents the segmentation result, separating the connected components of the foreign object adhering to the background.

[0109] When extracting the region of interest (ROI) for image segmentation, adaptive thresholding, mean thresholding, ISODATA algorithm, or Li minimum cross-entropy thresholding algorithm can be used; no specific method is specified here.

[0110] By comparing the pixel values ​​at coordinates within the region of interest with an adaptive threshold, background foreign matter adhesion in the preprocessed image can be eliminated, resulting in a second intermediate image. Alternatively, a third intermediate image can be obtained, containing multiple second intermediate connected regions that conform to solder joint features. Then, the second and third intermediate images are merged to obtain a second-level processed image. The second-level processed image represents the result of eliminating background foreign matter adhesion, such as... Figure 5 As shown.

[0111] S103: Calculate the fitted diameter of each connected region in the secondary processed image, and obtain the reference diameter of the solder joint through the fitted diameter.

[0112] like Figure 5 As shown, the secondary image processing only eliminates the interference of background foreign objects adhering to the image, but other foreign objects still exist.

[0113] In some embodiments, step S103 may include:

[0114] Calculate the fitted diameter of all connected components in the secondary processed image;

[0115] Calculate the mean of the fitted diameters to obtain the reference diameter of the weld joint.

[0116] Specifically, since there is no prior actual solder joint diameter, a precise solder joint reference diameter is needed. To accurately calculate the reference diameter of the solder joint, the fitted diameters of all connected components in the secondary processing image are first calculated, and then the average of multiple fitted diameters is calculated. For a single connected component in the secondary processing image, the fitted diameter is calculated using the following formula 9:

[0117]

[0118] Where, x C y C D C represents the x-coordinate of the fitted circle center, the y-coordinate of the fitted circle center, and the diameter of the fitted circle, respectively; index i represents the i-th contour point in the current connected domain.

[0119] After calculating the fitted diameter of a single connected region, the average of the fitted diameters corresponding to multiple connected regions is taken to obtain the reference diameter of the solder joint.

[0120] S104: Obtain the third-level processed image by eliminating foreign object interference in the second-level processed image based on the reference diameter.

[0121] After obtaining the reference diameter of the solder joint, morphological opening operations are performed on the connected components in the secondary processed image based on the reference diameter to eliminate foreign object interference in the secondary processed image and obtain the tertiary processed image.

[0122] Furthermore, the shape opening operation can be represented by the following formula 10:

[0123] Bin o (i,j)=MorphOpen(Bin(i,j)) Formula 10;

[0124] Where MorphOpen represents the opening operation, and Bin(i,j) represents the binary image of the secondary processed image. o (i,j) represents the binary image after eliminating foreign object interference.

[0125] Then, based on the reference diameter of the solder joint, the filter core size for the opening operation is determined according to a preset size ratio. This preset size ratio can be set based on the actual size of the foreign object, for example, taking 5% to 40% of the initial solder joint reference diameter. After morphological opening operation processing, the following is obtained: Figure 6 The result of the opening operation.

[0126] It is understandable that the larger the core size of the open-operation filter, the larger the size of the foreign matter that will be filtered out.

[0127] Furthermore, this application is not limited to morphological opening operations to eliminate foreign object interference; contour analysis-based methods can also be used to eliminate foreign object interference, so it is not limited here.

[0128] S105: Iteratively update the reference diameter in the three-level processed image, remove connected components other than the preset diameter deviation amplitude, and obtain the weld point contour.

[0129] After obtaining the third-level processed image, Equation 9 is used to iteratively calculate the fitted diameter of all connected components in the third-level processed image to update the reference diameter in the third-level processed image. That is, the fitted diameter of all connected components is calculated again based on the result of the opening operation, and then the mean of all fitted diameters is calculated to update the reference diameter of the solder joint, thereby obtaining a more accurate reference diameter of the solder joint as the final reference diameter.

[0130] Next, based on the final reference diameter, the lower limit of diameter deviation, and the upper limit of diameter deviation, the preset diameter deviation amplitude is determined; connected components that do not conform to the preset diameter deviation amplitude are removed to obtain the solder joint profile.

[0131] Specifically, the preset diameter deviation range can be set according to actual testing requirements, generally ranging from 50% to 100%. Based on the final reference diameter and the preset diameter deviation range, connected components that do not meet the diameter requirements in the opening operation result can be removed. These unsuitable connected components are either too large or too small. The preset diameter deviation range can be determined based on the final reference diameter of the solder joint, the lower limit of the diameter deviation, and the upper limit of the diameter deviation. Connected components that do not meet the preset diameter deviation range are then removed to obtain the solder joint profile. That is, for the i-th connected component, the retention or rejection is determined according to the following formula 11:

[0132] D R *(1-P L )≤D i ≤D R *(1+P U ) Formula 11;

[0133] Among them, D R P represents the final reference diameter of the solder joint. L P represents the percentage of the lower limit of the diameter requirement, used to remove excessively small connected components;U This represents the upper limit percentage of the deviation in the diameter requirement, used to remove excessively large connected components. Di is the fitted diameter of the i-th connected component. If Equation 11 holds, the current connected component is retained; otherwise, it is discarded. Figure 7 As shown, Figure 7 This is a schematic diagram showing the result of removing connected regions that do not conform to the preset diameter deviation. After completing this step, the final solder joint extraction result is obtained as the solder joint outline.

[0134] After obtaining the solder joint outline, combine the original image and the solder joint outline, as follows: Figure 8 As shown. By combining the original image and the solder joint outline, the relative position of the solder joint outline in the original image can be determined.

[0135] In the embodiments of this application, by preprocessing the original image, random noise interference and threshold segmentation can be eliminated to obtain a preprocessed image of an initial connected region including multiple solder joint contours. Then, background foreign object adhesion in the preprocessed image is eliminated according to the solder joint features to separate the initial connected region and obtain a secondary processed image, thereby eliminating background foreign object adhesion. The fitted diameter of each connected region in the secondary processed image is calculated, and the reference diameter of the solder joint is obtained through the fitted diameter. The actual solder joint diameter is not required beforehand. Foreign object interference in the secondary processed image is eliminated according to the reference diameter to obtain a tertiary processed image. The reference diameter in the tertiary processed image is updated iteratively. Connected regions other than the preset diameter deviation amplitude are removed to obtain the solder joint contour. Connected regions other than those that meet the reference diameter requirements are removed to obtain the accurate solder joint contour.

[0136] Correspondingly, such as Figure 9 As shown, this embodiment of the invention also provides a BGA solder joint contour extraction device, which can be used in electronic devices. The BGA solder joint contour extraction device 600 includes:

[0137] The preprocessing module 601 is used to preprocess the original image to obtain a preprocessed image, wherein the preprocessed image includes a plurality of initial connected regions containing the solder joint contours.

[0138] Background foreign object removal module 602 is used to remove background foreign object adhesion in the preprocessed image according to the solder joint characteristics, so as to separate the initial connected region and obtain the secondary processed image;

[0139] The reference diameter calculation module 603 is used to calculate the fitted diameter of each connected region in the secondary processed image, and obtain the reference diameter of the weld point through the fitted diameter;

[0140] Foreign object interference elimination module 604 is used to eliminate foreign object interference in the secondary processed image according to the reference diameter to obtain a tertiary processed image;

[0141] The removal module 605 is used to cyclically update the reference diameter in the three-level processed image, remove the connected components other than the preset diameter deviation amplitude, and obtain the weld point contour.

[0142] In this embodiment of the invention, by preprocessing the original image, random noise interference and threshold segmentation can be eliminated to obtain a preprocessed image of an initial connected region including multiple solder joint contours. Then, based on the solder joint features, background foreign object adhesion in the preprocessed image is eliminated to separate the initial connected region and obtain a secondary processed image, thereby eliminating background foreign object adhesion. The fitted diameter of each connected region in the secondary processed image is calculated, and the reference diameter of the solder joint is obtained through the fitted diameter, without requiring the prior actual solder joint diameter. Based on the reference diameter, foreign object interference in the secondary processed image is eliminated to obtain a tertiary processed image. The reference diameter in the tertiary processed image is cyclically updated, and connected regions outside the preset diameter deviation amplitude are removed to obtain the solder joint contour. Connected regions outside the reference diameter requirement are removed to obtain the accurate solder joint contour.

[0143] In other embodiments, the preprocessing module 601 is further configured to:

[0144] The original image is subjected to noise reduction processing to obtain a first intermediate image;

[0145] The first intermediate image is subjected to threshold segmentation to obtain the preprocessed image, which includes multiple initial connected regions containing solder joint contours.

[0146] In other embodiments, the preprocessing module 601 is further configured to:

[0147] Calculate the grayscale values ​​of all pixels in the first intermediate image;

[0148] The grayscale values ​​of all pixels in the first intermediate image are divided into foreground pixel values ​​and background pixel values ​​to determine the segmentation threshold;

[0149] Based on the segmentation threshold, the first intermediate image is segmented to obtain the preprocessed image.

[0150] In some embodiments, the preprocessing module 601 is further configured to:

[0151] The grayscale value with the largest variance between the foreground pixel value and the background pixel value is used as the segmentation threshold.

[0152] In some embodiments, the background foreign object removal module 602 is further configured to:

[0153] In the first intermediate image, extract the region of interest corresponding to the first intermediate connected region other than the solder joint features, where the first intermediate connected region is one or more of the plurality of initial connected regions;

[0154] Based on an adaptive threshold, threshold segmentation is performed on the region of interest in the preprocessed image to separate the background foreign objects adhering to the preprocessed image.

[0155] In some embodiments, the background foreign object removal module 602 is further configured to:

[0156] If the pixel value at the coordinates of the region of interest is greater than the adaptive threshold, then the pixel value at the coordinates of the region of interest that is greater than the adaptive threshold is set to 0, so as to eliminate the background foreign object adhesion in the preprocessed image and obtain the second intermediate image.

[0157] If the pixel value at the coordinates in the region of interest is less than the adaptive threshold, then the pixel value at the coordinates in the region of interest that is less than the adaptive threshold is set to a preset constant to obtain a third intermediate image, wherein the third intermediate image contains a plurality of second intermediate connected regions that conform to the solder joint features.

[0158] The second intermediate image and the third intermediate image are merged to obtain the secondary processed image;

[0159] The adaptive threshold is determined based on the mean value of the neighborhood centered at the coordinates in the region of interest, with a radius equal to the length of the shorter side of the region of interest.

[0160] In some embodiments, the reference diameter calculation module 603 is further configured to:

[0161] Calculate the fitted diameter of all connected components in the secondary processed image;

[0162] Calculate the mean of the fitted diameters to obtain the reference diameter of the weld joint.

[0163] In some embodiments, the foreign object interference cancellation module 604 is further configured to:

[0164] Based on the reference diameter, morphological opening operations are performed on the connected components in the secondary processed image to eliminate foreign object interference in the secondary processed image and obtain the tertiary processed image.

[0165] In some embodiments, the removal module 605 is further configured to:

[0166] The fitted diameter of all connected components in the three-level processed image is calculated iteratively to update the reference diameter in the three-level processed image as the final reference diameter;

[0167] The preset diameter deviation amplitude is determined based on the final reference diameter, the lower limit of diameter deviation, and the upper limit of diameter deviation.

[0168] Remove the connected regions that do not conform to the preset diameter deviation range to obtain the weld point profile.

[0169] In some embodiments, the device 600 further includes a combination module 606 for:

[0170] Combine the original image and the solder joint outline.

[0171] It should be noted that the above-described apparatus can execute the method provided in the embodiments of this application, and has the corresponding functional modules and beneficial effects for executing the method. Technical details not described in detail in the apparatus embodiments can be found in the method provided in the embodiments of this application.

[0172] Figure 10 A schematic diagram of the hardware structure of the controller in one embodiment of an electronic device, such as... Figure 10 As shown, the controller 13 includes:

[0173] One or more processors 131 and memory 132. Figure 10 The example uses a processor 131 and a memory 132.

[0174] Processor 131 and memory 132 can be connected via a bus or other means. Figure 10 Taking the example of a connection between China and Israel via a bus.

[0175] Memory 132, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the BGA solder joint contour extraction method in the embodiments of this application (e.g., attached...). Figure 9 The preprocessing module 601, background foreign object removal module 602, reference diameter calculation module 603, foreign object interference removal module 604, removal module 605, and combination module 606 are shown. The processor 131 executes various functional applications and data processing of the controller by running non-volatile software programs, instructions, and modules stored in the memory 132, thereby realizing the BGA solder joint contour extraction method of the above method embodiment.

[0176] Memory 132 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the BGA solder joint contour extraction device, etc. Furthermore, memory 132 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, memory 132 may optionally include memory remotely located relative to processor 131, and these remote memories can be connected to the electronic device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0177] The one or more modules are stored in the memory 132. When executed by the one or more processors 131, they perform the BGA solder joint contour extraction method in any of the above method embodiments, for example, performing the above-described... Figure 1 Method steps S101 to S105; implementation Figure 9 The functions of modules 601-606 in the document.

[0178] The above-described product can perform the methods provided in the embodiments of this application, and has the corresponding functional modules and beneficial effects for performing the methods. Technical details not described in detail in this embodiment can be found in the methods provided in the embodiments of this application.

[0179] This application provides a non-volatile computer-readable storage medium storing computer-executable instructions that are executed by one or more processors, for example... Figure 10 One of the processors 131 can enable the one or more processors to execute the BGA solder joint contour extraction method in any of the above method embodiments, for example, to perform the above-described... Figure 1 Method steps S101 to S105; implementation Figure 9 The functions of modules 601-606 in the document.

[0180] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0181] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software and a general-purpose hardware platform, or of course, using hardware. Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0182] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; under the concept of the present invention, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the present invention as described above, which are not provided in detail for the sake of brevity; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for extracting BGA solder joint contours, characterized in that, The method includes: The original image is denoised to obtain a first intermediate image. The first intermediate image is then thresholded to obtain a preprocessed image. The preprocessed image includes multiple initial connected regions containing solder joint contours. Based on the solder joint characteristics, a connected region conforming to the solder joint characteristics is determined in the initial connected region; In the first intermediate image, the region of interest corresponding to the first intermediate connected component is extracted. The first intermediate connected component is the connected component other than the connected component that conforms to the solder joint feature in the initial connected component. Threshold segmentation is performed on the region of interest to obtain the second intermediate image. The second intermediate image and the third intermediate image are merged to obtain the second-level processed image. The third intermediate image includes the connected component that conforms to the solder joint feature. Calculate the fitted diameter of each connected region in the secondary processed image, and obtain the reference diameter of the solder joint through the fitted diameter; The foreign object interference in the secondary processed image is eliminated according to the reference diameter to obtain the tertiary processed image; The reference diameter in the three-level processed image is updated cyclically, and the connected components other than the preset diameter deviation amplitude are removed to obtain the weld point contour.

2. The method according to claim 1, characterized in that, The step of thresholding the first intermediate image to obtain the preprocessed image includes: Calculate the grayscale values ​​of all pixels in the first intermediate image; The grayscale values ​​of all pixels in the first intermediate image are divided into foreground pixel values ​​and background pixel values ​​to determine the segmentation threshold; Based on the segmentation threshold, the first intermediate image is segmented to obtain the preprocessed image.

3. The method according to claim 2, characterized in that, The step of dividing the grayscale values ​​of all pixels in the first intermediate image into foreground pixel values ​​and background pixel values ​​to determine the segmentation threshold includes: The grayscale value with the largest variance between the foreground pixel value and the background pixel value is used as the segmentation threshold.

4. The method according to claim 1, characterized in that, The first intermediate connected component is one or more of the plurality of initial connected components, and the threshold segmentation process on the region of interest includes: Based on an adaptive threshold, the region of interest is segmented using a threshold.

5. The method according to claim 4, characterized in that, The threshold segmentation process for the region of interest based on an adaptive threshold includes: If the pixel value at the coordinates in the region of interest is greater than the adaptive threshold, then the pixel value at the coordinates in the region of interest that is greater than the adaptive threshold is set to 0; If the pixel value at the coordinates in the region of interest is less than the adaptive threshold, then the pixel value at the coordinates in the region of interest that is less than the adaptive threshold is set to a preset constant. The adaptive threshold is determined based on the mean value of the neighborhood centered at the coordinates in the region of interest, with a radius equal to the length of the shorter side of the region of interest.

6. The method according to claim 1, characterized in that, The step of calculating the fitted diameter of each connected component in the secondary processed image and obtaining the reference diameter of the solder joint using the fitted diameter includes: Calculate the fitted diameter of all connected components in the secondary processed image; Calculate the mean of the fitted diameters to obtain the reference diameter of the weld joint.

7. The method according to claim 1, characterized in that, The step of eliminating foreign object interference in the secondary processed image based on the reference diameter to obtain the tertiary processed image includes: Based on the reference diameter, morphological opening operations are performed on the connected components in the secondary processed image to eliminate foreign object interference in the secondary processed image and obtain the tertiary processed image.

8. The method according to any one of claims 1 to 7, characterized in that, The process of iteratively updating the reference diameter in the three-level processed image, removing connected components outside the preset diameter deviation amplitude, and obtaining the solder joint contour includes: The fitted diameter of all connected components in the three-level processed image is calculated iteratively to update the reference diameter in the three-level processed image as the final reference diameter; The preset diameter deviation amplitude is determined based on the final reference diameter, the lower limit of diameter deviation, and the upper limit of diameter deviation. Remove the connected regions that do not conform to the preset diameter deviation range to obtain the weld point profile.

9. The method according to claim 1, characterized in that, After obtaining the solder joint contour, the method further includes: Combine the original image and the solder joint outline.

10. An electronic device, characterized in that, The electronic device includes: At least one processor, and A memory communicatively connected to the processor, the memory storing instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1-9.

Images