A PCBA board surface defect detection method based on computer vision

Abstract

The present invention relates to the field of image data processing, and more particularly to a computer vision-based method for detecting surface defects in PCBA boards. The method comprises: determining a solder joint area from a grayscale image of a PCBA board's surface; determining an initial solder joint edge profile of the solder joint area; determining a gradient environment abnormality level of the current edge pixel point based on the gradient strength and gradient direction of the current edge pixel point in multiple connected directions in the initial solder joint edge profile; determining a loss level of the current edge pixel point based on the grayscale values of each pixel point in a neighborhood window of the current edge pixel point; correcting the initial solder joint edge profile using the gradient environment abnormality level and loss level; and comparing the final solder joint area edge profile with a standard solder joint edge profile to identify whether the solder joint area has a cold solder joint defect. The method improves the accuracy of identifying whether a solder joint area has a cold solder joint defect and the accuracy of judging the quality of the PCBA board.

Description

Technical Field

[0001] The present invention relates to the field of image data processing, and in particular to a method for detecting surface defects of PCBA boards based on computer vision. Background Art

[0002] In the electronics manufacturing industry, printed circuit board assemblies (PCBAs) serve as core components, carrying precision electronic components and achieving electrical connections and mechanical support through welding processes. Their quality directly impacts the overall performance and reliability of electronic products. During the soldering process of PCBA boards, solder joint formation is a key step in determining PCBA board quality. Ideal solder joints should exhibit a good metal bond, ensuring electrical continuity and mechanical stability between the components and the circuit board. However, solder joint defects can occur during the PCBA soldering process due to a variety of factors. Cold solder joints are one of the most common issues. These defects occur when the solder joints fail to form complete metal contact with the pads or component pins, affecting the conductivity of the circuit and the reliability of the electronic product. To ensure the quality of PCBA boards, solder joints must be inspected during the soldering process.

[0003] In some scenarios, the solder joint area of a PCBA board, due to its rough surface and uneven pixel grayscale distribution, is prone to noise points, resulting in blurred edges in the solder joint area. Furthermore, using a Gaussian filter to smooth the grayscale image of the PCBA board cannot completely remove these noise points, resulting in misjudgment when filtering pixels at the edge of the solder joint area. Consequently, it is impossible to accurately determine the edge contour of the solder joint area, affecting the accuracy of identifying cold solder joints and the assessment of PCBA board quality. Summary of the Invention

[0004] In order to solve the technical problem of low recognition accuracy of cold solder joint areas, the purpose of the present invention is to provide a PCBA board surface defect detection method based on computer vision. The technical solution adopted is as follows:

[0005] An embodiment of the present application provides a computer vision-based PCBA board surface defect detection method, comprising: determining a solder joint area from a surface grayscale image of the PCBA board; determining an initial solder joint edge profile of the solder joint area, the initial solder joint edge profile comprising a plurality of edge pixels; determining a gradient environment anomaly degree of the current edge pixel in the initial solder joint edge profile based on a gradient strength and a gradient direction of the current edge pixel in a plurality of connected directions, the gradient environment anomaly degree indicating the likelihood that the current edge pixel belongs to a true cold solder joint edge, the connected directions being directions of a plurality of adjacent pixels adjacent to the current edge pixel; determining a loss degree of the current edge pixel based on the grayscale values of each pixel in a neighborhood window of the current edge pixel, the loss degree indicating the degree to which the current edge pixel approaches a true solder joint edge; correcting the initial solder joint edge profile using the gradient environment anomaly degree and the loss degree to obtain a final solder joint area edge profile of the solder joint area; and comparing the final solder joint area edge profile with a standard solder joint edge profile to identify whether the solder joint area has a cold solder joint defect.

[0006] Optionally, determining the degree of abnormality of the gradient environment of the current edge pixel point based on the gradient strength and gradient direction of the current edge pixel point in the initial weld edge contour in multiple connected directions includes: determining the gradient strength and gradient direction of the current edge pixel point in multiple connected directions; determining the gradient environment factor of the current edge pixel point based on the gradient strength and the gradient direction, the gradient environment factor characterizing the environmental quality of the gradient environment of the current edge pixel point; determining the degree of abnormality of the gradient environment of the current edge pixel point based on the gradient environment factor of the current edge pixel point and the gradient environment factors of other pixels in the weld area.

[0007] Optionally, determining the gradient environment factor of the current edge pixel point based on the gradient strength and the gradient direction includes: determining the average gradient strength of the current edge pixel point based on the gradient strengths of the current edge pixel point in multiple connected directions, and determining the average gradient direction of the current edge pixel point based on the gradient directions of the current edge pixel point in multiple connected directions; calculating a first product of the average gradient strength and the cosine value of the angle at which the average gradient direction is located, and a second product of the gradient strength of the current edge pixel point in each of the connected directions and the cosine value of the angle at which the gradient direction is located; and using the first product and the second product to determine the gradient environment factor of the current edge pixel point.

[0008] Optionally, determining the degree of gradient environment abnormality of the current edge pixel point based on the gradient environment factor of the current edge pixel point and the gradient environment factors of other pixel points in the weld point area includes: determining the average gradient environment factor of the gradient environment factors of other pixel points in the weld point area; and determining the absolute value of the difference between the gradient environment factor of the current edge pixel point and the average gradient environment factor as the degree of gradient environment abnormality.

[0009] Optionally, determining the gradient strength and gradient direction of the current edge pixel point in multiple connected directions includes: using a Sobel operator to determine a first gradient value in the horizontal direction and a second gradient value in the vertical direction of the current edge pixel point in each connected direction; and determining the gradient strength and gradient direction of the current edge pixel point based on the first gradient value and the second gradient value.

[0010] Optionally, determining the degree of loss of the current edge pixel point based on the grayscale value of each pixel point in the neighborhood window of the current edge pixel point includes: determining the neighborhood window of the current edge pixel point with the current edge pixel point as the center, the neighborhood window being divided into a first part and a second part adjacent to each other by the initial weld edge contour; determining the degree of loss of the current edge pixel point based on the first grayscale value of each pixel point in the first part and the second grayscale value of each pixel point in the second part.

[0011] Optionally, determining the degree of loss of the current edge pixel point based on the first grayscale value of each pixel point in the first part and the second grayscale value of each pixel point in the second part includes: respectively calculating the square value of the difference between each pixel point in the first part and each pixel point in the second part; superimposing each of the square values and comparing them with the number of all pixel points in the neighborhood window to obtain the degree of grayscale value difference; and using a normalized inverse function to operate the degree of grayscale value difference to obtain the degree of loss.

[0012] Optionally, the initial weld edge contour is corrected using the gradient environment abnormality degree and the loss degree to obtain the final weld area edge contour of the weld area, including: determining the edge curve correction parameter of the current edge pixel point using the gradient environment abnormality degree and the loss degree; selecting the edge pixel point whose edge curve correction parameter is greater than a threshold value from all the pixel points in the weld area as the final edge pixel point, and marking the edge pixel point whose edge curve correction parameter is not greater than the threshold value in the initial weld edge contour of the weld area as a noise point; and connecting each of the final edge pixel points to obtain the final weld area edge contour.

[0013] Optionally, connecting each of the final edge pixel points to obtain the edge contour of the final weld area includes: obtaining adjacent edge curve correction parameters of each adjacent pixel point corresponding to the final edge pixel point in each of the connection directions; determining the maximum value of the adjacent edge curve correction parameters as the edge connection direction of the final edge pixel point; and connecting each of the final edge pixel points according to the edge connection direction.

[0014] Optionally, determining the solder joint area from the surface grayscale image of the PCBA board includes: calculating a segmentation threshold of the surface grayscale image of the PCBA board using the Otsu threshold algorithm; segmenting the surface grayscale image of the PCBA board based on the segmentation threshold, wherein pixel points exceeding the segmentation threshold are marked as pixel points of the solder joint area, and pixel points not exceeding the segmentation threshold are marked as pixel points of the ordinary area.

[0015] The present invention has the following beneficial effects: first, a solder joint area is determined from a surface grayscale image of a PCBA board; then, an initial solder joint edge profile of the solder joint area is determined, wherein the initial solder joint edge profile includes multiple edge pixels; secondly, a gradient environment abnormality degree of the current edge pixel is determined according to the gradient strength and gradient direction of the current edge pixel in multiple connected directions in the initial solder joint edge profile, wherein the gradient environment abnormality degree indicates the possibility that the current edge pixel belongs to the edge of a real virtual solder joint, wherein the connected direction is the direction of multiple adjacent pixels adjacent to the current edge pixel; then, a loss degree of the current edge pixel is determined according to the grayscale value of each pixel in a neighborhood window of the current edge pixel, wherein the loss degree indicates the degree to which the current edge pixel approaches the edge of the real virtual solder joint; then, the initial solder joint edge profile is corrected using the gradient environment factor and the loss degree to obtain a final solder joint area edge profile of the solder joint area; finally, the final solder joint area edge profile is compared with the standard solder joint edge profile to identify whether the solder joint area has a virtual solder joint defect.

[0016] Thus, the embodiment of the present application combines the gradient environment of all edge pixels on the initial solder joint edge contour with the degree of loss of the edge pixels, corrects the initial solder joint edge contour, accurately screens the edge pixels at the edge of the solder joint area, and thus accurately obtains the final solder joint area edge contour of the solder joint area. Ultimately, based on the final solder joint area edge contour and the standard solder joint edge contour, it accurately identifies whether the solder joint area has a cold solder joint defect. Therefore, the accuracy of identifying whether the solder joint area has a cold solder joint defect and the accuracy of judging the quality of the PCBA board are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 A flowchart of a PCBA board surface defect detection method based on computer vision provided by one embodiment of the present invention;

[0019] Figure 2 A schematic diagram of a surface image of a PCBA board provided by one embodiment of the present invention;

[0020] Figure 3 A schematic diagram of a surface binary image of a PCBA board provided by one embodiment of the present invention;

[0021] Figure 4 A schematic diagram of the structure of a PCBA board surface defect detection system based on computer vision provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0022] To further illustrate the technical means and effectiveness of the present invention in achieving its intended objectives, the following, in conjunction with the accompanying drawings and preferred embodiments, describes in detail the specific implementation, structure, features, and effectiveness of a computer vision-based PCBA surface defect detection method proposed in accordance with the present invention. In the following description, references to different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics of one or more embodiments may be combined in any suitable manner.

[0023] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0024] The following describes in detail a method for detecting PCBA board surface defects based on computer vision provided by the present invention with reference to the accompanying drawings.

[0025] Example 1:

[0026] See also Figure 1 , which shows a flow chart of a PCBA board surface defect detection method based on computer vision provided by one embodiment of the present invention, including:

[0027] Step S101: determining solder joint areas from a surface grayscale image of a PCBA board.

[0028] Specifically, the surface grayscale image of the PCBA board is obtained after preprocessing the acquired surface image of the PCBA board. When acquiring the surface image of the PCBA board, it is necessary to ensure sufficient light and use appropriate lighting equipment to provide a uniform light source. A high-resolution camera is selected and fixed above the soldered PCBA board to capture the top view image of the PCBA board and obtain a clear surface image of the PCBA board, such as Figure 2 The figure shows a surface image of a PCBA board obtained in an embodiment of the present application, which includes a solder joint area 101 and a cold solder joint defect 102. The collected surface image of the PCBA board is then preprocessed to obtain a grayscale image of the PCBA board surface, including operations such as denoising and grayscaling, to facilitate subsequent extraction of bubble information around the solder joints.

[0029] Furthermore, in the surface grayscale image of the PCBA board, the grayscale values of the pixels in the normal area are generally low, and the grayscale values of the pixels in the solder joint area are generally relatively high. Therefore, the surface grayscale image of the PCBA board can be segmented based on the threshold segmentation method to determine the solder joint area. As an optional embodiment of the present application, determining the solder joint area from the surface grayscale image of the PCBA board includes: calculating the segmentation threshold of the surface grayscale image of the PCBA board using the Otsu threshold algorithm; segmenting the surface grayscale image of the PCBA board based on the segmentation threshold, wherein the pixel points exceeding the segmentation threshold are marked as pixel points of the solder joint area, and the pixel points that do not exceed the segmentation threshold are marked as pixel points of the normal area.

[0030] Specifically, the segmentation threshold is calculated using the Otsu threshold algorithm as follows: the grayscale values of all pixels in the surface grayscale image of the PCBA board and the probability of the grayscale values appearing are obtained, and then the average grayscale value of the surface grayscale image of the PCBA board is obtained, and the initial threshold range is set to the entire grayscale value range. For each possible segmentation threshold, the surface grayscale image of the PCBA board is divided into two parts: foreground and background. The mean and variance of the grayscale values of the background and foreground are calculated respectively, and then the overall inter-class variance of the surface grayscale image of the PCBA board is calculated. All segmentation thresholds are traversed, and the segmentation threshold that maximizes the inter-class variance is selected as the final segmentation threshold. It is worth noting that the use of the Otsu threshold algorithm in the prior art to calculate the segmentation threshold also belongs to the technical content of the embodiments of the present application.

[0031] After calculating the segmentation threshold using the Otsu threshold algorithm, the surface grayscale image of the PCBA board is segmented using the segmentation threshold. Pixels exceeding the segmentation threshold are marked as foreground (solder joint area), and pixels not exceeding the segmentation threshold are marked as background (ordinary area). Figure 3 The figure shows the binary surface image of the PCBA board after segmenting the surface grayscale image of the PCBA board by the segmentation threshold, where: Figure 3The part marked with 301 is the solder joint area. Figure 3 The part marked with 302 is the general area.

[0032] Step S102 : determining an initial solder joint edge contour of the solder joint area, where the initial solder joint edge contour includes a plurality of edge pixels.

[0033] Specifically, the Canny edge detection algorithm can be used to determine the initial solder joint edge contour of the solder joint area, thereby obtaining all edge pixels on the initial solder joint edge contour. In actual operation, due to the presence of some scattered high-grayscale pixels inside or around the solder joint area, the Canny edge detection algorithm may mark some pixels inside or around the solder joint area as edge pixels when performing double threshold processing. Therefore, the multiple edge pixels in the initial solder joint edge contour may include some pixels inside or around the solder joint area, resulting in the initial solder joint edge contour not being representative of the actual edge contour of the solder joint area.

[0034] Step S103 , determining the degree of abnormality of the gradient environment of the current edge pixel point in the initial weld spot edge contour according to the gradient strength and gradient direction of the current edge pixel point in multiple connected directions.

[0035] The abnormal degree of the gradient environment represents the possibility that the current edge pixel point belongs to the edge of a real virtual solder joint, and the connection direction is the direction of multiple adjacent pixels adjacent to the current edge pixel point.

[0036] Specifically, the connection direction refers to the direction of multiple adjacent pixel points adjacent to the current edge pixel point. The number of connection directions can be 8. Of course, according to actual conditions, the connection direction can also be other values, which is not limited in this embodiment of the present application.

[0037] Furthermore, the gradient strength refers to the degree of change in pixel brightness or color in the grayscale image of the PCBA surface, while the gradient direction describes the direction of the change in pixel brightness or color. The gradient strength and gradient direction can be calculated using the Sobel operator. The gradient environment anomaly indicates the degree of difference between the gradient environment of the current edge pixel and that of other edge pixels. A greater degree of gradient environment anomaly indicates a greater likelihood that the current edge pixel is the edge of a real solder joint.

[0038] As an optional embodiment of the present application, determining the degree of gradient environment abnormality of the current edge pixel point based on the gradient strength and gradient direction of the current edge pixel point in multiple connected directions in the initial weld edge contour includes: determining the gradient strength and gradient direction of the current edge pixel point in multiple connected directions; determining the gradient environment factor of the current edge pixel point based on the gradient strength and gradient direction, the gradient environment factor characterizing the environmental quality of the gradient environment of the current edge pixel point; determining the degree of gradient environment abnormality of the current edge pixel point based on the gradient environment factor of the current edge pixel point and the gradient environment factors of other pixels in the weld area.

[0039] Specifically, determining the gradient strength and gradient direction of the current edge pixel in multiple connected directions, that is, the gradient strength and gradient direction of the current edge pixel in the direction of its adjacent adjacent pixels, the gradient strength and gradient direction can be calculated using a Sobel operator. As an optional embodiment of the present application, determining the gradient strength and gradient direction of the current edge pixel in multiple connected directions includes: using a Sobel operator to determine a first gradient value in the horizontal direction and a second gradient value in the vertical direction of the current edge pixel in each connected direction; and determining the gradient strength and gradient direction of the current edge pixel based on the first gradient value and the second gradient value.

[0040] Specifically, the gradient strength and direction of the current edge pixel in a connected direction are typically represented by the combination of its gradient component in the horizontal direction x (first gradient value) and its gradient component in the vertical direction y (second gradient value). That is, the gradient strength of the current edge pixel in a connected direction is calculated by taking the root sum of the squares of the first gradient value and the second gradient value; the root sum is the gradient strength. The gradient direction of the current edge pixel in a connected direction can be calculated by using the inverse tangent hyperbolic function to calculate the ratio of the second gradient value to the first gradient value; the resulting angle is the gradient direction.

[0041] Furthermore, the quality of the environment refers to the quality of the gradient environment of the current edge pixel point. The gradient environment factor represents the average value of the gradient intensity and the average value of the gradient direction of the current edge pixel point in its multiple connected directions, and the degree of difference between the gradient intensity and the gradient direction in each connected direction. The greater the difference, the worse the gradient environment of the current edge pixel point.

[0042] As an optional embodiment of the present application, determining the gradient environment factor of the current edge pixel point based on the gradient strength and gradient direction includes: determining the average gradient strength of the current edge pixel point based on the gradient strength of the current edge pixel point in multiple connected directions, and determining the average gradient direction of the current edge pixel point based on the gradient direction of the current edge pixel point in multiple connected directions; calculating a first product of the average gradient strength and the cosine value of the angle at which the average gradient direction is located, and a second product of the gradient strength of the current edge pixel point in each connected direction and the cosine value of the angle at which the gradient direction is located; and using the first product and the second product to determine the gradient environment factor of the current edge pixel point.

[0043] Specifically, the average gradient strength can be obtained by averaging the gradient strengths of the current edge pixel in multiple connected directions, and the average gradient direction can be obtained by averaging the gradient directions of the current edge pixel in multiple connected directions. Furthermore, the embodiment of the present application takes the number of connected directions as 8 as an example, that is, the eight directions under the eight-neighborhood as connected directions, to illustrate the calculation method of the gradient environment factor. The gradient environment factor can be calculated using the following formula:

[0044]

[0045] In the above formula, V i represents the gradient environment factor, i represents the i-th edge pixel point in the solder joint area, j represents the j-th connected direction of the i-th edge pixel point in the solder joint area among the 8 connected directions, It represents the mean value of the gradient strength of the ith edge pixel in the solder joint area in its eight different connected directions (average gradient strength). It represents the angle of the mean value of the gradient direction (average gradient direction) of the ith edge pixel in the solder joint area in its eight different connected directions. It represents the gradient strength of the i-th edge pixel point in the solder joint area in its j-th connected direction. Indicates the angle of the gradient direction of the i-th edge pixel point in the solder joint area in its j-th connectivity direction.

[0046] Further, Represents the product of the mean value of the gradient strength of the i-th edge pixel in the solder joint area in its eight different connected directions and the cosine value of the angle between the mean value of the gradient direction of the i-th edge pixel in the solder joint area in its eight different connected directions. Represents the product of the gradient strength of the i-th edge pixel point in the solder joint area in its j-th connected direction and the cosine value of the angle of the gradient direction of the i-th edge pixel point in the solder joint area in its j-th connected direction. Represents the degree of difference between the product of the mean of the gradient strength of the i-th edge pixel in its eight different connected directions and the cosine of the angle where the mean of the gradient direction is located, and the product of the gradient strength of the i-th edge pixel in its j-th connected direction and the cosine of the angle where the gradient direction is located. The greater the degree of difference, the worse the gradient environment of the i-th edge pixel in the solder joint area.

[0047] Furthermore, the gradient environment factor of each edge pixel point in the weld area is obtained, and then the gradient environment of the current edge pixel point and other edge pixel points are compared by comparing its gradient environment factor with the gradient environment factors of other edge pixel points, thereby obtaining the degree of abnormality of the gradient environment of the current edge pixel point.

[0048] As an optional embodiment of the present application, determining the degree of gradient environment abnormality of the current edge pixel point based on the gradient environment factor of the current edge pixel point and the gradient environment factors of other pixel points in the solder joint area includes: determining the average gradient environment factor of the gradient environment factors of other pixel points in the solder joint area; and determining that the absolute value of the difference between the gradient environment factor of the current edge pixel point and the average gradient environment factor is the degree of gradient environment abnormality.

[0049] Specifically, the gradient environment anomaly parameter can be calculated using the following formula:

[0050]

[0051] In the above formula, E i is the gradient environment anomaly parameter of the i-th edge pixel, V i represents the gradient environment factor of the ith edge pixel, N represents the number of all edge pixels, u represents the uth edge pixel among all edge pixels except the ith edge pixel, V u Represents the gradient environment factor of the u-th edge pixel.

[0052] Further, It represents the mean value of the gradient environment factor of the u-th edge pixel among all edge pixels except the i-th edge pixel in the solder joint area (average gradient environment factor). It represents the difference between the gradient environment factor of the ith edge pixel and the mean of the gradient environment factors of all edge pixels except the ith edge pixel. The greater the difference, the greater the possibility that the edge pixel belongs to the edge of a real weld joint.

[0053] Step S104 , determining the loss degree of the current edge pixel point according to the grayscale value of each pixel point in the neighborhood window of the current edge pixel point.

[0054] The loss degree represents the degree to which the current edge pixel point approaches the edge of the real solder joint.

[0055] Specifically, the grayscale values of pixels in the solder joint area and those in the general area differ significantly, with higher grayscale values in the solder joint area and lower grayscale values in the general area. Therefore, by analyzing the grayscale values of the pixels surrounding the edge pixel in the initial solder joint edge contour, the grayscale value differences of the pixels surrounding the edge pixel can be used to determine the degree of loss at that pixel in the solder joint area, that is, whether that pixel is affected by noise or interference.

[0056] Furthermore, the neighborhood window refers to an N*N window defined with the edge pixel of the initial weld edge contour of the weld area as the center, where N refers to the number of pixels. N can be set based on actual conditions, such as 5, and is not limited to this in the present embodiment. The degree of loss of the current edge pixel can be determined by calculating the difference in grayscale value between the current edge pixel and all pixels within the neighborhood window.

[0057] As an optional embodiment of the present application, determining the degree of loss of the current edge pixel point based on the grayscale value of each pixel point in the neighborhood window of the current edge pixel point includes: determining the neighborhood window of the current edge pixel point with the current edge pixel point as the center, and the neighborhood window is divided into a first part and a second part adjacent to each other by the initial weld edge contour; determining the degree of loss of the current edge pixel point based on the first grayscale value of each pixel point in the first part and the second grayscale value of each pixel point in the second part.

[0058] Specifically, after determining the neighborhood window of the current edge pixel, since the current edge pixel is at the center of the neighborhood window, the initial weld edge contour divides the neighborhood window into two adjacent parts. The difference in the grayscale values of each pixel in the two parts can be calculated as the loss degree of the current edge pixel. The loss degree of the current edge pixel represents the loss degree of the initial weld edge contour at the current edge pixel.

[0059] As an optional embodiment of the present application, determining the degree of loss of the current edge pixel point based on the first grayscale value of each pixel point in the first part and the second grayscale value of each pixel point in the second part includes: respectively calculating the square value of the difference between each pixel point in the first part and each pixel point in the second part; superimposing each square value and comparing it with the number of all pixels in the neighborhood window to obtain the degree of grayscale value difference; using a normalized inverse function to operate the degree of grayscale value difference to obtain the degree of loss.

[0060] Specifically, the degree of loss can be calculated using the following formula:

[0061]

[0062] In the above formula, K i represents the loss degree of the i-th edge pixel in the solder joint area, i represents the i-th edge pixel in the solder joint area, n represents the number of all pixels in the neighborhood window of the i-th edge pixel, a represents the pixel set sequence in the first part of the neighborhood window, b represents the pixel set sequence in the second part of the neighborhood window, l represents the l-th pixel in the a and b class set sequence in the neighborhood window, It represents the gray value of the lth pixel in the set of type a pixels in the neighborhood window where the i-th edge pixel in the solder joint area is located. It represents the grayscale value of the lth pixel in the set sequence of type b pixels in the neighborhood window where the i-th edge pixel is located in the solder joint area. exp-[] represents the normalized inverse function, that is, the opposite number of the content in the brackets is used as the power of an exponential function with a natural constant as the base to achieve the purpose of inverse proportional mapping and normalization.

[0063] Further, It represents the difference between the gray value of the lth pixel in the set sequence of type A pixels in the neighborhood window where the i-th edge pixel in the solder joint area is located and the gray value of the lth pixel in the set sequence of type B pixels in the neighborhood window where the i-th edge pixel in the solder joint area is located. It represents the difference in grayscale values between the set sequence of type A pixels in the neighborhood window where the i-th edge pixel point in the solder joint area is located and the set sequence of type B pixels in the neighborhood window where the i-th edge pixel point in the solder joint area is located. The greater the difference, the greater the possibility that the edge pixel point is at the boundary between the solder joint area and the ordinary area, that is, the closer the edge pixel point is to the real solder joint edge, indicating that the loss degree of the initial solder joint edge contour at the i-th edge pixel point is smaller.

[0064] Step S105 , correcting the initial solder joint edge contour using the gradient environment abnormality degree and loss degree to obtain a final solder joint region edge contour of the solder joint region.

[0065] Specifically, after obtaining the degree of gradient environmental anomaly and loss of each edge pixel point, it is possible to further determine whether the edge pixel point is a real edge pixel point or a noise point. Then, the initial solder joint edge contour is corrected to obtain the real solder joint edge of the solder joint area. Among them, for all edge pixel points of the initial solder joint edge contour of the solder joint area on the PCBA board, the degree of gradient environmental anomaly of the edge pixel point and the degree of loss at the edge pixel point can be combined to obtain the possibility parameter of the corresponding edge being a real solder joint edge pixel point. The possibility parameter is recorded as the edge curve correction parameter in the embodiment of the present application.

[0066] As an optional embodiment of the present application, the initial weld edge contour is corrected using the degree of gradient environment anomaly and the degree of loss to obtain the final weld area edge contour of the weld area, including: determining the edge curve correction parameter of the current edge pixel point using the degree of gradient environment anomaly and the degree of loss; selecting the edge pixel point whose edge curve correction parameter is greater than the threshold from all the pixel points in the weld area as the final edge pixel point, and marking the edge pixel point whose edge curve correction parameter is not greater than the threshold in the initial weld edge contour of the weld area as a noise point; and connecting each final edge pixel point to obtain the final weld area edge contour.

[0067] Specifically, the following formula can be used to calculate the edge curve correction parameters using the gradient environment anomaly degree and loss degree:

[0068]

[0069] In the above formula, τ i is the edge curve correction parameter of the i-th edge pixel point in the solder joint area, E i represents the abnormal degree of gradient environment of the i-th edge pixel point in the solder joint area, K i represents the loss degree at the i-th edge pixel point in the solder joint area, th[] represents the hyperbolic tangent function, and e is a natural constant.

[0070] Further, It represents the product of the gradient environment abnormality degree of the i-th edge pixel point in the solder joint area and the exponential function operation result of the loss degree at the i-th edge pixel point in the solder joint area. The larger the value, the greater the τ i The larger the value of , the greater the possibility that the i-th edge pixel point of the initial solder joint edge contour in the solder joint area is the real edge pixel point of the solder joint area.

[0071] Furthermore, all edge pixels in the solder joint area are judged and screened. For all edge pixels of the initial solder joint edge contour in the surface grayscale image of the PCBA board, their τ i The embodiment of the present application sets the threshold value T=0.5 based on experience, and compares the edge pixel points τ of the edge contour of all initial solder joints. i The value is equal to the threshold T. The pixel value greater than the threshold T is marked as the final edge pixel point of the solder joint area. The final edge pixel point can be used as the true edge pixel point of the solder joint area. The pixel value not less than the threshold T is marked as a noise point and is not considered in the subsequent edge connection.

[0072] As an optional embodiment of the present application, connecting each final edge pixel point to obtain the edge contour of the final weld area includes: obtaining adjacent edge curve correction parameters of each adjacent pixel point corresponding to the final edge pixel point in each connection direction; determining the maximum value of the adjacent edge curve correction parameters as the edge connection direction of the final edge pixel point; and connecting each final edge pixel point according to the edge connection direction.

[0073] Specifically, when connecting edge pixels in the area of the initial weld edge contour where noise points exist, the τ of the nearest adjacent pixels in the eight connected directions is obtained for the i-th final edge pixel. i The value of τ i The maximum value of is taken as the edge connection direction. Finally, all the final edge pixels are connected one by one according to the edge connection direction of each final edge pixel point to obtain a more accurate edge contour of the solder joint area, that is, the final solder joint area edge contour.

[0074] Furthermore, in order to further improve the accuracy of the edge contour of the solder joint area, the edge contour of the final solder joint area can be further processed, that is, morphological operations can be used to process the edge contour of the final solder joint area, which helps to fill the broken edge segments in the edge contour of the final solder joint area and eliminate small noise points.

[0075] The morphological operation process is as follows: first, the edge contour of the final weld area is corroded, the structural element is slid on the edge contour of the final weld area, and the pixel value of the edge contour of the final weld area is updated with the minimum value of the area covered by it. For each pixel position of the edge contour of the final weld area covered by the structural element, if all pixel values in the structural element are 1, the pixel value of the corresponding pixel position is updated to 1, otherwise it is updated to 0. Then, the structural element is slid on the edge contour of the final weld area using the expansion operation, and the pixel value of the edge contour of the final weld area is updated with the maximum value of the area covered by it. For each pixel position covered by the structural element, if any pixel value in the structural element is 1, the pixel value of the corresponding pixel position is updated to 1, thereby achieving the purpose of filling broken edge segments and eliminating small noise points.

[0076] Step S106 : comparing the edge contour of the final solder joint area with the edge contour of the standard solder joint to identify whether there is a cold solder joint defect in the solder joint area.

[0077] Specifically, the final solder joint area edge contour of all solder joint areas is obtained by the above method, and the solder joint edge curve features such as the area, perimeter, and edge contour features of the final solder joint area edge contour are determined, and compared with the standard solder joint edge curve features of the pre-defined standard solder joint edge contour. If the similarity between the two is greater than a threshold value, it indicates that there is no cold solder defect in the solder joint area. If the similarity between the two is not greater than the threshold value, it indicates that there is a cold solder defect in the solder joint area. Among them, the threshold value can be determined according to actual conditions, and the embodiments of the present application are not limited here. The standard solder joint edge contour refers to a standard solder joint area that does not have a cold solder defect.

[0078] The embodiment of the present application combines the gradient environment of all edge pixels on the initial solder joint edge contour with the degree of loss of the edge pixels, corrects the initial solder joint edge contour, accurately filters the edge pixels at the edge of the solder joint area, and thus accurately obtains the final solder joint area edge contour of the solder joint area. Finally, based on the final solder joint area edge contour and the standard solder joint edge contour, it accurately identifies whether the solder joint area has a cold solder joint defect. This improves the accuracy of identifying whether the solder joint area has a cold solder joint defect and improves the accuracy of judging the quality of the PCBA board.

[0079] Example 2:

[0080] Corresponding to the PCBA board surface defect detection method based on computer vision provided in the above embodiment, based on the same technical concept, the embodiment of the present application also provides a PCBA board surface defect detection system based on computer vision, which is used to execute the above PCBA board surface defect detection method based on computer vision. Figure 4 A schematic diagram of a PCBA board surface defect detection system based on computer vision is provided to implement various embodiments of the present application. Figure 4 As shown in the figure, at the hardware level, the computer vision-based PCBA surface defect detection system includes a processor and, optionally, an internal bus, a network interface, and memory. The memory may include internal memory, such as high-speed random-access memory (RAM), and may also include non-volatile memory, such as at least one disk drive. Of course, the computer vision-based PCBA surface defect detection system may also include other hardware required for the business.

[0081] The processor, network interface, and memory can be interconnected via an internal bus, which can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. These buses can be classified as address buses, data buses, control buses, and the like. For ease of illustration, the figure uses only one bidirectional arrow, but this does not imply that there is only one bus or only one type of bus.

[0082] The memory is used to store programs. Specifically, the program may include program code, which includes computer operation commands. The memory may include internal memory and non-volatile memory, and provides instructions and data to the processor.

[0083] The processor reads the corresponding computer program from the non-volatile memory into the internal memory and then runs it, forming a device for locating the specified user at the logical level. The processor executes the program stored in the memory and is specifically used to perform: Figure 4 The methods disclosed in the illustrated embodiments implement the functions and beneficial effects of the various methods described in the foregoing method embodiments, which will not be described in detail here.

[0084] It should be noted that the PCBA board surface defect detection system based on computer vision provided in the embodiment of the present application and the PCBA board surface defect detection method based on computer vision provided in the embodiment of the present application are based on the same application concept. Therefore, the specific implementation of this embodiment can refer to the implementation of the aforementioned PCBA board surface defect detection method based on computer vision, and has the same or similar beneficial effects, and the repeated parts will not be repeated.

[0085] It should be noted that the order in which the embodiments of the present invention are described above is for illustrative purposes only and does not necessarily represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0086] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

Claims

1. A PCBA board surface defect detection method based on computer vision, characterized in that: The PCBA board surface defect detection method based on computer vision includes: Determine the solder joint area from the surface grayscale image of the PCBA board; Determine an initial solder joint edge contour of the solder joint area, wherein the initial solder joint edge contour includes a plurality of edge pixel points; Determining a degree of gradient environment abnormality of a current edge pixel point in the initial weld edge contour based on the gradient strength and gradient direction of the current edge pixel point in multiple connected directions, wherein the degree of gradient environment abnormality indicates the likelihood that the current edge pixel point belongs to a real weld edge, and the connected directions are directions of multiple adjacent pixels adjacent to the current edge pixel point; Determining a loss degree of the current edge pixel point based on the grayscale value of each pixel point in a neighborhood window of the current edge pixel point, wherein the loss degree represents a degree of closeness of the current edge pixel point to a true weld edge; Correcting the initial weld spot edge profile using the gradient environment abnormality degree and the loss degree to obtain a final weld spot region edge profile of the weld spot region; Comparing the edge profile of the final solder joint area with the edge profile of a standard solder joint to identify whether there is a cold soldering defect in the solder joint area; Determining the degree of abnormality of the gradient environment of the current edge pixel point in the initial weld edge contour according to the gradient strength and gradient direction of the current edge pixel point in multiple connected directions includes: Determining the gradient strength and gradient direction of the current edge pixel in multiple connected directions; determining a gradient environment factor of the current edge pixel point according to the gradient strength and the gradient direction, wherein the gradient environment factor represents the quality of the gradient environment of the current edge pixel point; Determining the degree of abnormality of the gradient environment of the current edge pixel point based on the gradient environment factor of the current edge pixel point and the gradient environment factors of other pixels in the welding point area; Determining the loss degree of the current edge pixel point according to the grayscale value of each pixel point in the neighborhood window of the current edge pixel point includes: Determining a neighborhood window of the current edge pixel point with the current edge pixel point as the center, wherein the neighborhood window is divided into a first part and a second part adjacent to each other by the initial weld point edge contour; The loss degree of the current edge pixel is determined according to the first grayscale value of each pixel in the first part and the second grayscale value of each pixel in the second part.

2. The PCBA board surface defect detection method based on computer vision according to claim 1, characterized in that: Determining the gradient environment factor of the current edge pixel point according to the gradient strength and the gradient direction includes: Determining an average gradient strength of the current edge pixel point based on the gradient strengths of the current edge pixel point in multiple connected directions, and determining an average gradient direction of the current edge pixel point based on the gradient directions of the current edge pixel point in multiple connected directions; Calculating a first product of the average gradient strength and the cosine value of the angle at which the average gradient direction is located, and a second product of the gradient strength of the current edge pixel in each of the connected directions and the cosine value of the angle at which the gradient direction is located; The gradient environment factor of the current edge pixel is determined using the first product and the second product.

3. The PCBA board surface defect detection method based on computer vision according to claim 1, characterized in that: Determining the abnormality degree of the gradient environment of the current edge pixel point based on the gradient environment factor of the current edge pixel point and the gradient environment factors of other pixels in the welding point area includes: Determine an average gradient environment factor of the gradient environment factors of other pixel points in the solder joint area; The absolute value of the difference between the gradient environment factor of the current edge pixel point and the average gradient environment factor is determined as the degree of gradient environment abnormality.

4. The PCBA board surface defect detection method based on computer vision according to claim 1, characterized in that: Determining the gradient strength and gradient direction of the current edge pixel in multiple connected directions includes: Using the Sobel operator to determine the first gradient value in the horizontal direction and the second gradient value in the vertical direction of the current edge pixel in each connected direction; The gradient intensity and gradient direction of the current edge pixel are determined according to the first gradient value and the second gradient value.

5. The PCBA board surface defect detection method based on computer vision according to claim 1, characterized in that: The determining the loss degree of the current edge pixel points according to the first grayscale value of each pixel point in the first part and the second grayscale value of each pixel point in the second part includes: Calculating the square of the difference between each pixel in the first part and each pixel in the second part; The square values are superimposed and compared with the number of all pixels in the neighborhood window to obtain the grayscale value difference; The grayscale value difference degree is calculated using a normalized inverse function to obtain the loss degree.

6. The PCBA board surface defect detection method based on computer vision according to claim 1, characterized in that: The correcting the initial weld edge profile by using the gradient environment abnormality degree and the loss degree to obtain the final weld area edge profile of the weld area includes: Determining edge curve correction parameters of the current edge pixel point using the abnormality degree of the gradient environment and the loss degree; Selecting edge pixel points whose edge curve correction parameters are greater than a threshold value from all pixel points in the solder joint area as final edge pixel points, and marking edge pixel points whose edge curve correction parameters are not greater than the threshold value in the initial solder joint edge contour of the solder joint area as noise points; The final edge pixel points are connected to obtain the edge contour of the final welding point area.

7. The PCBA board surface defect detection method based on computer vision according to claim 6, characterized in that: Connecting the final edge pixel points to obtain the final edge contour of the welding point area includes: Obtaining adjacent edge curve correction parameters of adjacent pixel points corresponding to the final edge pixel point in each of the connected directions; Determining the maximum value among the adjacent edge curve correction parameters as the edge connection direction of the final edge pixel point; The final edge pixel points are connected according to the edge connection direction.

8. The PCBA board surface defect detection method based on computer vision according to claim 1, characterized in that: Determining the solder joint area from the surface grayscale image of the PCBA board includes: Calculate the segmentation threshold of the surface grayscale image of the PCBA board using the Otsu threshold algorithm; The surface grayscale image of the PCBA board is segmented based on the segmentation threshold, wherein the pixel points exceeding the segmentation threshold are marked as pixel points in the solder joint area, and the pixel points within the segmentation threshold are marked as pixel points in the general area.

Images