Method and apparatus for detecting defects in solder joints

Abstract

This disclosure relates to a method and apparatus for detecting solder joint defects. The method includes: acquiring a first grayscale image of the solder joint under illumination by a first light source and a second grayscale image under illumination by a second light source; determining first position information of marker points in the first grayscale image, and determining second position information of a conductor based on the first position information; determining third position information of the solder joint based on the second position information and the grayscale values ​​of pixels in the second grayscale image; and determining a defect detection result based on the third position information and the grayscale values ​​of pixels in the first grayscale image. The solder joint defect detection method according to embodiments of this disclosure can capture two images under illumination by light sources of two different heights to obtain their grayscale images, reducing halos caused by energy diffusion, and can use more prominent marker points to determine the position of the conductor. It can also utilize the characteristics of the two grayscale images to determine solder joint defects, thereby improving detection accuracy and reducing labor costs.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a method and apparatus for detecting solder joint defects. Background Technology

[0002] Surface defects in a product can negatively impact its appearance, comfort, and performance. Therefore, manufacturers can inspect for surface defects to detect and control them promptly.

[0003] In related technologies, detection can be performed using machine vision, which primarily studies how to use computers to simulate human visual functions. Images are acquired through cameras, then converted into digital image signals and sent to a computer. The computer obtains the necessary information and makes correct calculations and judgments. Through digital image processing and recognition algorithms, three-dimensional scenes and objects in the objective world are identified from the perspectives of shape and motion, and judgments and controls are made based on the recognition results.

[0004] Product defect detection is a crucial component of machine vision inspection, and its accuracy directly impacts the final product quality. Since manual inspection methods are no longer sufficient to meet the needs of modern production processes, machine vision inspection has effectively overcome this limitation. The widespread application of surface defect detection systems has promoted high-quality production in factories and the development of intelligent automation in the manufacturing industry.

[0005] The application of surface defect detection systems has improved the accuracy and efficiency of inspections. Currently, factory labor costs are increasing year by year. The small size and unique shape of the copper solder joints make manual visual inspection difficult under normal conditions, requiring high-magnification cameras, which is slow and demands a high level of experience from quality inspectors. Furthermore, due to the unique shape of the solder joints, ordinary lighting produces poor imaging results, and the area around the solder joints exhibits significant energy diffusion and halos, making it difficult to segment the solder joints using existing detection methods, and even more difficult to identify defects. Summary of the Invention

[0006] This disclosure presents a method and apparatus for detecting weld joint defects.

[0007] According to one aspect of this disclosure, a method for detecting solder joint defects is provided, comprising: acquiring a first grayscale image of at least one solder joint under illumination by a first light source, and a second grayscale image under illumination by a second light source, wherein the height of the first light source is higher than the height of the second light source; determining first position information of a marker point in the first grayscale image, and determining second position information of a conductor corresponding to the at least one solder joint in the region of the first grayscale image based on the first position information; determining third position information of the at least one solder joint in the second grayscale image based on the second position information and the grayscale values ​​of a plurality of pixels in the second grayscale image; and determining a defect detection result of the solder joint based on the third position information and the grayscale values ​​of a plurality of pixels in the first grayscale image.

[0008] In one possible implementation, determining the second position information of the conductor corresponding to the at least one solder joint in the region of the first grayscale image based on the first position information includes: obtaining the relative positional relationship between the marker point and the conductor; and performing an affine transformation on the first position information based on the relative positional relationship to obtain the second position information.

[0009] In one possible implementation, determining the third location information of the at least one solder joint in the second grayscale image based on the second location information and the grayscale values ​​of multiple pixels in the second grayscale image includes: determining a fourth location information of the region where the conductor is located in the second grayscale image based on the second location information; determining a first mean and a first variance of the grayscale values ​​of the pixels in the region where the conductor is located in the second grayscale image based on the fourth location information; and determining the third location information of the at least one solder joint in the region where the conductor is located in the second grayscale image based on the first mean and the first variance.

[0010] In one possible implementation, the defect detection results include process defect detection results and appearance defect detection results. Determining the defect detection result of the solder joint based on the third location information and the grayscale values ​​of multiple pixels in the first grayscale image includes: determining the process defect detection result of the solder joint based on the third location information; determining the fifth location information of the region where the solder joint is located within the first grayscale image based on the third location information; determining the second mean and second variance of the grayscale values ​​of the pixels within the region where the solder joint is located in the first grayscale image based on the fifth location information; and determining the appearance defect detection result of the solder joint based on the second mean and the second variance.

[0011] In one possible implementation, determining the process defect detection result of the solder joint based on the third location information includes: determining the size information of the area where the solder joint is located in the second grayscale image based on the third location information; and determining the process defect detection result of the solder joint based on the size information.

[0012] In one possible implementation, determining the appearance defect detection result of the solder joint based on the second mean and the second variance includes: determining a predicted defect region within the area where the solder joint is located based on the second mean, the second variance, and a first preset parameter; determining the area where the appearance defect is located within the predicted defect region based on the second mean, the second variance, and the second preset parameter; and determining the appearance defect detection result based on the area where the appearance defect is located and the grayscale values ​​of the pixels within the area where the solder joint is located in the first grayscale image.

[0013] In one possible implementation, determining a predicted defect region within the area where the solder joint is located, based on the second mean, the second variance, and the first preset parameter, includes: obtaining a first defect threshold based on the second mean, the second variance, and the first preset parameter; and determining the area where pixels with grayscale values ​​less than the first defect threshold are located within the area where the solder joint is located as the predicted defect region.

[0014] In one possible implementation, determining the region where the appearance defect is located within the predicted defect region based on the second mean, the second variance, and the second preset parameter includes: obtaining a second defect threshold based on the second mean, the second variance, and the second preset parameter; and determining the region where the pixels with grayscale values ​​less than the second defect threshold are located within the predicted defect region as the region where the appearance defect is located.

[0015] In one possible implementation, acquiring a first grayscale image of at least one solder joint under illumination by a first light source and a second grayscale image under illumination by a second light source includes: acquiring a first image of at least one solder joint under illumination by a first light source and a second image under illumination by a second light source; and converting the first image and the second image into the first grayscale image and the second grayscale image, respectively.

[0016] According to one aspect of this disclosure, a solder joint defect detection device is provided, comprising: a first light source, a second light source, a camera, and a processor, wherein the height of the first light source is higher than the height of the second light source; the camera is configured to acquire a first grayscale image of the at least one solder joint when the first light source illuminates the at least one solder joint, and to acquire a second grayscale image of the at least one solder joint when the second light source illuminates the at least one solder joint; the processor is configured to: determine first position information of a marker point in the first grayscale image, and determine second position information of a conductor corresponding to the at least one solder joint in the region of the first grayscale image based on the first position information; determine third position information of the at least one solder joint in the second grayscale image based on the second position information and the grayscale values ​​of a plurality of pixels in the second grayscale image; and determine a defect detection result of the solder joint based on the third position information and the grayscale values ​​of a plurality of pixels in the first grayscale image.

[0017] In one possible implementation, the camera can also be configured with a lens to form a high-magnification area array camera.

[0018] In one possible implementation, the processor is further configured to: determine fourth location information of the region where the conductor is located in the second grayscale image based on the second location information; determine a first mean and a first variance of the grayscale values ​​of the pixels in the region where the conductor is located in the second grayscale image based on the fourth location information; and determine third location information of the at least one solder joint in the region where the conductor is located in the second grayscale image based on the first mean and the first variance.

[0019] In one possible implementation, the defect detection results include process defect detection results and appearance defect detection results. The processor is further configured to: determine the process defect detection result of the solder joint based on the third location information; determine the fifth location information of the region where the solder joint is located in the first grayscale image based on the third location information; determine the second mean and second variance of the grayscale values ​​of the pixels in the region where the solder joint is located in the first grayscale image based on the fifth location information; and determine the appearance defect detection result of the solder joint based on the second mean and the second variance.

[0020] In one possible implementation, the processor is further configured to: determine the size information of the area where the solder joint is located in the second grayscale image based on the third location information; and determine the process defect detection result of the solder joint based on the size information.

[0021] In one possible implementation, the processor is further configured to: determine a predicted defect region within the area where the solder joint is located based on the second mean, the second variance, and the first preset parameter; determine a region containing an appearance defect within the predicted defect region based on the second mean, the second variance, and the second preset parameter; and determine the appearance defect detection result based on the region containing the appearance defect and the grayscale values ​​of pixels within the area where the solder joint is located in the first grayscale image.

[0022] In one possible implementation, the processor is further configured to: obtain a first defect threshold based on the second mean, the second variance, and the first preset parameter; and determine the region where the pixels with grayscale values ​​less than the first defect threshold are located within the area where the solder joint is located as the predicted defect region.

[0023] In one possible implementation, the processor is further configured to: obtain a second defect threshold based on the second mean, the second variance, and the second preset parameter; and determine the region where the pixels with gray values ​​less than the second defect threshold are located within the predicted defect region as the region where the appearance defect is located.

[0024] In one possible implementation, the processor is further configured to: acquire a first image of at least one solder joint under illumination by a first light source and a second image under illumination by a second light source; and convert the first image and the second image into the first grayscale image and the second grayscale image, respectively.

[0025] According to one aspect of this disclosure, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to invoke the instructions stored in the memory to perform the method described above.

[0026] According to one aspect of this disclosure, a computer-readable storage medium is provided that stores computer program instructions thereon, which, when executed by a processor, implement the above-described method.

[0027] The solder joint defect detection method according to the embodiments of the present disclosure can capture two images under illumination by light sources at two different heights and obtain their grayscale images, reduce the halo caused by energy diffusion, and use more prominent marker points to determine the position of the conductor. The characteristics of the two grayscale images can be used to determine the position of the solder joint and the defects of the solder joint in the image respectively, so as to improve the detection accuracy and reduce labor costs.

[0028] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0029] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the specification, serve to illustrate the technical solutions of this disclosure.

[0030] Figure 1 A flowchart is shown for a solder joint defect detection method according to an embodiment of the present disclosure;

[0031] Figure 2 This diagram illustrates an application of the solder joint defect detection method according to an embodiment of the present disclosure.

[0032] Figure 3 A schematic diagram of a solder joint defect detection apparatus according to an embodiment of the present disclosure is shown. Detailed Implementation

[0033] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0034] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0035] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0036] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0037] Figure 1 A flowchart illustrating a solder joint defect detection method according to an embodiment of the present disclosure is shown, such as... Figure 1 As shown, the method includes:

[0038] In step S11, a first grayscale image of at least one solder joint under illumination by a first light source and a second grayscale image under illumination by a second light source are acquired, wherein the height of the first light source is higher than the height of the second light source.

[0039] In step S12, the first location information of the marker point in the first grayscale image is determined, and the second location information of the conductor corresponding to the at least one solder joint in the area of ​​the first grayscale image is determined based on the first location information.

[0040] In step S13, based on the second location information and the grayscale values ​​of multiple pixels in the second grayscale image, the third location information of the at least one solder joint in the second grayscale image is determined.

[0041] In step S14, the defect detection result of the weld point is determined based on the third location information and the gray values ​​of multiple pixels in the first grayscale image.

[0042] The solder joint defect detection method according to the embodiments of the present disclosure can capture two images under illumination by light sources at two different heights and obtain their grayscale images, reduce the halo caused by energy diffusion, and use more prominent marker points to determine the position of the conductor. The characteristics of the two grayscale images can be used to determine the position of the solder joint and the defects of the solder joint in the image respectively, so as to improve the detection accuracy and reduce labor costs.

[0043] In one possible implementation, the first and second light sources can be ring-shaped light sources. The product (e.g., an electronic board) can be placed below a hole in the center of the ring. This reduces shadows caused by mutual shading between components when the ring light source illuminates the product, improving image quality and detection accuracy. The first light source can be higher than the second light source, thus providing a wider illumination range and more uniform brightness on the product. Consequently, in images obtained under the first light source, each solder joint is clearly visible, making it suitable for detecting cosmetic defects such as those affecting the product's visual appeal and comfort, such as cold solder joints or bulges. The second light source is lower and closer to the product, providing greater brightness. Therefore, in images obtained under the second light source, the contrast between the solder joints and the background is greater, making it suitable for detecting manufacturing defects such as those affecting product performance, such as missing solder joints or breaks.

[0044] In one possible implementation, since the solder joints are small in size, a high-magnification area array camera can be used to capture the first and second images. The camera can be positioned above the hole in the middle of the aforementioned ring to capture an image of the product illuminated by the first or second light source through the hole between the first and second light sources. The camera can be an RGB camera to capture images with higher resolution. This disclosure does not limit the type of camera.

[0045] In one possible implementation, in step S11, the first light source or the second light source can be turned on respectively to capture images through the camera, thereby obtaining a first grayscale image corresponding to the image captured under the illumination of the first light source and a second grayscale image corresponding to the image captured under the illumination of the second light source.

[0046] In one possible implementation, step S11 may include: acquiring a first image of at least one solder joint under illumination by a first light source and a second image under illumination by a second light source; and converting the first image and the second image into the first grayscale image and the second grayscale image, respectively.

[0047] In one possible implementation, a first light source can be turned on to illuminate the product with solder joints, and a second light source can be turned off to capture a first image of the solder joints under the illumination of the first light source using a camera. Alternatively, a second light source can be turned on to illuminate the product with solder joints, and the first light source can be turned off to capture a second image of the solder joints under the illumination of the second light source using a camera. The first and second images can be RGB images. Further, the first image can be converted to a first grayscale image, and the second image can be converted to a second grayscale image to facilitate subsequent processing, such as determining the location and type of defects based on the grayscale values ​​of pixels. In another example, the camera can be a grayscale camera, which can directly acquire the first grayscale image under the illumination of the first light source and the second grayscale image under the illumination of the second light source. This disclosure does not limit the type of camera.

[0048] In one possible implementation, in step S12, first location information of the marker points in the first grayscale image can be determined, and second location information of the conductors corresponding to the solder joints can be determined based on the first location information of the marker points. The marker points are distinctive points on the product, such as easily detectable points like markings of a certain shape or holes on the product. Furthermore, the relative positional relationship between the marker points and the conductors corresponding to each solder joint is known, and the position of conductors that are not easily detectable can be indirectly determined based on the position of the marker points. The conductors may include pads corresponding to the solder joints, such as copper sheets; this disclosure does not limit the type of conductor.

[0049] In one possible implementation, step S12 may include: obtaining the relative positional relationship between the marker point and the conductor; and performing an affine transformation on the first positional information based on the relative positional relationship to obtain the second positional information.

[0050] In one possible implementation, the area of ​​the conductor is typically slightly larger than the solder joint, and the conductor is usually laid flat on the electronic board of the product, making it difficult to detect in a grayscale image. Therefore, the position of a first marker point, which is easily detectable in the first grayscale image, can be detected, and the position of the conductor can be indirectly determined by the relative positional relationship between the marker point and each conductor. In the example, the first position information of the marker point can be detected by various image detection methods. For example, the first position information of the marker point can be detected by a deep learning neural network method, or by the NCC template matching algorithm in related technologies. This disclosure does not limit the method for detecting the first position information.

[0051] In one possible implementation, the relative positional relationship between the marker and the conductor can be obtained. For example, the position of the conductor on the product (e.g., an electronic board) is fixed, and the position of each conductor on the product can be predetermined, for example, by determining the coordinates of the center of each conductor and the dimensions of the conductor. Similarly, the position of the marker on the product is also fixed, and the position of the marker on the product can be predetermined, for example, by determining the coordinates of the center of the marker and the dimensions of the marker. Thus, the relative positional relationship between the marker and the conductor can be obtained based on their positions. For example, if the coordinates of the marker are (0, 0) and the coordinates of the center of a conductor are (20, 0), then the conductor is located 20 mm to the right of the marker.

[0052] In one possible implementation, after detecting the first position information of the marker points, an affine transformation can be performed on the first position information based on the relative positional relationship between each conductor and the marker points. For example, the first position information (i.e., the coordinates of the marker points in the first grayscale image) can be multiplied by an affine transformation matrix (a position transformation matrix obtained based on the relative positional relationship) to obtain the second position information of the conductors in the first grayscale image. In this example, the second position information is the position information of the region where the conductors are located. The coordinates of the center point of the conductor in the first grayscale image can be obtained through the above affine transformation. Furthermore, the second position information of the region where the conductors are located in the first grayscale image can be determined based on the size of the conductors.

[0053] In one possible implementation, in step S13, after determining the second position information of the conductor, the position of the solder joint can be determined using the second position information of the conductor. In the example, the third position information of the solder joint in the second grayscale image can be determined, thereby determining whether there is a process defect in the solder joint based on the third position information. Since only the light source was adjusted when the first and second images were captured, and other information such as the camera and the position of the product were not adjusted, the position of the conductor in the first and second grayscale images is consistent, and the position of the solder joint in the first and second grayscale images is also consistent.

[0054] In one possible implementation, step S13 may include: determining fourth location information of the region where the conductor is located in the second grayscale image based on the second location information; determining a first mean and a first variance of the grayscale values ​​of the pixels in the region where the conductor is located in the second grayscale image based on the fourth location information; and determining third location information of the at least one solder joint in the region where the conductor is located in the second grayscale image based on the first mean and the first variance.

[0055] In one possible implementation, as described above, the position of the conductor in the first grayscale image is the same as its position in the second grayscale image. Therefore, the fourth position information of the conductor in the second grayscale image can be directly determined based on the second position information of the conductor in the first grayscale image.

[0056] In one possible implementation, the fourth location information is the location information of the region where the conductor is located in the second grayscale image. The third location information of the solder joint can be determined based on the grayscale values ​​of the pixels in that region. In an example, the average value of the grayscale values ​​of the pixels in that region can be calculated to obtain a first mean. Based on the first mean, the first variance of the grayscale values ​​in that region can be calculated, and then the third location information of the solder joint can be determined based on the first mean and the first variance.

[0057] In one possible implementation, a grayscale difference also exists between the solder joint and the conductor. This grayscale difference can be used to determine the location of the solder joint. For example, the grayscale value at the location of the solder joint is higher than that at other locations on the conductor. Therefore, the location of the solder joint can be determined by setting a threshold. For instance, pixels with a grayscale value higher than the set threshold are the pixels in the region where the solder joint is located, and vice versa. This allows the location of the solder joints on each conductor to be determined. However, the grayscale values ​​of each conductor may differ. For example, in the region where a certain conductor is located, the overall grayscale value of the pixels is higher than that in the region where another conductor is located. Therefore, setting a uniform threshold for all regions of all conductors is not suitable. Instead, a threshold can be set individually for the location of each conductor.

[0058] In one possible implementation, as described above, a first mean and a first variance of the pixels in the region where each conductor is located can be obtained, and the location of the solder joint in the region where each conductor is located can be determined based on the first mean and the first variance. In the example, the threshold for each region where each conductor is located can be determined based on the following formula (1):

[0059] threshSp=Mean+0.63×Deviation (1)

[0060] Where, threshSp is the threshold of each conductor's region, Mean is the average gray level of the pixels in each conductor's region (i.e., the first mean), and Deviation is the variance of the gray levels of the pixels in each conductor's region (i.e., the first variance). The threshold of each conductor's region can be determined based on formula (1), and pixels with gray levels greater than the threshold in each conductor's region can be identified as solder joint pixels. Thus, the third location information of the solder joints in each conductor's region can be determined.

[0061] In one possible implementation, in step S14, after detecting the location of the solder joint, the defect detection result can be determined based on the grayscale value of the pixel at the location of the solder joint. As mentioned above, solder joint defects include process defects and appearance defects; therefore, the defect detection result can include both process defect detection results and appearance defect detection results. Furthermore, as mentioned above, the image obtained under the illumination of the first light source can be used to detect appearance defects, and the image obtained under the illumination of the second light source can be used to detect process defects. Therefore, the process defect detection result can be determined based on the third location information of the solder joint in the second grayscale image, and the appearance defect detection result can be determined based on the location information of the solder joint in the first grayscale image.

[0062] In one possible implementation, step S14 may include: determining the process defect detection result of the solder joint based on the third location information; determining the fifth location information of the region where the solder joint is located in the first grayscale image based on the third location information; determining the second mean and second variance of the grayscale values ​​of the pixels in the region where the solder joint is located in the first grayscale image based on the fifth location information; and determining the appearance defect detection result of the solder joint based on the second mean and the second variance.

[0063] In one possible implementation, the detection result of a process defect, such as missing weld or breakage, can be determined based on the third location information of the weld point in the second grayscale image. In the example, the detection result of the process defect can be determined based on the size of the area where the weld point is located, as indicated by the third location information. For example, a missing weld or breakage may be caused by insufficient welding material usage, and insufficient welding material usage will also result in insufficient weld point size, such as an incorrect aspect ratio.

[0064] In one possible implementation, determining the process defect detection result of the solder joint based on the third location information includes: determining the size information of the area where the solder joint is located in the second grayscale image based on the third location information; and determining the process defect detection result of the solder joint based on the size information.

[0065] In the example, the region where the solder joint is located in the second grayscale image can be determined based on the third location information, and then the size information of the region, such as its length and width, can be determined. Then, based on parameters such as the aspect ratio, it can be determined whether the size information meets the requirements. If it does not meet the requirements, the solder joint has a manufacturing defect; that is, the manufacturing defect detection result for the solder joint can be determined as having a manufacturing defect. Conversely, if the requirements are met, the manufacturing defect detection result for the solder joint can be determined as not having a manufacturing defect.

[0066] In one possible implementation, the appearance defect detection results for each solder joint can also be determined. As mentioned above, the appearance defect detection results can be determined based on the position information of the solder joint in the first grayscale image. Therefore, the fifth position information of the solder joint in the first grayscale image can be determined based on the third position information of the solder joint in the second grayscale image. In the example, the third position information of the solder joint in the second grayscale image is consistent with the fifth position information of the solder joint in the first grayscale image; therefore, the third position information can be directly determined based on the fifth position information.

[0067] In one possible implementation, when determining the detection result of appearance defects, the grayscale values ​​of the pixels in the region where the solder joint is located within the first grayscale image can also be used. For example, a threshold can be set, and the existence of an appearance defect detection result can be determined based on the relationship between the grayscale values ​​of the pixels in the region where the solder joint is located and the set threshold. Similarly, since the grayscale values ​​of the pixels in the regions of each solder joint are different, it is not possible to set a uniform threshold to judge the appearance defect detection results of all solder joints. Therefore, a threshold can be set separately according to the grayscale values ​​of the pixels in the regions where each solder joint is located.

[0068] In one possible implementation, the average grayscale value of each pixel within the region of each solder joint can be calculated to obtain a second mean. Based on the second mean, the second variance of the grayscale value of each pixel can be determined. Then, based on the second mean and the second variance, a threshold for each region of each solder joint can be determined, and based on the threshold for each region of each solder joint, the appearance defect detection result for each solder joint can be determined.

[0069] In one possible implementation, determining the appearance defect detection result of the solder joint based on the second mean and the second variance includes: determining a predicted defect region within the area where the solder joint is located based on the second mean, the second variance, and a first preset parameter; determining the area where the appearance defect is located within the predicted defect region based on the second mean, the second variance, and the second preset parameter; and determining the appearance defect detection result based on the area where the appearance defect is located and the grayscale values ​​of the pixels within the area where the solder joint is located in the first grayscale image.

[0070] In one possible implementation, a predicted defect region can be determined first, thereby allowing for the search of appearance defects within the predicted defect region to reduce the search range. Determining the predicted defect region within the area where the solder joint is located, based on the second mean, the second variance, and the first preset parameter, includes: obtaining a first defect threshold based on the second mean, the second variance, and the first preset parameter; and determining the area containing pixels with grayscale values ​​less than the first defect threshold within the area where the solder joint is located as the predicted defect region.

[0071] In the example, the first defect threshold can be determined according to the following formula (2):

[0072] gray_falsewelding_3=(MeanSp+DeviationSp) / 3 (2)

[0073] Wherein, gray_falsewelding_3 is the first defect threshold, MeanSp is the average gray value of the pixels in the area where the solder joint is located, i.e., the second mean, and DeviationSp is the variance of the gray values ​​of the pixels in the area where the solder joint is located, i.e., the second variance. The first preset parameter is set to 3. This disclosure does not limit the specific value of the first preset parameter.

[0074] In one possible implementation, the first defect threshold of the area where each solder joint is located can be determined according to the above formula (2), and it can be determined whether there is a predicted defect area in the area where each solder joint is located, that is, the area where the pixel point with gray value is less than the first defect threshold is located. If there is no predicted defect area in the area where a solder joint is located, then the solder joint does not have an appearance defect, and its appearance defect detection result can be determined as not having an appearance defect. Otherwise, further judgment can be made.

[0075] This method can narrow down the scope of finding appearance defects and improve search efficiency.

[0076] In one possible implementation, determining the region where the appearance defect is located within the predicted defect region based on the second mean, the second variance, and the second preset parameter includes: obtaining a second defect threshold based on the second mean, the second variance, and the second preset parameter; and determining the region where the pixels with grayscale values ​​less than the second defect threshold are located within the predicted defect region as the region where the appearance defect is located.

[0077] In one possible implementation, after excluding solder joints where no predicted defect area exists, the search for appearance defects can continue among solder joints where the predicted defect area exists. For example, appearance defects can be further searched within the predicted defect area. In the example, a second defect threshold can be set according to the following formula (3) to further determine the area where appearance defects are located within the predicted defect area.

[0078] gray_falsewelding_4=(MeanSp+DeviationSp) / 4 (3)

[0079] Wherein, gray_falsewelding_4 is the second defect threshold, and the second preset parameter can be set to 4. This disclosure does not restrict the specific value of the second preset parameter, as long as it is less than the first preset parameter. Therefore, since the second defect threshold is less than the first defect threshold, the region where the gray value of the pixels is less than the second defect threshold can be further searched within the predicted defect region, that is, the region where the appearance defect is located. In another example, MeanSp (i.e., the second mean) in formula (3) can also be replaced with the average gray value of the pixels in the predicted defect region, and DeviationSp (i.e., the second variance) can be replaced with the variance of the gray value of the pixels in the predicted defect region. This disclosure does not restrict this.

[0080] In one possible implementation, if there are no pixels with a grayscale value lower than the second defect threshold in the predicted defect area of ​​a solder joint, then the predicted defect area does not have an appearance defect. If there are no appearance defects in all predicted defect areas of the solder joint, then the solder joint does not have an appearance defect, and the appearance defect detection result of the solder joint can be determined as not having an appearance defect. Conversely, if there are no appearance defects, then the solder joint can be determined to have an appearance defect, and the appearance defect can be classified according to the second defect threshold.

[0081] In one possible implementation, the appearance defect detection result is determined based on the grayscale values ​​of pixels in the region where the appearance defect is located and the region where the solder joint is located in the first grayscale image. That is, the specific category of the appearance defect is determined, for example, whether the appearance defect is a cold solder joint or a bulge.

[0082] In the example, the mean and variance of the grayscale values ​​of pixels within each predicted defect region can be statistically analyzed. Regions with a mean lower than a second defect threshold, a difference from the second defect threshold greater than or equal to a set value, and a number of pixels greater than a set value, are identified as bulges. For example, the set value can be set to 8, and the set number can be set to 2000. This disclosure does not limit the specific values ​​set.

[0083] In the example, a predicted defect area is identified as a cold solder joint if the variance of the pixels within the predicted defect area is greater than a set variance threshold, the number of pixels is greater than a set quantity value, the number of connected components is less than or equal to a quantity threshold, the density is less than a density threshold, the ratio of the area to the solder joint area is less than a preset percentage, and the length is greater than a preset percentage of the length of the solder joint area. In the example, the variance threshold can be set to 10, the quantity value can be set to 2000, the quantity threshold can be set to 3, the density threshold can be set to 30%, the preset percentage can be set to 70%, and the preset percentage can be set to 25%. This disclosure does not limit the specific values ​​set.

[0084] In the example, other conditions can be set to identify other appearance defects, and predicted defect areas that do not meet the above conditions can be identified as areas without appearance defects. This disclosure does not limit the conditions set.

[0085] In one possible implementation, the specific category of the predicted defect area of ​​each solder joint can be determined based on the above conditions, thereby determining which appearance defects exist in the solder joint, or whether there are no appearance defects, to obtain the appearance defect detection result.

[0086] The solder joint defect detection method according to embodiments of this disclosure can capture two images under illumination from two different light sources at different heights and obtain their grayscale images, reducing halos caused by energy diffusion. It can also utilize more prominent marker points to determine the positions of conductors and solder joints, thereby improving detection accuracy. Furthermore, the first grayscale image can be used to determine the detection results of appearance defects, and the second grayscale image can be used to determine the detection results of process defects. Further, when determining the detection results of appearance defects, multiple thresholds and conditions can be set to progressively search for appearance defects, reducing the search range and thus determining whether each solder joint has appearance defects and the type of appearance defects, improving search efficiency and accuracy. It can also reduce labor costs.

[0087] Figure 2 This diagram illustrates the application of a solder joint defect detection method according to an embodiment of the present disclosure, such as... Figure 2 As shown in the figure, the area array camera 1 and the lens 2 constitute a high-magnification area array camera, and capture a first image of the product with solder joints under the illumination of the first light source 3, and capture a second image of the product under the illumination of the second light source 4.

[0088] In one possible implementation, the first image and the second image can be converted into a first grayscale image and a second grayscale image, respectively. The first grayscale image can be used to determine the appearance defect detection results of each solder joint, and the second grayscale image can be used to determine the process defect detection results of each solder joint.

[0089] In one possible implementation, the positions of marker points in the first grayscale image can be determined, and the positions of each conductor can be determined based on the positional relationship between the conductor and the marker points. Then, in the second grayscale image, the region where the solder joint is located is determined based on the mean and variance of the grayscale values ​​of the pixels in the region where the conductor is located and formula (1).

[0090] In one possible implementation, the presence of process defects at each solder joint can be determined based on the aspect ratio of the area where the solder joint is located in the second grayscale image, thereby determining the detection result of process defects. Furthermore, the predicted defect area can be determined based on the mean and variance of the grayscale values ​​of the pixels in the area where the solder joint is located in the first grayscale image and formula (2), so that appearance defects can be further searched in the predicted defect area to narrow down the search range.

[0091] In one possible implementation, the region containing pixels with gray values ​​less than the threshold determined by formula (3) can be found in each predicted defect region and designated as the region containing the appearance defect. Furthermore, various conditions can be set to classify the appearance defect, thereby determining whether the appearance defect belongs to bulge, missing solder or other categories, and thus determining the appearance defect detection result.

[0092] It is understood that the various method embodiments mentioned above in this disclosure can be combined with each other to form combined embodiments without violating the principle and logic. Due to space limitations, this disclosure will not elaborate further. Those skilled in the art will understand that in the above methods of specific implementation, the specific execution order of each step should be determined by its function and possible internal logic.

[0093] In addition, this disclosure also provides a solder joint defect detection device, electronic equipment, computer-readable storage medium, and program, all of which can be used to implement any of the solder joint defect detection methods provided in this disclosure. The corresponding technical solutions and descriptions are described in the corresponding section of the method and will not be repeated here.

[0094] Figure 3 A schematic diagram of a solder joint defect detection apparatus according to an embodiment of the present disclosure is shown, such as... Figure 3 As shown, the device includes: a first light source 3, a second light source 4, a camera 1, and a processor, wherein the height of the first light source is higher than the height of the second light source; the camera is used to acquire a first grayscale image of the at least one solder joint when the first light source illuminates the at least one solder joint, and to acquire a second grayscale image of the at least one solder joint when the second light source illuminates the at least one solder joint; the processor is used to: determine first position information of a marker point in the first grayscale image, and determine second position information of the conductor corresponding to the at least one solder joint in the region of the first grayscale image based on the first position information; determine third position information of the at least one solder joint in the second grayscale image based on the second position information and the grayscale values ​​of multiple pixels in the second grayscale image; and determine a defect detection result of the solder joint based on the third position information and the grayscale values ​​of multiple pixels in the first grayscale image.

[0095] In one possible implementation, the camera 1 can also be combined with the lens 2 to form a high-magnification area array camera.

[0096] In one possible implementation, the processor is further configured to: determine fourth location information of the region where the conductor is located in the second grayscale image based on the second location information; determine a first mean and a first variance of the grayscale values ​​of the pixels in the region where the conductor is located in the second grayscale image based on the fourth location information; and determine third location information of the at least one solder joint in the region where the conductor is located in the second grayscale image based on the first mean and the first variance.

[0097] In one possible implementation, the defect detection results include process defect detection results and appearance defect detection results. The processor is further configured to: determine the process defect detection result of the solder joint based on the third location information; determine the fifth location information of the region where the solder joint is located in the first grayscale image based on the third location information; determine the second mean and second variance of the grayscale values ​​of the pixels in the region where the solder joint is located in the first grayscale image based on the fifth location information; and determine the appearance defect detection result of the solder joint based on the second mean and the second variance.

[0098] In one possible implementation, the processor is further configured to: determine the size information of the area where the solder joint is located in the second grayscale image based on the third location information; and determine the process defect detection result of the solder joint based on the size information.

[0099] In one possible implementation, the processor is further configured to: determine a predicted defect region within the area where the solder joint is located based on the second mean, the second variance, and the first preset parameter; determine a region containing an appearance defect within the predicted defect region based on the second mean, the second variance, and the second preset parameter; and determine the appearance defect detection result based on the region containing the appearance defect and the grayscale values ​​of pixels within the area where the solder joint is located in the first grayscale image.

[0100] In one possible implementation, the processor is further configured to: obtain a first defect threshold based on the second mean, the second variance, and the first preset parameter; and determine the region where the pixels with grayscale values ​​less than the first defect threshold are located within the area where the solder joint is located as the predicted defect region.

[0101] In one possible implementation, the processor is further configured to: obtain a second defect threshold based on the second mean, the second variance, and the second preset parameter; and determine the region where the pixels with gray values ​​less than the second defect threshold are located within the predicted defect region as the region where the appearance defect is located.

[0102] In one possible implementation, the processor is further configured to: acquire a first image of at least one solder joint under illumination by a first light source and a second image under illumination by a second light source; and convert the first image and the second image into the first grayscale image and the second grayscale image, respectively.

[0103] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0104] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for detecting weld joint defects, characterized in that, include: A first grayscale image of at least one solder joint under illumination by a first light source and a second grayscale image under illumination by a second light source are acquired, wherein the height of the first light source is higher than the height of the second light source. The first grayscale image is used to detect appearance defects of the solder joint, and the second grayscale image is used to detect process defects of the solder joint. The appearance defects include defects that affect the visual effect and comfort of the product, and the process defects include defects that affect the performance of the product. Determine the first location information of the marker point in the first grayscale image, and determine the second location information of the conductor corresponding to the at least one solder joint in the region of the first grayscale image based on the first location information; Based on the second location information and the gray values ​​of multiple pixels in the second grayscale image, the third location information of the at least one solder joint in the second grayscale image is determined. Based on the third location information and the gray values ​​of multiple pixels in the first grayscale image, the defect detection result of the weld joint is determined, and the defect detection result includes process defect detection result and appearance defect detection result. The defect detection result of the weld joint is determined based on the third location information and the grayscale values ​​of multiple pixels in the first grayscale image, including: Based on the third location information, the detection result of the process defect of the weld point is determined; Based on the third location information, the fifth location information of the solder joint within the first grayscale image is determined; Based on the fifth location information, determine the second mean and second variance of the grayscale values ​​of the pixels in the region where the solder joint is located within the first grayscale image; The appearance defect detection result of the weld joint is determined based on the second mean and the second variance.

2. The method according to claim 1, characterized in that, Determining the second location information of the conductor corresponding to the at least one solder joint in the region of the first grayscale image based on the first location information includes: Obtain the relative positional relationship between the marker point and the conductor; The second position information is obtained by performing an affine transformation on the first position information based on the relative positional relationship.

3. The method according to claim 1, characterized in that, Based on the second location information and the grayscale values ​​of multiple pixels in the second grayscale image, the third location information of the at least one solder joint in the second grayscale image is determined, including: Based on the second location information, determine the fourth location information of the conductor in the region of the second grayscale image; Based on the fourth location information, determine the first mean and first variance of the gray values ​​of the pixels in the region where the conductor is located in the second grayscale image; Based on the first mean and the first variance, the third location information of the at least one solder joint is determined within the region where the conductor is located in the second grayscale image.

4. The method according to claim 1, characterized in that, Based on the third location information, the detection result of the process defect of the weld point is determined, including: Based on the third location information, determine the size information of the area where the solder joint is located in the second grayscale image; Based on the dimensional information, the detection result of the process defect of the weld point is determined.

5. The method according to claim 1, characterized in that, Based on the second mean and the second variance, the appearance defect detection results of the weld joint are determined, including: Based on the second mean, the second variance, and the first preset parameter, a predicted defect area is determined within the area where the weld point is located. Based on the second mean, the second variance, and the second preset parameter, the region where the appearance defect is located is determined within the predicted defect region; The detection result of the appearance defect is determined based on the gray values ​​of the pixels in the area where the appearance defect is located and the area where the solder joint is located in the first grayscale image.

6. The method according to claim 5, characterized in that, Based on the second mean, the second variance, and the first preset parameter, a predicted defect area is determined within the region where the solder joint is located, including: The first defect threshold is obtained based on the second mean, the second variance, and the first preset parameter; The region containing pixels with grayscale values ​​less than the first defect threshold within the area of ​​the solder joint is defined as the predicted defect region.

7. The method according to claim 5, characterized in that, Based on the second mean, the second variance, and the second preset parameter, the region containing appearance defects within the predicted defect region is determined, including: The second defect threshold is obtained based on the second mean, the second variance, and the second preset parameter; Within the predicted defect area, the region containing pixels with grayscale values ​​less than the second defect threshold is determined as the region containing the appearance defect.

8. The method according to claim 1, characterized in that, Acquiring a first grayscale image of at least one solder joint under illumination by a first light source, and a second grayscale image under illumination by a second light source, including: A first image of at least one solder joint under illumination by a first light source and a second image under illumination by a second light source are obtained respectively. The first image and the second image are converted into the first grayscale image and the second grayscale image, respectively.

9. A weld joint defect detection device, characterized in that, include: First light source, second light source, camera, and processor. The height of the first light source is higher than the height of the second light source. The camera is used to acquire a first grayscale image of the at least one solder joint when the first light source illuminates the at least one solder joint, and to acquire a second grayscale image of the at least one solder joint when the second light source illuminates the at least one solder joint. The first grayscale image is used to detect appearance defects of the solder joint, and the second grayscale image is used to detect process defects of the solder joint. The appearance defects include defects that affect the visual effect and comfort of the product, and the process defects include defects that affect the performance of the product. The processor is used for: Determine the first location information of the marker point in the first grayscale image, and determine the second location information of the conductor corresponding to the at least one solder joint in the region of the first grayscale image based on the first location information; Based on the second location information and the gray values ​​of multiple pixels in the second grayscale image, the third location information of the at least one solder joint in the second grayscale image is determined. Based on the third location information and the gray values ​​of multiple pixels in the first grayscale image, the defect detection result of the weld joint is determined, and the defect detection result includes process defect detection result and appearance defect detection result. The processor is further configured to: determine the process defect detection result of the solder joint based on the third location information; determine the fifth location information of the region where the solder joint is located in the first grayscale image based on the third location information; determine the second mean and second variance of the grayscale values ​​of the pixels in the region where the solder joint is located in the first grayscale image based on the fifth location information; and determine the appearance defect detection result of the solder joint based on the second mean and the second variance.

Images