Soldering tin defect intelligent detection method and system based on machine vision

Abstract

The invention discloses an intelligent soldering defect detection method and system based on machine vision, and relates to the technical field of machine vision, and the method comprises the steps: obtaining a circuit board surface image, carrying out the preprocessing, precisely segmenting a soldering region through employing a Canny edge recognition algorithm, and calculating a soldering area deviation index according to the coverage area of a soldering liquid; soldering tin color deviation is detected based on an RGB color channel, and the first color deviation index and the second color deviation index obtained through analysis are combined to obtain a color deviation index; surface defects such as cracks and bubbles are recognized through edge point curvature analysis, and a surface defect deviation index is calculated according to the area of the areas where the cracks and the bubbles are located; the system comprehensively evaluates deviation indexes in three aspects of area, color and surface flaws, compares the deviation indexes with a preset threshold value, and judges whether defect signals are sent out or not and the types of the sent defect signals; the automatic intelligent detection of soldering tin defects is realized, and the accuracy and efficiency of defect detection are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision, and in particular to a method and system for intelligently detecting solder defects based on machine vision. Background Art

[0002] With the rapid development of industrial automation and intelligent manufacturing, welding, as one of the key processes in the manufacturing industry, has a quality that directly affects the performance and reliability of products. Solder defects are common quality problems in the welding process, such as insufficient solder, excessive solder, cold soldering, and false soldering. These defects not only affect the appearance of the product, but may also cause the electrical performance of the product to deteriorate or fail. Therefore, it is of great significance to quickly and accurately detect solder defects.

[0003] In a Chinese invention application with application publication number CN116523880A, a wire solder defect detection method and a wire solder defect detection system are disclosed, including: collecting a solder image of the wire after soldering; in the solder image, using the plane outside the solder as a reference plane to perform height correction on the solder; and based on the solder image, performing one or more of cold solder joint detection, excessive tin amount detection, continuous solder joint detection, and over-soldering detection.

[0004] In the above invention application, the solder image after the wire is soldered is first collected, and then the solder quality is tested based on the solder image, which can realize the detection of multiple detection items such as cold solder joint detection, excessive tin content detection, continuous soldering detection, and over-soldering detection. However, it does not take into account insufficient solder, abnormal solder color, and defects on the solder surface, which will lead to incomplete solder defect detection, and then lead to low defect detection results; in addition, most of the existing solder defect detection methods rely on manual detection, which also reduces the efficiency of defect detection. Summary of the invention

[0005] 1. Technical issues to be resolved In response to the technical problems in the background technology, the present invention proposes a method and system for intelligent detection of solder defects based on machine vision, which analyzes solder defects from three aspects: the area occupied by the solder liquid, the color of the solder, and whether there are defects on the solder surface. When defects are found in the solder, corresponding defect signals can be automatically sent; thereby solving the technical problems recorded in the background technology.

[0006] (II) Technical solution To achieve the above objectives, the present invention is implemented through the following technical solutions: A method for intelligent detection of solder defects based on machine vision, comprising: Obtain the surface image of the circuit board where the solder is located and perform corresponding preprocessing operations; perform regional segmentation on the preprocessed image based on the Canny edge recognition algorithm, including non-solder areas, pad areas covered by solder liquid, and pad areas not covered by solder liquid; calculate the area deviation index of solder based on the area of ​​the pad area covered by solder liquid ; The pad area image covered by solder liquid is divided into three channels: red R, green G, and blue B. Based on the RGB color channel, each pixel in the segmented pad area covered by solder liquid is subjected to color detection to obtain the first and second color deviation indexes respectively. The first and second color deviation indexes are combined to obtain the color deviation index ; Get the edge points identified in the pad area covered by solder liquid and calculate the curvature of each edge point , according to the curvature The surface defects of the solder liquid are classified for each edge point, specifically into crack defects and bubble defects. The area of ​​the defect area where all defect points are located is calculated based on the minimum enclosing rectangle algorithm, and the surface defect deviation index is obtained by combining it with the area of ​​the pad covered by the solder liquid. ; Get the area deviation index of solder , Color Deviation Index and surface defect deviation index , determine whether each deviation index exceeds the corresponding deviation threshold one by one as well as , and send out a corresponding defect signal; if it is determined that there is more than one deviation index exceeding the corresponding deviation threshold, send a circuit board replacement signal to the system. .

[0007] Specifically, the Sobel operator is used to calculate the gradient and direction of each pixel in the pad area; the Sobel operator is convolved with the coordinates of each pixel in the pad area to obtain and Direction gradient value and Compare the gradient strength of the current pixel with the two pixels along the positive and negative gradient directions. If the gradient strength of the current pixel is the largest compared with the other two pixels, the pixel point is taken as the maximum point, otherwise the pixel point is taken as the non-maximum point. When all the pixels in all pad areas have been checked, all the maximum points are retained and all the non-maximum points are suppressed. The retained maximum points are the edge points. The points represent each pixel in the image.

[0008] Furthermore, two thresholds are set, a high threshold and a low threshold; the gradient values ​​of the retained edge pixels are compared with the two thresholds, and for pixels below the high threshold, they are compared with the low threshold; if the pixel intensity is lower than the low threshold, the pixel is considered not to be an edge pixel; after determining the edge pixels, try to connect these pixels to form a complete edge; when connecting, if an endpoint is encountered, the algorithm will look for edges that can be connected in the low threshold image; edges that can be connected are continuously collected until all gaps are connected, and an image after edge processing is obtained; for several identified areas, the average grayscale value of each area is used to determine whether the area is a pad area not covered by solder liquid; after screening out all pad areas not covered by solder liquid, the remaining area in the pad is the pad area covered by solder liquid; the image processing software is used to identify the total area of ​​the pad covered by solder liquid on the pad, and the area of ​​the area is obtained. , then the area deviation index of solder Calculated according to the following formula: ,in, is the area of ​​the pad.

[0009] Specifically, take out the pad area image covered by solder liquid, divide the image into three channels: red R, green G, and blue B, traverse each pixel point in the pad area image covered by solder liquid, and use image processing tools to obtain the R, G, and B values ​​at each pixel point; calculate the color difference of each pixel point according to the following formula :

[0010] in, , , are the required thresholds for the red channel, green channel, and blue channel at each pixel respectively; Preset color difference threshold , determine the color difference of each pixel one by one The relationship between the size of the color difference threshold; if , it means that the color at the current pixel has a large deviation, and the pixel is marked as an abnormal pixel; otherwise, no marking is performed.

[0011] Furthermore, the mean color difference of all pixels is taken as the first color deviation index , the ratio of abnormal pixels to total pixels is used as the second color deviation index , the expression is:

[0012] in, Represents the total number of pixels in the pad area image covered by solder liquid. Indicates the total number of abnormal pixels; The color deviation index is obtained by combining the first and second color deviation indices. , the expression is: .

[0013] Specifically, the edge points identified in the pad area covered by the solder liquid and the position coordinates of each edge point are obtained. , calculate the curvature of each edge point according to the coordinates of the nearest edge point , the expression is:

[0014] in, and is with point The coordinate increment of the adjacent left or top edge point, and is with point The coordinate increment of the adjacent right or bottom edge point; When the curvature When , the edge point is a crack defect point; all edge points with a curvature of 0 are taken from the edge point set, and the slope of the straight line connecting any two edge points is calculated one by one. The two points with the same slope are edge points on the same crack; after all edge points with a curvature of 0 are analyzed, a number of edge points with the same slope are counted, and these edge points are connected to form a number of straight lines. These straight lines are the identified cracks, and the edge points that form the same straight line are put into a crack set; for a single edge point, that is, the slope of the straight line connected by the edge point and any other edge point is not the same as the slope of the straight line formed by any other edge points, it is put back into the edge point set.

[0015] Furthermore, all edge points with the same curvature and not zero in the edge point set are counted, and these edge points with the same curvature are grouped into edge point subsets. The following processing is performed on the edge points in each edge point subset: the straight-line distance between the edge point and other edge points in the edge point subset is calculated, and the largest straight-line distance calculated is used as the fitting diameter of the edge point , and calculate the fitting diameter of each edge point in turn ; Calculate the standard deviation of the fitted diameters of all edge points in this edge subset , the expression is:

[0016] in, is the number of edge points in this edge subset, is the mean of the fitted diameters of each edge point in the edge subset; Setting the standard deviation threshold ,like , which means that the edge points in the edge subset form a circle, that is, there is a bubble defect on the solder liquid surface, and these edge points that form the same circle are put into a bubble set; otherwise, it means that the edge points in the edge subset do not form a circle, and the edge points in the edge subset are put back into the edge point set. Furthermore, based on the minimum enclosing rectangle algorithm, the minimum bounding rectangle of all edge points is found so that the cracks and bubble areas obtained by analysis can be placed in the rectangle. The area of ​​the rectangle is recorded as ; The area of ​​the rectangle is combined with the area of ​​the pad covered by the solder liquid to obtain the surface defect deviation index , the expression is: .

[0017] Specifically, obtain the area deviation index of solder , Color Deviation Index and surface defect deviation index , respectively judge the corresponding deviation threshold If a certain deviation index exceeds the corresponding deviation threshold, a corresponding defect signal is sent to the system; When it is determined that more than one deviation index exceeds the corresponding deviation threshold, multiple corresponding defect signals and circuit board replacement signals are sent to the system. Solder with multiple defects requires the circuit board to be replaced.

[0018] A solder defect intelligent detection system based on machine vision, comprising: The data acquisition and processing module acquires the surface image of the circuit board where the solder is located and performs corresponding preprocessing operations; based on the Canny edge recognition algorithm, the preprocessed image is segmented into regions, including non-solder regions, pad regions covered by solder liquid, and pad regions not covered by solder liquid; The defect analysis module detects defects on the solder surface by analyzing the image, and specifically includes an area defect detection unit, a color defect detection unit, and a surface defect detection unit; wherein the area defect detection unit analyzes whether there is an area defect by the area of ​​the pad area covered by the solder liquid; the color defect detection unit determines whether there is a color defect by performing color detection on each pixel point in the pad area covered by the solder liquid; the surface defect detection unit determines whether there is a surface defect by detecting cracks and bubbles on the surface of the solder liquid; The defect signal sending module determines whether to send a defect signal and the type of the defect signal according to the area deviation index, color deviation index and surface defect deviation index and the preset deviation threshold.

[0019] (III) Beneficial effects The present invention provides a method and system for intelligent detection of solder defects based on machine vision, which has the following beneficial effects: 1. Obtain clear images through a monocular camera, improve image quality through preprocessing, and then use advanced area segmentation technology to accurately identify solder and non-solder areas, and further finely divide the pad coverage through edge recognition algorithm; use image processing software to identify the area of ​​the pad covered by solder liquid and compare it with the actual area of ​​the pad to obtain the area deviation index of the solder; simplify the subsequent solder defect analysis process and improve the accuracy and efficiency of solder defect detection; 2. By decomposing the pad area image covered by solder liquid into three channels of red, green and blue, and performing color detection based on the pixel values ​​of these channels, the color change of the solder liquid can be accurately analyzed; by calculating the color difference of each pixel point and setting the threshold, the pixel points with abnormal color can be effectively identified, and then the quality or state of the solder liquid can be judged; by combining multiple color deviation indexes, the uniformity and defects of the solder liquid surface can be comprehensively evaluated, which improves the accuracy and efficiency of solder defect detection; 3. By accurately identifying the edge points of the solder liquid covering the pad area and using curvature calculation to classify the defect points, the crack defect points and bubble defect points are effectively distinguished; the area of ​​the defect area is calculated using the minimum enclosing rectangle algorithm, and the surface defect deviation index is obtained in combination with the pad area, which provides a basis for the quantitative evaluation of solder liquid surface defects and improves the accuracy of defect detection; 4. By comprehensively considering the area deviation index, color deviation index and surface defect deviation index of solder, the accuracy and comprehensiveness of solder defect detection are effectively improved; when any deviation index exceeds the corresponding preset threshold, the system will promptly send out the corresponding defect signal to facilitate rapid positioning and processing of solder defects; if there are multiple deviation indices exceeding the corresponding deviation threshold, a signal to replace the circuit board is sent, reducing a large amount of cost and time caused by maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A flowchart of the steps of the intelligent detection method for solder defects based on machine vision provided by the present invention; Figure 2 This is a structural schematic diagram of the machine vision-based solder defect intelligent detection system provided by the present invention. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] refer to Figure 1 The present invention provides a method for intelligent detection of solder defects based on machine vision, comprising: Step 1: Obtain the surface image of the circuit board where the solder is located, and perform corresponding preprocessing operations; perform regional segmentation on the preprocessed image based on the Canny edge recognition algorithm, including non-solder areas, pad areas covered by solder liquid, and pad areas not covered by solder liquid; calculate the area deviation index of solder based on the area of ​​the pad area covered by solder liquid ; The step 1 comprises the following steps: Step 101: Obtain a surface image of a circuit board where solder is located by using a monocular camera, and determine the optimal distance between the monocular camera and the circuit board before shooting to obtain the clearest image; and determine the horizontal position of the monocular camera so that the solder pad is in the center area of ​​the image; Step 102: further preprocessing the acquired circuit board surface image to improve the image quality and recognition rate, and provide a better basis for subsequent image analysis and processing, including: using various filters, such as median filter, Gaussian filter, etc., to smooth the image to reduce the influence of noise; using histogram equalization, contrast stretching and other algorithms to enhance the image contrast, so as to better identify the features in the image; Step 103, segmenting the surface image of the circuit board into three regions, namely, a non-solder region, an uncovered pad region, and a solder liquid region; first, obtaining the position coordinate information of the pad in the current circuit board image from the circuit board database, and judging the position of the pad in the current circuit board image according to the horizontal distance of the circuit board image from the lower left of the circuit board, and marking it; segmenting the marked region from the unmarked region to form two regions, namely, a non-solder region and a pad region, and removing the mark of the pad region after segmentation; For the pad area, the pad area covered by the solder liquid and the pad area not covered by the solder liquid are identified based on the Canny edge recognition algorithm, including: Step 1031, using the Sobel operator to calculate the gradient and direction of each pixel in the pad area, where: and The Sobel operators in the directions are and ; Convolve the Sobel operator with the coordinates of each pixel in the pad area to obtain and Direction gradient value and ; The gradient of each pixel is calculated by the following formula With the gradient direction :

[0023] Step 1032: compare the gradient strength of the current pixel with the two pixels along the positive and negative gradient directions. If the gradient strength of the current pixel is the largest compared with the other two pixels, the pixel point is taken as a maximum point, otherwise the pixel point is taken as a non-maximum point. When all the pixels in all the pad areas have been checked, all the maximum points are retained and all the non-maximum points are suppressed. The retained maximum points are edge points. The points represent each pixel in the image. Step 1033, set two thresholds, a high threshold and a low threshold; compare the gradient value of the retained edge pixel with the two thresholds, and for pixels below the high threshold, compare with the low threshold; if the pixel intensity is lower than the low threshold, it is considered that the pixel is not an edge pixel; after determining the edge pixel, try to connect these pixel points to form a complete edge; when connecting, if an endpoint is encountered, the algorithm will look for edges that can be connected in the low threshold image; continuously collect edges that can be connected until all gaps are connected, and then obtain an image after edge processing; for several identified areas, determine whether the area is a pad area not covered by solder liquid according to the average gray value of each area; after screening out all pad areas not covered by solder liquid, the remaining area in the pad is the pad area covered by solder liquid; S104, merging adjacent areas in the pad area covered by the solder liquid to form a total pad area covered by the solder liquid, and recording edge points between adjacent areas, which can reflect defects inside the solder liquid; using image processing software to identify the total pad area covered by the solder liquid on the pad, and obtaining the area of ​​the area as , then the area deviation index of solder Calculated according to the following formula:

[0024] in, is the area of ​​the pad.

[0025] The proportion of solder liquid to pad area directly reflects the quality and effect of the welding process. When the solder liquid does not cover enough pad area, it may lead to loose welding, false welding or cold welding, thus affecting the stability and reliability of electronic products.

[0026] When using, combine the contents in steps 101 to 103: A monocular camera is used to obtain clear images, which are then pre-processed to improve image quality. Advanced region segmentation technology is then used to accurately identify solder and non-solder areas. The pad coverage is further refined through an edge recognition algorithm. Image processing software is used to identify the area of ​​the pad covered by the solder liquid and compare it with the actual area of ​​the pad to derive the area deviation index of the solder. This design can simplify the subsequent solder defect analysis process and improve the accuracy and efficiency of solder defect detection.

[0027] Step 2: The pad area image covered by solder liquid is divided into three channels: red R, green G and blue B. Based on the RGB color channel, each pixel in the segmented pad area covered by solder liquid is subjected to color detection to obtain the first and second color deviation indexes respectively. The first and second color deviation indexes are combined to obtain the color deviation index ; Color changes in solder liquid may directly reflect its welding quality or status. For example, solder liquid should present a specific color during normal soldering, but the color of the solder liquid may change if oxidation, contamination or other abnormal conditions occur. Color detection can also help identify specific defects on the surface of the solder liquid. Certain solder defects, such as insufficient solder, excessive solder or uneven solder, may cause uneven color distribution or abnormal color in the solder liquid area. Color detection can accurately identify these defects and provide a basis for subsequent repairs and improvements.

[0028] The step 2 comprises the following steps: Step 201, taking out an image of the pad area covered by the solder liquid, dividing the image into three channels of red R, green G and blue B, and performing color analysis on the solder liquid surface based on the intensity of the three color channels in the RGB color channel; Step 202: traverse each pixel in the image of the pad area covered by the solder liquid, use image processing tools to obtain the R, G, and B values ​​at each pixel; calculate the color difference of each pixel according to the following formula: :

[0029] in, , , are the required thresholds for the red channel, green channel, and blue channel at each pixel respectively; Step 203: Preset color difference threshold , determine the color difference of each pixel one by one The relationship between the size of the color difference threshold; if , it means that the color at the current pixel has a large deviation, and the pixel is marked as an abnormal pixel; otherwise, no marking is performed; Step 204: taking the average color difference of all pixels as the first color deviation index , the ratio of abnormal pixels to total pixels is used as the second color deviation index , the expression is:

[0030] in, Represents the total number of pixels in the pad area image covered by solder liquid. Indicates the total number of abnormal pixels; The color deviation index is obtained by combining the first and second color deviation indices. , the expression is: , wherein the first color deviation index determines whether the color of the solder liquid surface meets the requirements, and the second color deviation index analyzes and calculates the number of defects on the solder liquid surface, and the above two analyses are independent of each other, and the first and second color deviation indices are combined to comprehensively determine whether the color of the solder liquid surface has a large color deviation; When using, combine the contents in steps 201 to 204: By decomposing the image of the pad area covered by the solder liquid into three channels of red, green and blue, and performing color detection based on the pixel values ​​of these channels, the color changes of the solder liquid can be accurately analyzed; by calculating the color difference of each pixel point and setting a threshold, the pixels with abnormal colors can be effectively identified, and then the quality or state of the solder liquid can be judged; combining multiple color deviation indexes, the uniformity and defects of the solder liquid surface can be comprehensively evaluated, improving the accuracy and efficiency of solder defect detection.

[0031] Step 3: Get the edge points identified in the pad area covered by solder liquid and calculate the curvature of each edge point , according to the curvature The surface defects of the solder liquid are classified for each edge point, specifically into crack defects and bubble defects. The area of ​​the defect area where all defect points are located is calculated based on the minimum enclosing rectangle algorithm, and the surface defect deviation index is obtained by combining it with the area of ​​the pad covered by the solder liquid. ; The step three comprises the following steps: Step 301: Obtain edge points identified in the pad area covered by solder liquid and the position coordinates of each edge point , calculate the curvature of each edge point according to the coordinates of the nearest edge point , the expression is:

[0032] in, and is with point The coordinate increment of the adjacent left or top edge point, and is with point The coordinate increment of the adjacent right or bottom edge point; Step 302: When the curvature When , the edge point is a crack defect point; take out all edge points with a curvature of 0 from the edge point set, and calculate the slope of the straight line connecting any two edge points one by one. The two points with the same slope are edge points on the same crack; after analyzing all edge points with a curvature of 0, count a number of edge points with the same slope, and connect these edge points to form a number of straight lines. These straight lines are the identified cracks, and put these edge points that form the same straight line into a crack set; for a single edge point, that is, the slope of the straight line connecting the edge point and any other edge point is not the same as the slope of the straight line formed by any other edge points, put it back into the edge point set; Step 303: Count all edge points in the edge point set that have the same curvature and are not zero, and group these edge points with the same curvature into edge point subsets. Perform the following processing on the edge points in each edge point subset: calculate the straight-line distance between the edge point and other edge points in the edge point subset to which it belongs, and use the calculated maximum straight-line distance as the fitting diameter of the edge point. , and calculate the fitting diameter of each edge point in turn ; Calculate the standard deviation of the fitted diameters of all edge points in this edge subset , the expression is:

[0033] in, is the number of edge points in this edge subset, is the mean of the fitted diameters of each edge point in the edge subset; Setting the standard deviation threshold ,like , which means that the edge points in the edge subset form a circle, that is, bubble defects appear on the solder liquid surface, and these edge points forming the same circle are put into a bubble set; otherwise, it means that the edge points in the edge subset do not form a circle, and the edge points in the edge subset are put back into the edge point set; Step 304: Based on the minimum enclosing rectangle algorithm, find the minimum bounding rectangle of all edge points so that the cracks and bubble areas obtained by analysis can be placed in the rectangle. The area of ​​the rectangle is recorded as ; The area of ​​the rectangle is combined with the area of ​​the pad covered by the solder liquid to obtain the surface defect deviation index , the expression is: .

[0034] When used, combine the contents in steps 301 to 304: By accurately identifying the edge points of the solder liquid covering the pad area and using curvature calculation to classify the defect points, crack defect points and bubble defect points are effectively distinguished; the area of ​​the defect area is calculated using the minimum enclosing rectangle algorithm, and the surface defect deviation index is obtained in combination with the pad area, which provides a basis for the quantitative evaluation of solder liquid surface defects; this design not only improves the accuracy of defect detection, but also helps to make a more comprehensive evaluation of solder quality.

[0035] Step 4: Get the area deviation index of solder , Color Deviation Index and surface defect deviation index , determine whether each deviation index exceeds the corresponding deviation threshold one by one as well as , and send out a corresponding defect signal; if it is determined that there is more than one deviation index exceeding the corresponding deviation threshold, send a circuit board replacement signal to the system; The step 4 comprises the following steps: Step 401: Obtaining the area deviation index of solder , Color Deviation Index and surface defect deviation index , respectively judge the corresponding deviation threshold If a certain deviation index exceeds the corresponding deviation threshold, a corresponding defect signal is sent to the system; Step 402: When it is determined that more than one deviation index exceeds the corresponding deviation threshold, multiple corresponding defect signals and circuit board replacement signals are sent to the system. Solder with multiple defects requires circuit board replacement.

[0036] When using, combine the contents in steps 401 to 402: By comprehensively considering the area deviation index, color deviation index and surface defect deviation index of solder, the accuracy and comprehensiveness of solder defect detection are effectively improved; when any deviation index exceeds the corresponding preset threshold, the system will promptly send out the corresponding defect signal to facilitate rapid positioning and processing of solder defects; if there are multiple deviation indices exceeding the corresponding deviation threshold, a signal to replace the circuit board is sent, reducing a large amount of cost and time generated by maintenance.

[0037] refer to Figure 2 The present invention also provides a solder defect intelligent detection system based on machine vision, comprising: The data acquisition and processing module acquires the surface image of the circuit board where the solder is located and performs corresponding preprocessing operations; based on the Canny edge recognition algorithm, the preprocessed image is segmented into regions, including non-solder regions, pad regions covered by solder liquid, and pad regions not covered by solder liquid; The defect analysis module detects defects on the solder surface by analyzing the image, and specifically includes an area defect detection unit, a color defect detection unit, and a surface defect detection unit; wherein the area defect detection unit analyzes whether there is an area defect by the area of ​​the pad area covered by the solder liquid; the color defect detection unit determines whether there is a color defect by performing color detection on each pixel point in the pad area covered by the solder liquid; the surface defect detection unit determines whether there is a surface defect by detecting cracks and bubbles on the surface of the solder liquid; The defect signal sending module determines whether to send a defect signal and the type of the defect signal according to the area deviation index, color deviation index and surface defect deviation index and the preset deviation threshold.

[0038] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer storage medium or transmitted via a computer storage medium.

[0039] Computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. Computer storage media can be any available media that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media. Available media can be magnetic media (e.g., floppy disks, hard disks, tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid state drives (SSDs)).

[0040] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for intelligent detection of solder defects based on machine vision, characterized in that: include: Obtain the surface image of the circuit board where the solder is located and perform corresponding pre-processing operations; The preprocessed image is segmented based on the Canny edge recognition algorithm, including the non-solder area, the pad area covered by the solder liquid, and the pad area not covered by the solder liquid; the solder area deviation index is calculated based on the area of ​​the pad area covered by the solder liquid ; The pad area image covered by solder liquid is divided into three channels: red R, green G, and blue B. Based on the RGB color channel, each pixel in the segmented pad area covered by solder liquid is subjected to color detection to obtain the first and second color deviation indexes respectively. The first and second color deviation indexes are combined to obtain the color deviation index ; Get the edge points identified in the pad area covered by solder liquid and calculate the curvature of each edge point , according to the curvature The surface defects of the solder liquid are classified for each edge point, specifically into crack defects and bubble defects. The area of ​​the defect area where all defect points are located is calculated based on the minimum enclosing rectangle algorithm, and the surface defect deviation index is obtained by combining it with the area of ​​the pad covered by the solder liquid. ; Get the area deviation index of solder , Color Deviation Index and surface defect deviation index , determine whether each deviation index exceeds the corresponding deviation threshold one by one as well as , and send out a corresponding defect signal; if it is determined that there is more than one deviation index exceeding the corresponding deviation threshold, a signal to replace the circuit board is sent to the system.

2. The method for intelligent detection of solder defects based on machine vision as claimed in claim 1, characterized in that: The Sobel operator is used to calculate the gradient and direction of each pixel in the pad area; the Sobel operator is convolved with the coordinates of each pixel in the pad area to obtain and Direction gradient value and Compare the gradient strength of the current pixel with the two pixels along the positive and negative gradient directions. If the gradient strength of the current pixel is the largest compared with the other two pixels, the pixel point is taken as the maximum point, otherwise the pixel point is taken as the non-maximum point. When all the pixels in all pad areas have been checked, all the maximum points are retained and all the non-maximum points are suppressed. The retained maximum points are the edge points. The points represent each pixel in the image.

3. The method for intelligent detection of solder defects based on machine vision as claimed in claim 2, characterized in that: Set two thresholds, a high threshold and a low threshold; compare the gradient values ​​of the retained edge pixels with the two thresholds, and for pixels below the high threshold, compare them with the low threshold; if the pixel intensity is lower than the low threshold, the pixel is considered not an edge pixel; after determining the edge pixels, try to connect these pixels to form a complete edge; when connecting, if an endpoint is encountered, the algorithm will look for edges that can be connected in the low threshold image; continue to collect edges that can be connected until all gaps are connected to obtain an image after edge processing; for several identified areas, determine whether the area is a pad area not covered by solder liquid based on the average grayscale value of each area; After filtering out all the pad areas not covered by the solder liquid, the remaining area in the pad is the pad area covered by the solder liquid; the total area of ​​the pad covered by the solder liquid on the pad is identified using image processing software, and the area of ​​the area is , then the area deviation index of solder Calculated according to the following formula: ,in, is the area of ​​the pad.

4. The method for intelligent detection of solder defects based on machine vision as claimed in claim 3, characterized in that: Take out the pad area image covered by solder liquid, divide the image into three channels: red R, green G, and blue B, traverse each pixel in the pad area image covered by solder liquid, and use image processing tools to obtain the R, G, and B values ​​at each pixel; calculate the color difference of each pixel according to the following formula : in, , , are the required thresholds for the red channel, green channel, and blue channel at each pixel respectively; Preset color difference threshold , determine the color difference of each pixel one by one The relationship between the size of the color difference threshold; if , it means that the color at the current pixel is deviated, and the pixel is marked as an abnormal pixel; otherwise, no marking is performed.

5. The method for intelligent detection of solder defects based on machine vision as claimed in claim 4, characterized in that: The mean color difference of all pixels is used as the first color deviation index , the ratio of abnormal pixels to total pixels is used as the second color deviation index , the expression is: in, Represents the total number of pixels in the pad area image covered by solder liquid. Indicates the total number of abnormal pixels; the first and second color deviation indexes are combined to obtain the color deviation index of the pad area , the expression is: .

6. The method for intelligent detection of solder defects based on machine vision as claimed in claim 5, characterized in that: Get the edge points identified in the pad area covered by solder liquid and the position coordinates of each edge point , calculate the curvature of each edge point according to the coordinates of the nearest edge point , the expression is: in, and is with point The coordinate increment of the adjacent left or top edge point, and is with point The coordinate increment of the adjacent right or bottom edge point; When the curvature When , the edge point is a crack defect point; all edge points with a curvature of 0 are taken from the edge point set, and the slope of the straight line connecting any two edge points is calculated one by one. The two points with the same slope are edge points on the same crack; after all edge points with a curvature of 0 are analyzed, a number of edge points with the same slope are counted, and these edge points are connected to form a number of straight lines. These straight lines are the identified cracks, and the edge points that form the same straight line are put into a crack set; for a single edge point, that is, the slope of the straight line connected by the edge point and any other edge point is not the same as the slope of the straight line formed by any other edge points, it is put back into the edge point set.

7. The intelligent solder defect detection method based on machine vision as claimed in claim 6, characterized in that: Count all edge points with the same curvature and not 0 in the edge point set, and group these edge points with the same curvature into edge point subsets. Perform the following processing on the edge points in each edge point subset: calculate the straight-line distance between the edge point and other edge points in its edge point subset, and use the calculated maximum straight-line distance as the fitting diameter of the edge point , and calculate the fitting diameter of each edge point in turn ; Calculate the standard deviation of the fitted diameters of all edge points in this edge subset , the expression is: in, is the number of edge points in this edge subset, is the mean of the fitted diameters of each edge point in the edge subset; Setting the standard deviation threshold ,like , indicating that the edge points in the edge subset form a circle, that is, bubble defects appear on the surface of the solder liquid, and these edge points that form the same circle are put into a bubble set; otherwise, it means that the edge points in the edge subset do not form a circle, and the edge points in the edge subset are put back into the edge point set.

8. The method for intelligent detection of solder defects based on machine vision as claimed in claim 7, characterized in that: Based on the minimum enclosing rectangle algorithm, the minimum bounding rectangle of all edge points is found so that the cracks and bubble areas obtained by analysis can be placed in the rectangle. The area of ​​the rectangle is recorded as ; The area of ​​the rectangle is combined with the area of ​​the pad covered by the solder liquid to obtain the surface defect deviation index , the expression is: .

9. The method for intelligent detection of solder defects based on machine vision as claimed in claim 8, characterized in that: Get the area deviation index of solder , Color Deviation Index and surface defect deviation index , respectively judge the corresponding deviation threshold as well as If a certain deviation index exceeds the corresponding deviation threshold, the corresponding defect signal is sent to the system; when it is determined that more than one deviation index exceeds the corresponding deviation threshold, multiple corresponding defect signals and circuit board replacement signals are sent to the system, and solder with multiple defects requires the circuit board replacement operation.

10. A solder defect intelligent detection system based on machine vision, characterized in that: include: The data acquisition and processing module acquires the surface image of the circuit board where the solder is located and performs corresponding pre-processing operations; Based on the Canny edge recognition algorithm, the preprocessed image is segmented into regions, including non-solder regions, pad regions covered by solder liquid, and pad regions not covered by solder liquid; The defect analysis module detects defects on the solder surface by analyzing the image, and specifically includes an area defect detection unit, a color defect detection unit, and a surface defect detection unit; wherein the area defect detection unit analyzes whether there is an area defect by the area of ​​the pad area covered by the solder liquid; the color defect detection unit determines whether there is a color defect by performing color detection on each pixel point in the pad area covered by the solder liquid; the surface defect detection unit determines whether there is a surface defect by detecting cracks and bubbles on the surface of the solder liquid; The defect signal sending module determines whether to send a defect signal and the type of the defect signal according to the area deviation index, color deviation index and surface defect deviation index and the preset deviation threshold.

Images