A method for rapid detection and positioning of metal terminals

Abstract

The present application discloses a method for rapid detection and positioning of metal terminals, which relates to a method for detecting and positioning metal terminals in a circuit board image. The preprocessing part mainly completes the preprocessing of the image, such as performing ROI region selection and mean filtering on the image; the detection part extracts multi-segment gray-scale regions from the preprocessed image and classifies the terminals; the positioning part extracts gray-scale regions, dilates, fills regions, erodes, performs opening operations, divides the connected region area and locates the center point of the terminals on the detected image. Finally, the pixel coordinates are converted into the physical coordinate system to perform subsequent welding operations. A method for rapid detection and positioning of metal terminals is provided. This method greatly reduces the time for terminal detection and positioning. It only takes less than 0.15 s to complete the detection and positioning tasks of all terminals on the entire circuit board, and the average accuracy can reach over 90%. It greatly reduces the manual workload, and this method can achieve fully automated process production.

Description

Technical Field

[0001] The present application relates to a method for quickly detecting and positioning metal terminals, which facilitates subsequent welding operations and belongs to the technical field of metal components. Background Art

[0002] The terminal part is a component of the circuit board. It is usually made of metal materials such as copper and can be used interchangeably in similar products. It is mainly used for transmitting electrical signals and conducting electricity.

[0003] At present, the terminal welding work on the circuit board is mainly carried out by manually setting the physical position for welding operations. First, it is necessary to sort out and summarize the design data of different types of circuit boards. Through the label position information in the circuit board data, the upper computer is controlled to offset the physical position of the welding robot arm, and then the welding operation is performed.

[0004] However, the efficiency of manually setting the position offset in the prior art is not high, far from meeting the industry's needs. Therefore, a large amount of labor cost is required to support it. Moreover, the accuracy of manually setting the position offset is not high, resulting in uneven quality of the entire batch of products. For the terminals on the circuit board that have been welded, in order to avoid secondary welding, only manual identification can be used, which is time-consuming and laborious. Summary of the Invention

[0005] The technical problem to be solved by the present application is that the welding of the metal terminals of the circuit board depends on manually setting the position offset, and the efficiency of manually setting the position offset is not high, far from meeting the industry's needs.

[0006] To solve the above technical problem, the technical solution of the present application is to provide a method for quickly detecting and positioning metal terminals. Through one-time detection and positioning, it can distinguish whether the terminals have been welded and accurately obtain the center position of the metal terminals. The method is characterized by including the following steps:

[0007] Step 1: Obtain the circuit board image;

[0008] Step 2: Extract the ROI region of the image and filter the surrounding noise through the Mean algorithm;

[0009] Step 3: Set three gray-scale threshold values for extraction, namely black, dark, and bright. The terminals are divided into four categories according to the set ratio, namely: already welded, the terminal area is darker, the terminal area is alternately bright and dark, and the terminal area is brighter;

[0010] Step 4: Adopt the gray-scale threshold extraction method to perform different threshold extractions on the three types of non-welded terminals in Step 3 to obtain a binary image;

[0011] Step 5: Perform dilation processing on the binary image obtained in Step 4 to obtain the dilated binary image;

[0012] Step Six: Perform region filling on the binary image obtained in Step Five to obtain the filled binary image;

[0013] Step Seven: For the binary image obtained in Step Six, preliminarily remove the interference regions through the erosion algorithm and the opening operation algorithm to obtain the binary image after preliminary removal;

[0014] Step Eight: Perform split connected component processing on the binary image obtained in Step Seven, and only retain the binary image of the largest terminal part;

[0015] Step Nine: Fit the binary image of the terminal part into a rectangle and obtain the center point coordinates of this rectangular binary region;

[0016] Step Ten: Convert the pixel coordinates obtained in Step Nine into physical world coordinates through the calibration method;

[0017] Step Eleven: The world coordinates obtained in Step Ten can be used for subsequent welding.

[0018] Preferably, in Step One, the circuit board is imaged by a CMOS camera and an annular blue light source to obtain a sample image.

[0019] Preferably, Step Two specifically includes:

[0020] First, extract the rectangular region containing the terminal through the ROI region, and then filter the noise through the Mean algorithm. Taking the current pixel as the center, the average value of the pixels of all pixel points in a region where the number of m rows and n columns is equal is calculated; the specific formula is as follows,

[0021]

[0022] where g(s,t) represents the original image, and f(x,y) represents the image obtained after mean filtering.

[0023] Preferably, Step Three specifically includes: setting up three threshold ranges to extract the gray region of the terminal, dividing the extracted area by the ROI area to obtain three ratios, namely a, b, and c. When a>0.4, it is determined that the terminal has been welded. The remaining terminals are classified into: the terminal region is darker, the terminal region has alternating light and dark, and the terminal region is brighter by judging the maximum value of a, b, and c.

[0024] Preferably, in the fifth step, the binary image obtained in the fourth step is dilated using a square structuring element with a side length of 90: if the pixel value of this point is 1, this point is not processed; if the pixel value of this point is 0, all pixel points within the structuring element where this point is located are scanned. If all pixel values of the pixel points within the structuring element are 0, the pixel value of this point is 0; otherwise, the pixel value is set to 1.

[0025] Preferably, in the sixth step, the binary image obtained in the fifth step is subjected to region filling. To connect the holes within the connected region, the seed growth method is used to find all background pixel points for marking. The remaining points with a gray value of 0 are the hole points within the connected region. The pixels of all gray value 0 pixel points that are not background pixel points are assigned a value of 255, achieving the purpose of filling the image.

[0026] Preferably, in the seventh step, the binary image obtained in the sixth step is eroded using a square structuring element with a side length of 90, and then an opening operation is performed using a square structuring element with a side length of about 60. This is achieved through the following process: The binary image erosion operation is performed point by point. If the pixel value of this point is 0, this point is not processed; if the pixel value of this point is 1, all pixel points within the structuring element where this point is located are scanned. If all pixel values of the pixel points within the structuring element are 1, the pixel value of this point is 1; otherwise, the pixel value is set to 0.

[0027] Preferably, in the eighth step, the binary image obtained in the seventh step is subjected to a connected component segmentation process. If the area of the connected component is between 10,000 and 100,000, the connected component is determined to be the terminal area, and those with an area less than 10,000 are the surrounding noise interferences.

[0028] Preferably, in the ninth step, the binary image obtained in the eighth step is subjected to a rectangular fitting process. The terminal area is close to a standard rectangle. Through the rectangular fitting process, the binary area can be made to coincide with the terminal area, and then the center point coordinates of this binary rectangular image can be obtained.

[0029] Preferably, in the conversion process of the tenth step, the transformation matrix for converting pixel coordinates to world coordinates needs to be calculated first. The calculation method is as follows:

[0030] By operating the welding head at the end of the robotic arm to weld N points on the calibration plate, where N is more than 3. After each welding is completed, the coordinates of the end of the robotic arm at this time are saved, thus obtaining N pairs of world coordinates;

[0031] After N points are welded, a photo sample is obtained by taking a photo with a camera, and N pairs of pixel coordinates in the photo sample are obtained through processing;

[0032] After that, the transformation matrix for converting pixel coordinates to world coordinates is obtained through an affine transformation;

[0033] Then multiply this transformation matrix by the pixel coordinates of each terminal to obtain the world coordinates corresponding to all terminals.

[0034] Compared with the prior art, the present application has the following beneficial effects:

[0035] 1. The method for rapid detection and center point positioning of the metal terminals uses the image grayscales of the welded terminals and the non-welded terminals as features to rapidly identify the welding conditions of the terminals. Compared with manual individual identification, the efficiency is greatly improved, and there is no need for a large amount of labor cost support.

[0036] 2. The method for rapid detection and center point positioning of the metal terminals roughly locates the candidate area of the terminals for the non-welded terminals through the image grayscale extraction algorithm, and then accurately calculates the position of the center point of the terminals through vision algorithms such as the dilation algorithm, the erosion algorithm, and the opening operation. Then, the pixel coordinate system is converted into the physical world coordinate system to perform subsequent welding operations. Compared with the operation method of manually setting a fixed offset position, a fully automated process production can be realized, the efficiency is greatly improved, and the production efficiency of a small amount of equipment can exceed that of a large number of manual workers. The industry requirements can be met with the minimum cost, and there is no need for a large amount of labor cost support. The accuracy of automatically setting the position offset is higher, and the production quality of the products is guaranteed. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a flowchart of a method for rapid detection and positioning of a metal terminal according to the present invention;

[0038] Figure 2 It is the original circuit board diagram in the first step of the detection method;

[0039] Figure 3 It is the circuit board diagram obtained in the second step of the detection method;

[0040] Figure 4 It is a three-category area comparison diagram obtained in the third step of the detection method;

[0041] Figure 5 It is the result diagram of the ninth step of the detection method. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0042] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the embodiments and the drawings of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0043] The method provided in the embodiments of the present application is for the rapid inspection and center point positioning of metal terminals. Through one-time detection and positioning, it can quickly detect whether the terminals are welded and accurately locate the center positions of the terminals. It is applicable to circuit boards of various models, and the positioning results are used for subsequent welding, enabling fully automated production. The specific steps are as follows:

[0044] Step 1: Obtain the circuit board image;

[0045] In Step 1, the acquisition of the circuit board image is specifically achieved through the following process. The circuit board is imaged by a CMOS camera and an annular blue light source, which can clearly present the characteristics of the metal terminals and obtain a sample image. The image is as Figure 2 shown.

[0046] Step 2: Extract the ROI region from the image and filter out surrounding noise through the Mean algorithm;

[0047] In Step 2, first, a rectangular region containing the terminals is extracted through ROI region extraction, and then the noise is filtered through the Mean(11*11) algorithm. As a filtering algorithm, the Mean algorithm is different from other commonly used bilinear filtering, median filtering, etc. Mean filtering takes the average value of the pixels of all pixel points in a region with equal number of m rows and n columns centered on the current pixel. The specific formula is as follows,

[0048]

[0049] where g(s,t) represents the original image, and f(x,y) represents the image obtained after mean filtering. The filtered image is as Figure 3 shown.

[0050] Step 3: Set three gray-scale thresholds of black, dark, and bright for extraction, and divide the terminals into four categories according to the set ratio, namely: already welded, the terminal area is darker, the terminal area is alternately bright and dark, and the terminal area is brighter; the already welded terminals do not need to go through the following steps;

[0051] In step three, due to reasons such as different oxidation degrees of the terminals in the circuit board, some terminals may be too dark and some may be too bright. Accordingly, three threshold ranges are set up to extract the gray-scale regions of the terminals. These three gray-scale ranges are: (8, 15), (15, 43), and (59, 146). Divide the extracted area by the area of the ROI region to obtain three ratios, namely a, b, and c. Since the darker the pixel points, the closer the gray-scale value is to 0, and conversely, the brighter the gray-scale value, the closer it is to 255. Since the average gray-scale value of the terminals that have been soldered does not exceed 15 in the sample picture, it is determined through experiments that when a > 0.4, it is judged that the terminal has been soldered. The remaining terminals are classified by judging the maximum value among a, b, and c into: the terminal area is too dark, the terminal area has alternating light and dark, and the terminal area is too bright. If b is the largest, it is the terminal area with alternating light and dark; if c is the largest, it is the terminal area that is too bright. As Figure 4 shown.

[0052] Step four: Adopt the gray-scale threshold extraction method to perform different threshold extractions on the three types of unsoldered terminals in step three to obtain a binary image;

[0053] In step four, three threshold ranges are respectively set up for the three types of the terminal area being too dark, the terminal area having alternating light and dark, and the terminal area being too bright, which are: (8, 34), (17, 40), and (17, 65). Since there is a gray-scale difference between the terminal part and the interference part outside the terminal, a binary image is obtained through gray-scale feature extraction.

[0054] Step five: Perform dilation processing on the binary image obtained in step four to obtain the dilated binary image;

[0055] Specifically, in step five, the binary image obtained in step four is dilated using a square structuring element with a side length of 90, which is achieved through the following process:

[0056] The dilation operation of the binary image is performed point by point. If the pixel value of this point is 1, no processing is performed on this point. If the pixel value of this point is 0, then all pixel points within the structuring element where this point is located are scanned. If all pixel values of the pixel points within the structuring element are 0, then the pixel value of this point is 0; otherwise, the pixel value is set to 1.

[0057] Step six: Perform region filling processing on the binary image obtained in step five to obtain the filled binary image;

[0058] Specifically, in step six, the binary image obtained in step five is region-filled. In order to connect the holes within the connected region, the seed growth method is used to find all background pixel points for marking. The remaining points with a gray-scale value of 0 are the hole points within the connected region. The pixel values of all pixel points with a gray-scale value of zero that are not background pixel points are assigned 255 to achieve the purpose of filling the image.

[0059] Step 7: For the binary image obtained in Step 6, preliminarily remove the interference regions through the erosion algorithm and the opening operation algorithm to obtain the binary image after preliminary removal.

[0060] Specifically, in Step 7, the binary image obtained in Step 6 is processed by erosion using a square structuring element with a side length of 90, and then processed by opening using a square structuring element with a side length of about 60, which is achieved through the following process:

[0061] The binary image erosion operation is performed point by point. If the pixel value of a point is 0, the point is not processed. If the pixel value of a point is 1, all pixel points within the structuring element where the point is located are scanned. If all pixel values of the pixel points within the structuring element are 1, the pixel value of the point is 1; otherwise, the pixel value is set to 0.

[0062] The opening operation is actually an erosion process followed by a dilation process, which can eliminate the noise outside the image region.

[0063] Step 8: Perform a connected component segmentation process on the binary image obtained in Step 7, and only retain the binary image of the largest terminal part.

[0064] In Step 8, for the binary image obtained in Step 7, a connected component segmentation process is performed. If the area (the number of points in the binary image) is between 10,000 and 100,000, the connected component is determined to be the terminal region, and those with an area less than 10,000 are the surrounding noise interferences.

[0065] Step 9: Perform a rectangular fitting process on the binary image of the terminal part and obtain the center point coordinates of this rectangular binary region.

[0066] In Step 9, for the binary image obtained in Step 8, a rectangular fitting process is performed. The terminal region is close to a standard rectangle. Through the rectangular fitting process, the binary region can be made to coincide with the terminal region, and then the center point coordinates of this binary rectangular image are obtained, as Figure 5 shown.

[0067] Step 10: Convert the pixel coordinates obtained in Step 9 into physical world coordinates through a calibration method.

[0068] Since the central point coordinates of the metal terminals finally obtained by the metal terminal rapid detection and positioning method provided by the embodiments of the present application need to be used for subsequent welding operations, they must be converted into physical world coordinates before being input into the welding robot for the robotic arm to perform welding operations. In the conversion process in Step Ten, the transformation matrix for converting pixel coordinates to world coordinates needs to be calculated first. The calculation method is as follows: Operate the welding head at the end of the robotic arm to weld N points (more than 3) on the calibration board. After each welding is completed, save the coordinates of the end of the robotic arm at this time, that is, obtain N pairs of world coordinates; after N points are welded, take a photo sample through the camera, and process it to obtain N pairs of pixel coordinates in the photo sample; then obtain the transformation matrix for converting pixel coordinates to world coordinates through affine transformation; then multiply this transformation matrix by the pixel coordinates of each terminal to obtain the world coordinates corresponding to all terminals.

[0069] Step Eleven: The world coordinates obtained in Step Ten can be used for subsequent welding. For example, the world coordinates obtained in Step Ten are transmitted to the PLC, and the control host computer offsets the physical position of the welding robotic arm to complete the welding work.

[0070] When the metal terminal rapid detection and positioning method provided by the embodiments of the present application is used, the image grayscales of the welded terminals and the non-welded terminals are used as features to quickly identify the welding situation of the terminals. For non-welded terminals, the candidate area of the terminals is roughly located through the image grayscale extraction algorithm, and then the visual algorithms such as dilation algorithm, erosion algorithm, and opening operation are used to accurately calculate the position of the center point of the terminals. Then, the pixel coordinate system is converted into the physical world coordinate system to perform subsequent welding operations. Due to complex working conditions, different degrees of oxidation of metal terminals, and some terminals that have been welded, it is impossible to use a single visual algorithm to distinguish and locate them. In the traditional process, manual workers judge the welding situation of the terminals according to the design drawings with the naked eye, and then manually control the welding head to perform welding. The present application distinguishes the welded terminals and the non-welded terminals through the difference in image grayscale values. Then, for non-welded terminals, rough positioning is performed through grayscale extraction, and then the metal terminal area is accurately located through visual algorithms such as dilation, erosion, and opening operation to obtain the position of the center point. Then, the pixel coordinates are converted into the physical coordinate system to perform subsequent welding operations. Compared with the manual welding method, this method greatly reduces the time for terminal detection and positioning. It only takes less than 0.15 s to complete the detection and positioning tasks of all terminals on the entire circuit board, and the average accuracy can reach more than 90%. It greatly reduces the manual workload, and this method can realize fully automated process production.

[0071] In summary, for the method of rapid detection and center point positioning of the metal terminal, the image grayscale of the welded terminal and the unwelded terminal is used as a feature to quickly identify the welding situation of the terminal. For the unwelded terminal, the candidate area of the terminal is roughly located through the image grayscale extraction algorithm, and then the center point position of the terminal is accurately calculated through vision algorithms such as the dilation algorithm, erosion algorithm, and opening operation. Subsequently, the pixel coordinate system is converted into the physical world coordinate system to perform subsequent welding operations. Compared with the operation method of manually setting a fixed offset position, a fully automated production process can be realized, greatly improving the efficiency. The production efficiency of a small number of devices can exceed that of a large number of manual workers, meeting the industry requirements at the lowest cost without relying on a large amount of labor costs. The accuracy of automatically setting the position offset is higher, ensuring the production quality of the product.

[0072] Although the embodiments of the present application have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principle and spirit of the present application. The scope of the present application is defined by the appended claims and their equivalents.

Claims

1. A method for rapid detection and positioning of metal terminals, which can distinguish whether the terminals have been welded through one-time detection and positioning and accurately obtain the center position of the metal terminals, is characterized in that It includes the following steps: Step 1: Obtain the circuit board image; Step 2: Extract the ROI region from the image and filter out the surrounding noise through the Mean algorithm; Step 3: Set three grayscale thresholds for extraction, namely black, dark, and bright. Divide the terminals into four categories according to the set ratio, which are: already welded, the terminal area is darker, the terminal area is alternately bright and dark, and the terminal area is brighter; Step 4: Adopt the grayscale threshold extraction method to perform different threshold extractions on the three types of unwelded terminals in Step 3 to obtain a binary image; Step 5: Perform dilation processing on the binary image obtained in Step 4 to obtain the dilated binary image; Step 6: Perform region filling processing on the binary image obtained in Step 5 to obtain the filled binary image; Step 7: For the binary image obtained in Step 6, preliminarily remove the interference area through the erosion algorithm and the opening operation algorithm to obtain the binary image after preliminary removal; Step 8: Perform segmentation and connected component processing on the binary image obtained in Step 7, and only retain the binary image of the largest terminal part; Step 9: Fit the binary image of the terminal part into a rectangle and obtain the center point coordinates of this rectangular binary region; Step 10: Convert the pixel coordinates obtained in Step 9 into physical world coordinates through the calibration method; Step 11: The world coordinates obtained in Step 10 can be used for subsequent welding.

2. The rapid detection and positioning method of a metal terminal according to claim 1, wherein In Step 1, the circuit board is imaged by a CMOS camera and a ring-shaped blue light source to obtain a sample image.

3. A method for rapid detection and positioning of a metal terminal according to claim 1, characterized in that, Step 2 specifically includes: First, extract the rectangular region containing the terminals through ROI region extraction, and then filter out the noise through the Mean algorithm. Taking the current pixel as the center, take the average value of all pixel points within a region where the number of m rows and n columns is equal. The specific formula is as follows, where g(s,t) represents the original image, and f(x,y) represents the image obtained after mean filtering.

4. The rapid detection and positioning method of a metal terminal according to claim 1, characterized in that Step 3 specifically includes: Set three threshold ranges, extract the grayscale region of the terminals, divide the extracted area by the ROI region area to obtain three ratios, namely a, b, and c. When a > 0.4, it is determined that the terminal has been welded. The remaining terminals are classified into: the terminal area is darker, the terminal area is alternately bright and dark, and the terminal area is brighter by judging the maximum value of a, b, and c.

5. A method for rapid detection and positioning of a metal terminal according to claim 1, characterized in that, In Step 5, perform dilation processing on the binary image obtained in Step 4 using a square structuring element with a side length of 90: If the pixel value of this point is 1, do not process this point. If the pixel value of this point is 0, scan all pixel points within the structuring element where this point is located. If all pixel points within the structuring element have a pixel value of 0, the pixel value of this point is 0, otherwise the pixel value is set to 1.

6. The rapid detection and positioning method of a metal terminal according to claim 1, characterized in that, In Step 6, perform region filling on the binary image obtained in Step 5. In order to connect the holes within the connected region, use the seed growth method to find all background pixel points for marking. The remaining points with a grayscale value of 0 are the hole points within the connected region. Assign the pixel value of all pixel points with a grayscale value of zero that are not background pixel points to 255 to achieve the purpose of filling the image.

7. A method for rapid detection and positioning of a metal terminal according to claim 1, characterized in that, In Step 7, the binary image obtained in Step 6 is eroded with a square structuring element with a side length of 90, and then an opening operation is performed with a square structuring element with a side length of about 60. This is achieved through the following process: The erosion operation of the binary image is carried out point by point. If the pixel value of a point is 0, the point is not processed. If the pixel value of a point is 1, all pixel points within the structuring element where the point is located are scanned. If all pixel values of the pixel points within the structuring element are 1, the pixel value of this point is 1; otherwise, the pixel value is set to 0.

8. A method for rapid detection and positioning of a metal terminal according to claim 1, characterized in that, In Step 8, the binary image obtained in Step 7 is processed for segmented connected components. If the area of a connected component is between 10,000 and 100,000, the connected component is determined to be a terminal area, and those with an area less than 10,000 are surrounding noise interferences.

9. A method for rapid detection and positioning of a metal terminal as described in claim 1, characterized in that, In Step 9, the binary image obtained in Step 8 is subjected to rectangular fitting. The terminal area is close to a standard rectangle. Through rectangular fitting, the binary area can be made to coincide with the terminal area, and then the center point coordinates of this binary rectangle image are obtained.

10. A method for rapid detection and positioning of metal terminals according to claim 1, characterized in that, In the conversion process of Step 10, the transformation matrix for converting pixel coordinates to world coordinates needs to be calculated first. The calculation method is as follows: By operating the welding head at the end of the robotic arm to weld N points on the calibration plate, where N is more than 3. After each welding is completed, the coordinates of the end of the robotic arm at this time are saved, thus obtaining N pairs of world coordinates. After N points are welded, a photo sample is obtained by taking a photo with a camera, and N pairs of pixel coordinates in the photo sample are obtained through processing. After that, the transformation matrix for converting pixel coordinates to world coordinates is obtained through affine transformation. Then, this transformation matrix is multiplied by the pixel coordinates of each terminal to obtain the world coordinates corresponding to all terminals.

Images