PCB welding method and system based on visual positioning

Abstract

The present invention discloses a PCB welding method based on visual positioning, comprising: collecting an image of a current PCB chip to be identified, matching the collected image with each rough positioning template, and matching the image coordinates of the center of the PCB chip to be identified in the collected image; constructing a sample image based on the center of the PCB chip to be identified in the collected image, inputting the sample image into a recognition network, and identifying the category of the PCB chip to be identified in the collected image; determining a precise positioning area in the collected image based on the image coordinates of the center of the PCB chip to be identified, and determining the image coordinates of the welding point of the precise positioning template in the current collected image based on the matching between the precise positioning area and the precise positioning template; converting the image coordinates of the welding point in the collected image to the robot coordinate system, and controlling the solder gun at the end of the robot to move to the welding point position for welding operation. The method realizes the autonomous identification and automatic welding of PCB chips by industrial robots, and can be applied to the mixed-line production of various types of PCB chip products.

Description

Technical Field

[0001] The present invention belongs to the technical field of visual positioning, and more specifically, the present invention relates to a PCB welding method and system based on visual positioning. Background Art

[0002] PCB board welding technology is widely used in the 3C industry in the industrial field. Using machine vision and industrial robots to replace manual chip welding can not only improve production efficiency and reduce production costs, but also prevent operators from being exposed to harmful gases generated during the soldering process, and better protect the personal safety of practitioners. In the production process, there is often a need to weld chips of similar shapes but different types. In this case, traditional machine vision positioning methods cannot identify and distinguish them well. Therefore, it is of great significance to study the welding system of various types of chips of industrial robots combined with deep learning.

[0003] Deep learning can supervise the effective features of targets in the learning image and use more comprehensive feature information to describe and locate targets, which makes this type of algorithm have better generalization performance than traditional positioning algorithms. Since there are usually many parts and circuits on the surface of PCB chips, the existing visual positioning methods mainly rely on single template matching technology. When used for PCB chip recognition and positioning, there are too many features in the image due to the excessive number of parts on the workpiece surface of the PCB chip and the complex circuits. The interference of too many features will reduce the positioning accuracy of the matching algorithm. Summary of the invention

[0004] The present invention provides a PCB welding method based on vision positioning, aiming to improve the above-mentioned problem.

[0005] The present invention is implemented in this way: a PCB welding method based on visual positioning, the method specifically comprises the following steps:

[0006] S1, collecting an image of the current PCB chip to be identified, matching the collected image with each coarse positioning template, and matching the image coordinates of the center of the PCB chip to be identified in the collected image;

[0007] S2. Construct a sample image based on the center of the PCB chip to be identified in the collected image. The sample image covers the surface circuit loop of the entire PCB chip. Input the sample image into the recognition network to identify the category of the PCB chip to be identified in the collected image.

[0008] S3, determining the precise positioning area in the acquired image based on the image coordinates of the center of the PCB chip to be identified, and determining the image coordinates of the welding point of the precise positioning template in the current acquired image based on the matching between the precise positioning area and the precise positioning template;

[0009] S4, converting the image coordinates of the welding point in the acquired image into the robot coordinate system, and controlling the soldering gun at the end of the robot to move to the welding point position to perform welding operations.

[0010] Furthermore, before step S1, the method further includes:

[0011] S5. Collect images of various types of PCB chips, called template images, build coarse positioning templates and fine positioning templates of various types of PCB chips based on the template images, determine the image coordinates of the center of the coarse positioning template and the center of the fine positioning template, and the image coordinates of the corresponding PCB chip welding points.

[0012] Furthermore, the coarse positioning template is based on a feature area formed in the center area of ​​the workpiece, and the fine positioning template is a feature area formed on the circuit loop on the surface of the chip.

[0013] Furthermore, the matching method of the collected image and the coarse positioning template is as follows:

[0014] Construct a sliding window of the same size as the coarse positioning template, form a to-be-matched area on the acquired image based on the sliding window, move the sliding window, put the formed to-be-matched area into the to-be-matched area set, calculate the matching degree of all to-be-matched areas in the to-be-matched area set with each coarse matching template, output the coarse matching template with the highest matching degree and its corresponding to-be-matched area, and the image coordinates of the output to-be-matched area center are the image coordinates of the center of the PCB chip to be identified in the acquired image.

[0015] Furthermore, the image coordinates of the welding points of the precise positioning template in the current captured image are obtained as follows:

[0016] In the precise positioning area, the point p is extracted as the gradient intensity greater than the intensity threshold. i , put it into the set P, and extract the position point q whose gradient intensity is greater than the intensity threshold in the precise positioning template i , put it into the set Q, iteratively calculate the rotation matrix R and translation vector T from the image coordinates of the acquired image to the image coordinates of the precise positioning template, and use the rotation matrix and translation vector corresponding to the minimum matching error E as the rotation matrix R and translation vector T from the acquired image to the precise positioning template image, and then transform the image coordinates of the welding point in the template image to the acquired image.

[0017] Furthermore, the sample image is input into multiple recognition networks, each recognition network outputs the PCB chip category corresponding to the sample image, and a voting mechanism is used to determine the category of the PCB chip to be identified in the captured image.

[0018] The present invention is implemented in this way: a PCB welding system based on visual positioning, the system comprising:

[0019] The workstation on the workbench is used to place PCB chips. The industrial camera is fixed directly above the workstation. The industrial camera is used to collect images of the PCB chips to be identified and send the collected images to the industrial computer. The industrial computer is connected to the SCARA robot via TCP / IP. A soldering gun is provided at the end of the actuator of the SCARA robot.

[0020] The template library integrated on the industrial computer is used to store the coarse positioning templates and fine positioning templates of various PCB chips based on the template images, and determine the relative positions of the center of the coarse positioning template and the center of the fine positioning template, the center of the fine positioning template and the corresponding PCB chip welding points;

[0021] The industrial computer receives the image of the PCB chip to be identified, determines the welding point position of the current PCB chip to be identified based on the above-mentioned PCB welding method based on visual positioning, and sends it to the SCARA robot. The SCARA robot controls the soldering gun to move to the welding point for welding operation.

[0022] The present invention realizes autonomous identification and automatic welding of PCB chips by industrial robots without manual intervention, and can be applied to mixed-line production of various types of PCB chip products. Through the coordination of coarse positioning and fine positioning, the positioning accuracy of PCB chip welding points is greatly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A flow chart of a PCB welding method based on human visual positioning provided by an embodiment of the present invention;

[0024] Figure 2 A schematic diagram of the structure of a PCB welding system based on visual positioning provided by an embodiment of the present invention;

[0025] Figure 3 A schematic diagram of a PCB chip category recognition process based on a ResNet recognition network, a DenseNet recognition network, and a VOVNet recognition network provided in an embodiment of the present invention;

[0026] Figure 4 A schematic diagram of a PCB chip with a rectangular center provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0027] The specific implementation modes of the present invention are further explained in detail below by describing the embodiments with reference to the accompanying drawings, so as to help those skilled in the art to have a more complete, accurate and in-depth understanding of the inventive concept and technical solution of the present invention.

[0028] The present invention proposes a PCB welding method based on visual positioning, which is based on a voting judgment mechanism of multiple deep learning detection networks to realize the judgment of the type of operating chip. On this basis, combined with a model matching algorithm based on gradient information, the visual positioning of the key position of the target workpiece is realized.

[0029] Figure 1 A flow chart of a PCB welding method based on human visual positioning provided by an embodiment of the present invention, the method specifically comprises the following steps:

[0030] S1, calibrate the transformation relationship M between the robot coordinate system and the visual coordinate system;

[0031] In an embodiment of the present invention, a method for obtaining the conversion relationship between the robot coordinate system and the visual coordinate system is specifically as follows:

[0032] S11, obtaining the internal and external parameters of the camera through camera calibration;

[0033] S12. After the camera is calibrated, the visual system needs to be calibrated to eliminate the mirror distortion and tangential distortion of the industrial camera lens;

[0034] S13. Use the multi-point method to establish the transformation relationship M between the visual coordinate system and the robot coordinate system. The operation process is as follows:

[0035] A checkerboard calibration plate is fixedly placed on the workstation, and the industrial camera collects the image of the checkerboard calibration plate, and records the coordinates of each corner point on the checkerboard calibration plate in the visual coordinate system (image coordinate system). The robot is controlled to move the welding gun to each corner point on the checkerboard calibration plate in turn, and the coordinates of each corner point in the robot coordinate system are recorded. The calibration of the transformation relationship M can be completed based on the following formula;

[0036]

[0037] Among them, (jx, jy) is the coordinate of the corner point in the chessboard calibration plate in the robot coordinate system, and (px, py) is the coordinate of the corner point in the chessboard calibration plate in the visual coordinate system. The conversion relationship between the two is calculated

[0038] In an embodiment of the present invention, in order to ensure the calibration accuracy, the invention proposes a least squares support vector machine (abbreviated as: GWO-LS-SVM) optimized by the gray wolf algorithm to establish the conversion relationship between the visual coordinate system and the robot coordinate system. GWO-LS-SVM is an intelligent optimization algorithm for multiple inputs and multiple outputs. It uses the least squares-support vector machine (LS-SVM) to establish the conversion relationship between the two-dimensional coordinate points in the visual coordinate system and the three-dimensional coordinate points in the robot coordinate system. At the same time, the gray wolf algorithm (GWO) is used to optimize the kernel function of the LS-SVM. The least squares support vector machine (LSSVM) is an improvement on the SVM algorithm. It changes its constraint conditions from inequality constraints to equality constraints. This improvement reduces the algorithm processing time. For the parameter σ in the RBF kernel function that affects the performance of LSSVM, the present invention uses the GWO algorithm to optimize it, avoiding the problem of manually selecting parameters.

[0039] S2, collecting images of various PCB chips, called template images, constructing coarse positioning templates and fine positioning templates of various PCB chips based on the template images, determining the image coordinates of the center of the coarse positioning template and the center of the fine positioning template, and the image coordinates of the welding points of the corresponding PCB chips;

[0040] In the embodiment of the present invention, the coarse positioning template is a feature area formed based on the center area of ​​the workpiece, and the fine positioning template is a feature area formed on the circuit loop on the surface of the chip; for the feature area of ​​the coarse positioning template, if the center of the PCB chip to be identified is a rectangle, then the feature area of ​​the coarse positioning template is a rectangle, such as Figure 4 As shown in the figure, if the center of the PCB chip to be identified is a circle, then the feature area of ​​the coarse positioning template is a circle. If the center of the PCB chip to be identified has both a circle and a rectangle, then the feature area of ​​the coarse positioning template is a rectangle and a circle. For the feature area of ​​the fine positioning template, an area with obvious features on the circuit loop on the chip surface is selected as the feature area.

[0041] S3, collecting an image of the current PCB chip to be identified, matching the collected image with each coarse positioning template, and matching the coordinates of the center of the PCB chip to be identified in the collected image in the image coordinate system, referred to as image coordinates;

[0042] In the embodiment of the present invention, the method for matching the captured image with the coarse positioning template is as follows:

[0043] Construct a sliding window of the same size as the coarse positioning template, form a to-be-matched area on the captured image based on the sliding window, move the sliding window, put the formed to-be-matched area into the to-be-matched area set, calculate the matching degree of all to-be-matched areas in the to-be-matched area set with each coarse matching template, output the coarse matching template with the highest matching degree and its corresponding to-be-matched area, and the image coordinates of the center of the output to-be-matched area are the image coordinates of the center of the PCB chip to be identified in the captured image;

[0044] In the embodiment of the present invention, the to-be-matched area I j With the rough matching template O i The matching degree ε(O i ,I j ) The calculation formula is as follows:

[0045]

[0046] Among them, ori(O i ,r k ) represents the rough matching template O i The gradient amplitude of position r in the gradient direction k, the set G stores the positions where the gradient amplitude (in one of the gradient directions) in the rough matching template is greater than the amplitude threshold, ori(I j ,t k ) represents the area to be matched I j The gradient strength of position t in the gradient direction k, R(t) represents the area to be matched I j The T neighborhood of position t, Represents a square area centered at position t, where the length of the square is T.

[0047] S4. Construct a sample image based on the center of the PCB chip to be identified in the collected image. The sample image covers the surface circuit loop of the entire PCB chip. Input the sample image into multiple recognition networks. Each recognition network outputs the PCB chip category corresponding to the sample image. A voting mechanism is used to determine the category of the PCB chip to be identified in the collected image.

[0048] In the embodiment of the present invention, the surface circuit loop of the PCB chip contains more feature information. Therefore, the size of the sample image is required to cover the surface circuit loop of the entire PCB chip. The sample image is input into the ResNet recognition network, the DenseNet recognition network and the VOVNet recognition network respectively. Of course, the ResNet recognition network, the DenseNet recognition network and the VOVNet recognition network are trained before use. Different weights are assigned to these three recognition networks, and then the scores of each recognition result can be calculated. The recognition result (chip category) with the highest score is the category of the PCB chip to be recognized in the current collected image, such as Figure 3 shown.

[0049] S5, determining the precise positioning area in the acquired image based on the image coordinates of the center of the PCB chip to be identified, and determining the image coordinates of the welding point of the precise positioning template in the current acquired image based on the matching between the precise positioning area and the precise positioning template;

[0050] In the embodiment of the present invention, the image coordinates of the welding points of the precise positioning template in the current collected image are obtained as follows:

[0051] Extract the position point p where the gradient intensity is greater than the intensity threshold in the precise positioning area i , put it into the set P, and extract the position point q whose gradient intensity is greater than the intensity threshold in the precise positioning template image i , put it into the set Q, and iteratively calculate the rotation matrix R and translation vector T from the image coordinates of the acquired image to the image coordinates of the precise positioning template. The calculation formula is as follows;

[0052]

[0053] The rotation matrix and translation vector corresponding to the minimum matching error E are used as the rotation matrix R and translation vector T from the image coordinates of the acquired image to the image coordinates of the precise positioning template, thereby converting the image coordinates of the welding point in the template image to the acquired image.

[0054] S6. Convert the image coordinates of the welding point in the acquired image to the robot coordinate system, and control the soldering gun at the end of the robot to move to the welding point to perform welding operations.

[0055] Figure 2 The schematic diagram of the structure of the PCB welding system based on visual positioning provided by the embodiment of the present invention only shows the part related to the embodiment of the present invention for the convenience of explanation. The system includes:

[0056] The workstation on the workbench is used to place PCB chips. The industrial camera is fixed directly above the workstation. The industrial camera is used to collect images of the PCB chips to be identified and send the collected images to the industrial computer. The industrial computer is connected to the SCARA robot via TCP / IP. A soldering gun is provided at the end of the actuator of the SCARA robot.

[0057] The template library integrated on the industrial computer is used to store the coarse positioning templates and fine positioning templates of various PCB chips based on the template images, and determine the relative positions of the center of the coarse positioning template and the center of the fine positioning template, the center of the fine positioning template and the corresponding PCB chip welding points;

[0058] The industrial computer receives the image of the PCB chip to be identified. The PCB welding method based on visual positioning determines the welding point position of the current PCB chip to be identified and sends it to the SCARA robot. The SCARA robot controls the soldering gun to move to the welding point for welding operation.

[0059] The present invention has been described exemplarily. Obviously, the specific implementation of the present invention is not limited to the above-mentioned method. As long as various non-substantial improvements are made using the method concept and technical solution of the present invention, or the concept and technical solution of the present invention are directly applied to other occasions without improvement, they are all within the protection scope of the present invention.

Claims

1. A PCB welding method based on visual positioning, characterized in that: The method specifically comprises the following steps: S1, collecting an image of the current PCB chip to be identified, matching the collected image with each coarse positioning template, and matching the image coordinates of the center of the PCB chip to be identified in the collected image; S2. Construct a sample image based on the center of the PCB chip to be identified in the collected image. The sample image covers the surface circuit loop of the entire PCB chip. Input the sample image into the recognition network to identify the category of the PCB chip to be identified in the collected image. S3, determining the precise positioning area in the acquired image based on the image coordinates of the center of the PCB chip to be identified, and determining the image coordinates of the welding point of the precise positioning template in the current acquired image based on the matching between the precise positioning area and the precise positioning template; S4, converting the image coordinates of the welding point in the collected image into the robot coordinate system, and controlling the soldering gun at the end of the robot to move to the welding point position to perform welding operation; The matching method between the collected image and the rough positioning template is as follows: Construct a sliding window of the same size as the coarse positioning template, form a to-be-matched area on the captured image based on the sliding window, move the sliding window, put the formed to-be-matched area into the to-be-matched area set, calculate the matching degree of all to-be-matched areas in the to-be-matched area set with each coarse matching template, output the coarse matching template with the highest matching degree and its corresponding to-be-matched area, and the image coordinates of the center of the output to-be-matched area are the image coordinates of the center of the PCB chip to be identified in the captured image; Area to be matched I j With the rough matching template O i The matching degree ε(O i ,I j ) The calculation formula is as follows: Among them, ori(O i ,r k ) represents the rough matching template O i The gradient amplitude of position r in the gradient direction k, the set G stores the positions where the gradient amplitude in the rough matching template is greater than the amplitude threshold, ori(I j ,t k ) represents the area to be matched I j The gradient strength of position t in the gradient direction k, R(t) represents the area to be matched I j The T neighborhood of position t, It represents a square area centered at position t, and the length of the square is T; The image coordinates of the welding points of the precise positioning template in the current acquired image are obtained as follows: In the precise positioning area, the point p is extracted as the gradient intensity greater than the intensity threshold. i , put it into the set P, and extract the position point q whose gradient intensity is greater than the intensity threshold in the precise positioning template i , put it into the set Q, iteratively calculate the rotation matrix R and translation vector T from the image coordinates of the acquired image to the image coordinates of the precise positioning template, and use the rotation matrix and translation vector corresponding to the minimum matching error E as the rotation matrix R and translation vector T from the acquired image to the precise positioning template image, and then transform the image coordinates of the welding point in the template image to the acquired image; Among them, the sample images are respectively input into the ResNet recognition network, the DenseNet recognition network and the VOVNet recognition network. The ResNet recognition network, the DenseNet recognition network and the VOVNet recognition network are trained before use. Different weights are assigned to the trained ResNet recognition network, the DenseNet recognition network and the VOVNet recognition network, and the scores of each recognition result are calculated. The recognition result with the highest score is used as the category of the PCB chip to be identified in the current collected image.

2. The PCB welding method based on visual positioning as claimed in claim 1, characterized in that: Before step S1, the method further includes: S5. Collect images of various types of PCB chips, called template images, build coarse positioning templates and fine positioning templates of various types of PCB chips based on the template images, determine the image coordinates of the center of the coarse positioning template and the center of the fine positioning template, and the image coordinates of the corresponding PCB chip welding points.

3. The PCB welding method based on visual positioning as claimed in claim 1 or 2, characterized in that: The coarse positioning template is based on the feature area formed in the center area of ​​the workpiece, and the fine positioning template is the feature area formed on the circuit loop on the surface of the chip.

4. The PCB welding method based on visual positioning as claimed in claim 1, characterized in that: The sample image is input into multiple recognition networks, each recognition network outputs the PCB chip category corresponding to the sample image, and a voting mechanism is used to determine the category of the PCB chip to be identified in the captured image.

5. A PCB welding system based on visual positioning, characterized in that: The system comprises: A workstation on the workbench is used to place PCB chips. An industrial camera is fixed directly above the workstation. The industrial camera is used to collect images of the PCB chips to be identified and send the collected images to an industrial computer. The industrial computer is connected to the SCARA robot for communication. A soldering gun is provided at the end of the actuator of the SCARA robot. The template library integrated on the industrial computer is used to store the coarse positioning templates and fine positioning templates of various PCB chips based on the template images, and determine the relative positions of the center of the coarse positioning template and the center of the fine positioning template, the center of the fine positioning template and the corresponding PCB chip welding points; The industrial computer receives the image of the PCB chip to be identified, determines the welding point position of the current PCB chip to be identified based on the PCB welding method based on visual positioning described in any one of claims 1 to 4, and sends it to the SCARA robot. The SCARA robot controls the soldering gun to move to the welding point to perform the welding operation.

Images