A dual-camera high-speed flying alignment and lamination visual application method

Abstract

The present invention provides a dual-camera high-speed flying alignment and fitting visual application method, including: using an upper camera and a machine to perform nine-point calibration, and using a lower camera and an upper camera to perform nine-point calibration to convert the upper camera visual coordinates and the lower camera visual coordinates of the workpiece into mechanical coordinates respectively; according to the position of the lower camera during modeling and the rotation angle of the workpiece, the rotation center of the suction nozzle when sucking the workpiece is calculated; according to the actual position of the workpiece and the calculated rotation center, the actual target moving position of the manipulator can be calculated according to the point-around-point rotation formula. The present invention improves the detection speed and fitting accuracy, and can be compatible with a fast speed, the fastest of which can reach 800mm / s, and the distance between the equipment nozzles can be reduced to 28mm; the fitting accuracy is high, the pure visual accuracy can reach ±0.03mm, and the overall accuracy of the equipment can reach ±0.05mm; the detection speed is fast, and the processing time of a single-board product (a total of 8 images need to be processed by the upper and lower cameras) does not exceed 1s.

Description

Technical Field

[0001] The present invention relates to the field of visual inspection technology, and more specifically, to a dual-camera high-speed flying alignment and bonding visual application method. Background Art

[0002] Visual alignment technology is a technology that matches perceived images with pre-established models. It can analyze information such as feature points, edges, and colors in the image and compare it with known models to achieve target object recognition, positioning, and tracking. This technology is widely used in various fields, including industrial automation, robot navigation, medical imaging, augmented reality, and virtual reality.

[0003] In industrial automation, visual alignment technology can be used for product quality inspection, automatic positioning and assembly of parts, etc. In robot navigation, it can help robots locate and navigate in complex environments. In medical imaging, visual alignment technology can be used for surgical navigation and tumor positioning, etc. In augmented reality and virtual reality applications, it can achieve the alignment and interaction of virtual objects with the real environment. Visual alignment technology can provide precise image analysis and recognition capabilities, providing more possibilities and convenience for applications in various fields.

[0004] In the production workshop, high-precision electronic components produced on the workshop production line are bonded and assembled. In order to ensure the quality of the electronic components produced, high bonding accuracy is often required. However, it is generally difficult for the human eye to achieve mm-level errors, so high-precision bonding equipment came into being; in order to improve the production rhythm, flying shooting has become the best choice; however, the machine has to bond at a fixed position each time, and the accuracy of the tooling material and the suction nozzle is very high, but the accuracy of the tooling material and the suction nozzle cannot be guaranteed, so the production line personnel need to adjust the mechanical bonding position at any time; the yield is low, and raw materials are lost; the degree of automation is low, and manual intervention is required; the efficiency of low-speed flying shooting bonding equipment is low, and the equipment output rate cannot meet the requirements; in order to allow more time for the camera to take pictures, the distance between the suction nozzles will be a big problem. Summary of the invention

[0005] The purpose of the present invention is to provide a dual-camera high-speed flying shooting alignment and fitting visual application method.

[0006] The present invention aims to solve the problems of low efficiency, low yield and waste of raw materials in the traditional method.

[0007] Compared with the prior art, the technical solution of the present invention and its beneficial effects are as follows:

[0008] A dual-camera high-speed flying alignment and fitting vision application method comprises: using an upper camera and a machine to perform nine-point calibration, and using a lower camera and an upper camera to perform nine-point calibration to convert the upper camera visual coordinates and the lower camera visual coordinates of the workpiece into mechanical coordinates respectively; according to the position of the lower camera during modeling and the rotation angle of the workpiece, the rotation center of the suction nozzle when sucking the workpiece is rotated is calculated; according to the actual position of the workpiece and the calculated rotation center, the actual target moving position of the manipulator can be calculated according to the point-around-point rotation formula.

[0009] The beneficial effects of the present invention are:

[0010] The present invention improves the detection speed and fitting accuracy. It can be compatible with a fast speed, the fastest of which can reach 800mm / s, and the distance between the equipment nozzles can be reduced to 28mm; the fitting accuracy is high, the pure visual accuracy can reach ±0.03mm, and the overall equipment accuracy can reach ±0.05mm; the detection speed is fast, and the processing time of a single-board product (the upper and lower cameras need to process a total of 8 images) does not exceed 1s. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 It is a schematic diagram of a dual-camera high-speed flying shot alignment and bonding visual application method provided by an embodiment of the present invention.

[0012] Figure 2 This is a diagram of the lighting effect of the upper camera provided by an embodiment of the present invention.

[0013] Figure 3 This is a diagram of the lighting effect of the lower camera provided by an embodiment of the present invention.

[0014] Figure 4 It is a schematic diagram of a visual program running interface provided by an embodiment of the present invention.

[0015] Figure 5 Schematic diagram of a calibration image of an upper camera provided in an embodiment of the present invention.

[0016] Figure 6 It is a first schematic diagram of a lower camera calibration image provided by an embodiment of the present invention.

[0017] Figure 7 This is a second schematic diagram of a lower camera calibration image provided by an embodiment of the present invention.

[0018] Figure 8 This is a third schematic diagram of a lower camera calibration image provided by an embodiment of the present invention.

[0019] Fig. 9 This is a fourth schematic diagram of a lower camera calibration image provided by an embodiment of the present invention.

[0020] Fig.10 It is a schematic diagram of the position change of the workpiece and vision in the visual guidance provided by an embodiment of the present invention.

[0021] Fig.11 It is a schematic diagram of point-around-point rotation provided by an embodiment of the present invention.

[0022] Fig.12 It is a schematic diagram of a dual-camera high-speed flying shooting alignment and bonding visual device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents the selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0024] In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0025] Reference Figures 1 to 12 As shown, a dual-camera high-speed flying alignment and fitting vision application method includes: using an upper camera and a machine to perform nine-point calibration, and using a lower camera and an upper camera to perform nine-point calibration to convert the upper camera visual coordinates and the lower camera visual coordinates of the workpiece into mechanical coordinates respectively; according to the position of the lower camera during modeling and the rotation angle of the workpiece, the rotation center of the suction nozzle when it rotates while sucking the workpiece is calculated; according to the actual position of the workpiece and the calculated rotation center, the actual target moving position of the manipulator can be calculated according to the point-around-point rotation formula.

[0026] Reference Fig.12 As shown, in the device, the following operation steps are included:

[0027] In the first step, PLC controls four suction nozzles 1 to go to the material taking position to take materials;

[0028] In the second step, the PLC sends four trigger signals to the upper camera 2 and the lower camera 3 of the flying camera respectively. The upper camera 2 and the lower camera 3 of the flying camera respectively take four pictures and save them into the memory;

[0029] In the third step, after the image is taken, the visual program calls the visual detection algorithm for calculation;

[0030] In the fourth step, the visual program merges the calculation results of the upper camera 2 and the lower camera 3 and sends the results to the PLC via TCP / IP;

[0031] In the fifth step, PLC controls the equipment to perform product bonding based on the results sent by vision.

[0032] The method of using the upper camera and the machine to perform nine-point calibration, and using the lower camera and the upper camera to perform nine-point calibration to convert the upper camera visual coordinates and the lower camera visual coordinates of the workpiece into the machine coordinates respectively, includes:

[0033] Mechanical coordinate system = rotation coefficient * image coordinate + displacement coefficient, specifically: ;

[0034] Expands to: ;

[0035] Transformed into the following three-variable linear equation ;

[0036] It can be expressed as ;

[0037] as well as ;

[0038] It can be expressed as .

[0039] According to the above formula, at least 3 points are needed to calculate the coefficients of the matrix transformation, but in order to improve accuracy and reduce errors, 9 points are generally selected for calculation.

[0040] The effect of this step in this embodiment is as follows (the four positions only have offsets in the X and Y directions, so the four positions can use exactly the same data):

[0041] The calibration data used are as follows, and a total of eight sets of data are used for calibration:

[0042]

[0043] Reference Figure 5 As shown, the light blue points represent uncorrected points, the dark red points represent the original corrected points, and the calibration error is 0.38 pixels.

[0044] Reference Figure 6 As shown, the calibration data of the nine-point calibration of the lower camera is as follows:

[0045] The first set of data

[0046]

[0047] Reference Figure 7 As shown,

[0048] The second set of data

[0049]

[0050] Reference Figure 8 As shown,

[0051] The third set of data

[0052]

[0053] Reference Fig. 9 As shown,

[0054] The fourth set of data

[0055]

[0056] The light blue points represent uncorrected points, the dark red points represent the original corrected points, and the calibration errors of the four groups are 0.26, 1.23, 1.12, and 0.54 pixels, respectively.

[0057] Reference Fig.10 As shown in the figure, it can be seen that the changes of the workpiece are mainly divided into two types, one is that the workpiece only undergoes translation, and the other is that the workpiece undergoes both translation and rotation. In practical applications, in most cases, the workpiece undergoes both translation and rotation.

[0058] The CCD in the figure refers to the camera, FOV refers to the field of view of the camera, and the Mark point refers to the feature point on the workpiece that can be used as a positioning feature (the change of the feature point is the change of the workpiece).

[0059] The method further comprises: calculating the rotation center of the suction nozzle when sucking the workpiece and rotating the workpiece according to the position of the lower camera during modeling and the rotation angle of the workpiece;

[0060] The actual position of the workpiece is translated and rotated relative to the standard position. According to the difference in coordinates and the rotation angle, the compensation value of the manipulator of the actual position of the workpiece of the upper camera and the lower camera relative to the standard position is calculated respectively. The calculation formula of the compensation value of the manipulator is: ;

[0061] Among them, CDx is the compensation value in the x direction, and Cdy is the compensation value in the y direction;

[0062] The calculation formulas for StDx and StDy in the above formula are as follows: ;

[0063] Wherein, α is the rotation angle of the workpiece, Mdx is the difference between the x-coordinate of the actual position of the workpiece and the standard position, and Mdy is the difference between the y-coordinate of the actual position of the workpiece and the standard position.

[0064] Reference Fig.11 As shown, according to the actual position of the workpiece and the calculated rotation center, the actual target moving position of the manipulator can be calculated according to the point-around-point rotation formula, where the point-around-point rotation formula is: ;

[0065] Among them, (x, y) represents the coordinates of the rotation starting point A; (rx0, ry0) represents the coordinates of the rotation center point B; (x0, y0) represents the coordinates of the rotation end point C.

[0066] The above embodiments are only used to explain the technical solutions of the present invention rather than to limit them. Those skilled in the art should understand that any modification and equivalent substitution that does not depart from the spirit and scope of the present invention should fall within the protection scope of the claims of the present invention.

Claims

1. A dual-camera high-speed flying alignment and lamination visual application method, characterized in that: include: The upper camera and the manipulator are calibrated at nine points, and the lower camera and the upper camera are calibrated at nine points to convert the upper camera visual coordinates and the lower camera visual coordinates of the workpiece into mechanical coordinates respectively; According to the position of the lower camera during modeling and the rotation angle of the workpiece, the rotation center of the nozzle when sucking the workpiece is calculated; According to the actual position of the workpiece and the calculated rotation center, the actual target moving position of the manipulator can be calculated according to the point-around-point rotation formula; The method of calculating the rotation center of the suction nozzle when the suction nozzle sucks the workpiece and rotating it according to the position of the lower camera during modeling and the rotation angle of the workpiece also includes: The actual position of the workpiece is translated and rotated relative to the standard position. According to the difference in coordinates and the rotation angle, the compensation value of the manipulator of the actual position of the workpiece of the upper camera and the lower camera relative to the standard position is calculated respectively. The calculation formula of the compensation value of the manipulator is: CDx=(COS(α)-1)*StDx-Sin(α)*StDy+MDx CDy=(COS(α)-1)*StDy+Sin(α)*StDx+MDy Among them, CDx is the compensation value in the x direction, and CDy is the compensation value in the y direction; The calculation formulas for StDx and StDy in the above formula are as follows: StDx=-0.5*(MDx*(cos(α)-1)+MDy*sin(α)) / (1-cos(α)) StDy=0.5*(MDx*sin(α)-MDy*(cos(α)-1)) / (1-cos(α)) where α is the rotation angle of the workpiece, MDx is the difference between the x-coordinate of the actual position of the workpiece and the standard position, and MDy is the difference between the y-coordinate of the actual position of the workpiece and the standard position; According to the actual position of the workpiece and the calculated rotation center, the actual target moving position of the manipulator can be calculated according to the point-around-point rotation formula, wherein the point-around-point rotation formula is: x0=cos(α)*(x-rx0)-sin(α)*(y-ry0)+rx0 y0=cos(α)*(y-ry0)+sin(α)*(x-rx0)+ry0 Among them, (x, y) represents the coordinates of the rotation starting point A; (rx0, ry0) represents the coordinates of the rotation center point B; (x0, y0) represents the coordinates of the rotation end point C.

2. The dual-camera high-speed flying shooting alignment and lamination visual application method according to claim 1, characterized in that: The method of using the upper camera and the manipulator to perform nine-point calibration, and using the lower camera and the upper camera to perform nine-point calibration to convert the upper camera visual coordinates and the lower camera visual coordinates of the workpiece into mechanical coordinates respectively, includes: Mechanical coordinate system = rotation coefficient * image coordinate + displacement coefficient, specifically: Expands to: X′=aX+bY+c Y′=a′X+b′Y+c′ Transformed into the following three-variable linear equation X0′=aX0+bY0+c X1′=aX1+bY1+c X2′=aX2+bY2+c It can be expressed as as well as Y0′=a′X0+b′Y0+c′ Y1′=a′X1+b′Y1+c′ Y2′=a′X2+b′Y2+c′ It can be expressed as Among them, a, b, c, a', b', c' are the conversion coefficients from image coordinates to mechanical coordinate system.

Images