A method for hand-eye correction of a three-dimensional galvanometer laser marking system and a vision system

Abstract

The application provides a three-dimensional galvanometer laser marking system and a visual system hand-eye correction method. Firstly, the binocular stereo visual imaging principle is used to obtain coordinates in two height plane camera coordinate systems and to fit corresponding light paths, and a rotation matrix R of corresponding point light paths in the camera coordinate system and the laser coordinate system is obtained. Then, a point without distortion when laser emission is found, a common intersection point corresponding to the position of the laser emission point in the camera coordinate system is solved, a translation vector between the laser coordinate system and the camera coordinate system is obtained, and a conversion relationship between the two coordinate systems is obtained, so that on-site correction of the laser and the visual system is completed. The method has the advantages of simple operation process, high correction accuracy, no need of precise equipment for correction, and guaranteed operability and accuracy on site.

Description

Technical Field

[0001] This invention relates to the field of laser marking technology, and in particular to a three-dimensional galvanometer laser marking system and a method for hand-eye correction of a vision system. Background Technology

[0002] Laser marking technology is an emerging processing technology that integrates optics, mechanics, and electronics. It features high marking speed, wide applicability, clear and permanent markings, and a high degree of automation. Laser marking has been widely used in automobile manufacturing, footwear and apparel processing, and electronic component manufacturing. Compared to traditional marking methods, it can significantly save manufacturing and processing costs, and reduce intermediate processes and consumables.

[0003] The biggest benefit of machine vision for laser marking is improved workpiece positioning, especially when the workpiece position is uncertain. It enables the machine to locate the target workpiece for marking, increasing efficiency and reducing error rates. The process of establishing the visual coordinate system of the machine vision system and the laser coordinate system of the galvanometer laser system is called hand-eye calibration. Through calibration, the transformation relationship is obtained, thereby determining the coordinates of any point in the visual coordinate system within the laser coordinate system.

[0004] Vision-guided laser marking systems typically use an image coordinate system as a reference. However, the accuracy of a calibrated vision system can decrease due to vibrations during transportation and processing, or camera equipment may need to be replaced depending on site requirements. In such cases, it is necessary to recalibrate the binocular vision system and the transformation relationship between the vision system and the laser system. Existing calibration methods for 3D galvanometer laser and vision systems include methods that require specific calibration and adjustment tools, which are costly and have requirements for the processing environment. Other methods use laser parallel projection, but existing methods that solve the problem through optical paths are complex and do not consider the focal plane and focal point position during 3D galvanometer marking. Still other methods require multiple markings of complex patterns, demanding high levels of image recognition and feature point extraction, and suffer from computational complexity and inability to guarantee accuracy. Considering the lack of precisely adjustable lifting platforms in industrial processing environments, conveniently and quickly calibrating the transformation relationship between vision and laser, enabling efficient marking of laser marking systems in different working environments with different vision systems, remains a technical challenge. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide a method for hand-eye correction of a three-dimensional galvanometer laser marking system and a vision system. This method is simple to operate and has high correction accuracy. It does not require the use of precision equipment for correction, thus ensuring on-site operability and accuracy.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: a method for hand-eye correction of a three-dimensional galvanometer laser marking system and a vision system, comprising the following steps:

[0007] Step 1: Adjust the camera focus position based on the size and position of the marking area obtained from laser calibration and recalibrate the binocular camera; place the marking paper on the laser focusing plane within the marking area;

[0008] Step 2: Load the pre-made trajectory pattern and mark it with a three-dimensional galvanometer laser device to obtain a clear cross-shaped target pattern on the marking paper;

[0009] Step 3: Use a binocular camera to acquire images of the markings. The three-dimensional coordinates of the center point of the crosshair target in the camera's world coordinate system are obtained using the principle of binocular vision 3D imaging. k = 1, 2, ..., n;

[0010] Step four: Adjust the laser galvanometer to the defocused state, making the laser focusing position dynamically adjustable along the focusing axis. Adjust the focusing position to focus on another plane, reposition the marking paper, and repeat steps two and three to obtain the image. and the three-dimensional coordinates of the center point of the crosshair target in the camera's world coordinate system.

[0011] Step 5: Based on the 3D coordinates of the corresponding marker points in the camera coordinate system obtained from two separate shots taken at different heights, calculate the corresponding direction vector in the camera coordinate system. And normalized to a unit vector

[0012] Step 6: Load the optical path obtained from the camera coordinate system using the hand-eye transformation matrix obtained during laboratory calibration, and transform it to the optical path direction vector in the laser coordinate system.

[0013] Step 7: Let R be the rotation matrix from a point in the camera coordinate system to its corresponding point in the laser coordinate system, and let T = (x, y, z) be the coordinates of the origin of the camera coordinate system in the laser coordinate system; construct the unitized direction vector in the camera coordinate system. With respect to the direction vector in the laser coordinate system Relationship i = 1, 2, ... n, n such markers yield n similar equations, and solving these equations simultaneously with the least squares solution yields the rotation matrix R;

[0014] Step 8: Select m points from the two images where the z-coordinate is larger in the camera coordinate system and the x-coordinates are equal. i = 1, 2, ..., m; based on the direction vector in the visual coordinate system of the corresponding point. Find the ray between corresponding points This is the optical path of the laser emitted in the visual coordinate system; by fitting these n rays, the common intersection point of these rays in the camera's world coordinate system is obtained, and the intersection point P2(x2,y2,z2) is obtained by weighted averaging these points;

[0015] Step 9: Load the common intersection point of the optical path obtained from the original camera coordinate system in the laboratory calibration, and transform it to the common intersection point P1(x1,y1,z1) of the optical path in the laser coordinate system according to the original transformation matrix. According to the calculated rotation matrix R and the correspondence between points P1 and P2, P1=R·P2+T, calculate the corresponding translation vector T, and finally calculate the rigid body transformation matrix [R|T].

[0016] Finally, the recalibrated matrix [R|T] is used to... Transform to the laser coordinate system and re-calibrate to obtain Root mean square error To perform error verification.

[0017] In a preferred embodiment, the marking paper is specifically a smooth paper that can leave clear marking marks under laser action.

[0018] In a preferred embodiment, the pre-made trajectory pattern in step two is a cross-shaped target pattern of appropriate size, which facilitates the extraction of marker points and the generation of three-dimensional coordinates.

[0019] In a preferred embodiment, the direction vector corresponding to the marker point in step five... Take the larger z-coordinate value from the two shots. If the direction of the light path is downward, then the equation of the straight light path is:

[0020] In a preferred embodiment, the relationship between the direction vectors in the camera coordinate system and the laser coordinate system in step seven is... Construct a system of equations When k = 1, 2, ... n, a total of n similar equations are obtained. By solving these n equations simultaneously and using singular value decomposition to find the least squares solution, the rotation matrix R is obtained.

[0021] In a preferred embodiment, in step eight, selecting points near x = 0 can eliminate the influence of distortion correction, allowing the optical paths at these n positions to be focused at a single point. Therefore, the calculated common intersection point is the exit point when x = 0. The exit point is solved using G·M = d, where G is a (k×3)×(k+3) matrix.

[0022]

[0023] M is a column vector of length (k+3):

[0024]

[0025] d is a column vector of length (k×3);

[0026]

[0027] Solving for (x, y, z) in matrix M yields the coordinates P2(x2, y2, z2) of the light-emitting point in the camera's world coordinate system.

[0028] Compared with existing technologies, this invention has the following advantages: In industrial processing environments with limited conditions, it enables rapid hand-eye calibration to obtain highly accurate calibration results, and the root mean square error (RMSE) can be used to verify accuracy. The calibration process considers the influence of the three-dimensional galvanometer focusing position on the calibration results, while avoiding the situation where the light output points are not at the same point due to distortion correction during galvanometer calibration. The light output point at x=0 is fitted from the point in the camera coordinate system corresponding to the point where the x-coordinate is zero in the laser coordinate system. Simultaneously, the height of the galvanometer's light output point from the measured object is obtained to further verify the accuracy of the results.

[0029] Using a crosshair target as the trajectory for generating the marking pattern facilitates the extraction of corner points and the generation of 3D coordinates, avoiding the need for computer calculations based on complex patterns. Because the crosshair target is not easily deformed on an inclined plane, it ensures the accuracy of corner point recognition, reduces the requirements for the application environment, and broadens the range of applications.

[0030] Application: This invention is used in the field of laser marking. In the absence of precision calibration equipment in the industrial field, it performs hand-eye correction on the transformation relationship between laser and binocular vision, so that vision can reguide the laser galvanometer to complete the marking task. Attached Figure Description

[0031] Figure 1 This is a schematic diagram illustrating the principle of the three-dimensional galvanometer laser marking system and the hand-eye rapid on-site correction method of the binocular vision system of the present invention.

[0032] Figure 2 This is a schematic diagram illustrating the principle of laser marking using the three-dimensional galvanometer laser marking system of the present invention.

[0033] Figure 3 This is a flowchart illustrating the steps of the three-dimensional galvanometer laser marking system and the rapid on-site hand-eye calibration method for the binocular vision system of the present invention.

[0034] Figure 4 This is the marking trajectory diagram described in the hand-eye rapid on-site correction method of the three-dimensional galvanometer laser marking system and binocular vision system of the present invention.

[0035] Figure 5 This is a schematic diagram showing the position of the light-emitting point when x=0 in the visual coordinates, which does not cause distortion, for the hand-eye rapid on-site correction method of the three-dimensional galvanometer laser marking system and binocular vision system of the present invention. Detailed Implementation

[0036] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0037] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0038] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations according to this application; as used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise; furthermore, it should be understood that when the terms “comprising” and / or “including” are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0039] A method for hand-eye correction of a three-dimensional galvanometer laser marking system and vision system, referenced Figure 1-5 This includes the following steps:

[0040] Step 1: Adjust the camera focus position according to the size and location of the laser calibration area and recalibrate the camera. Place the marking paper at the laser focus position within the calibration area.

[0041] Step two: Load the pre-made trajectory pattern and mark it with a galvanometer laser to obtain a cross-shaped target pattern of appropriate size and clarity.

[0042] Step 3: Use a binocular camera to acquire images of the markings. The three-dimensional coordinates of the marker point in the visual coordinate system are obtained using the principle of binocular vision imaging. k = 1, 2, ..., n.

[0043] Step four: Adjust the laser galvanometer's z-axis to a defocused state. Without changing the x and y-axis focusing states, make the z-axis focal length independently adjustable. Adjust the galvanometer's z-axis focusing position to focus on a different height plane. Reposition the marking paper and repeat steps two and three to obtain the image. and the three-dimensional coordinates of the marker in the visual coordinate system

[0044] Step 5: Based on the three-dimensional coordinates of the corresponding marker points in the visual coordinate system obtained from two separate shots taken at different heights, calculate the corresponding direction vector in the visual coordinate system. And normalized to a unit vector

[0045] Step 6: Load the optical path obtained during laboratory calibration from the visual coordinate system based on the original hand-eye transformation matrix, and transform it to the optical path direction vector in the laser coordinate system.

[0046] Step 7: Let R be the rotation matrix from a point in the visual coordinate system to its corresponding point in the laser coordinate system, and let T = (x, y, z) be the coordinates of the origin of the visual coordinate system in the laser coordinate system. Construct the unitized direction vector in the camera coordinate system. With respect to the direction vector in the laser coordinate system Relationship i = 1, 2, ... n, n such markers yield n similar equations, and solving these equations simultaneously with the least squares solution yields the rotation matrix R.

[0047] Step 8: Select m points from the two images where the z-coordinate of the image with the larger z-coordinate in the camera coordinate system has the same x-coordinate. i = 1, 2, ..., m. Based on the direction vector in the visual coordinate system of the corresponding point. Find the ray between corresponding points This represents the optical path of the laser beam emitted in the visual coordinate system. The common intersection point P2(x2,y2,z2) of these n rays in the visual coordinate system is obtained by fitting these rays together.

[0048] Step 9: Load the common intersection point of the optical path obtained from the original camera coordinate system in the laboratory calibration, and transform it to the common intersection point P1(x1,y1,z1) of the optical path in the laser coordinate system according to the original transformation matrix. According to the calculated rotation matrix R and the correspondence between points P1 and P2, P1=R·P2+T, calculate the corresponding translation vector T, and finally calculate the transformation matrix [R|T].

[0049] Finally, the recalibrated matrix [R|T] can be used to... Transform to the laser coordinate system and re-calibrate to obtain Square root error To verify the error.

[0050] Specifically, the marking paper is a smooth paper that can leave clear marking marks under the action of a laser.

[0051] Specifically, the pre-made trajectory pattern described in step two is a crosshair target pattern of appropriate size, which facilitates the extraction of marker points and the generation of three-dimensional coordinates. (See attached image) Figure 3 .

[0052] The direction vector corresponding to the marker point in step five Take the larger z-coordinate value from the two shots. If the direction of the light path is downward, then the equation of the straight light path is:

[0053] The relationship between the direction vectors in the camera coordinate system and the laser coordinate system in step seven. Construct a system of equations When k = 1, 2, ... n changes, a total of n similar equations are obtained. By solving these n equations simultaneously and using singular value decomposition, the least squares solution is obtained to obtain the rotation matrix R.

[0054] In step eight, since the laser galvanometer has been calibrated, eliminating the effects of distortion, the fitted optical paths at each point will not intersect at the same point. Therefore, selecting a point near x=0 can eliminate the influence of distortion correction, allowing the optical paths at these n positions to focus on a single point. Thus, the calculated common intersection point, i.e., the exit point, is relatively accurate. The exit point is solved using G·M=d, where G is a (k×3)×(k+3) matrix.

[0055]

[0056] M is a column vector of length (k+3):

[0057]

[0058] d is a column vector of length (k×3).

[0059]

[0060] The (x,y,z) in the obtained matrix M are the points P2(x2,y2,z2) that we are looking for.

[0061] The binocular camera calibration adopts the Zhang Zhengyou calibration method and uses a checkerboard calibration board. The binocular camera calibration parameters are as follows:

[0062] Left camera internal reference:

[0063] Right camera internal parameters:

[0064] Left camera distortion: dist l = [-0.0234-0.00330.0001-0.00021.7077]

[0065] Right camera distortion: dist r = [-0.03086 0.1421 0.0003 0.0004 0.1345]

[0066] The transformation relationship between the left camera coordinate system and the right camera coordinate system is as follows:

[0067]

[0068] The calculated coordinates of the emission point in the laser coordinate system are:

[0069] P (x,y,z) =(8.86618,0.000138174,-0.00337959)

[0070] The transformation relationship between the visual coordinate system and the laser coordinate system obtained through hand-eye calibration is as follows:

[0071]

[0072] The root mean square error (RMSE) calibrated using the method shown in this invention is 0.0167323.

Claims

1. A method for hand-eye correction of a three-dimensional galvanometer laser marking system and a vision system, characterized in that... Includes the following steps: Step 1: Adjust the camera focus position based on the size and position of the marking area obtained from laser calibration, and then recalibrate the binocular camera. Place the marking paper on the laser focusing plane within the marking area; Step 2: Load the pre-made trajectory pattern and mark it with a three-dimensional galvanometer laser device to obtain a clear cross-shaped target pattern on the marking paper; Step 3: Use a binocular camera to acquire the left image obtained from the marking. And the right image The three-dimensional coordinates of the center point of the crosshair target in the camera's world coordinate system are obtained using the principle of binocular vision 3D imaging. ; Step four: Adjust the laser galvanometer to the defocused state, making the laser focusing position dynamically adjustable along the focusing axis. Adjust the focusing position to focus on another plane, reposition the marking paper, and repeat steps two and three to obtain the left image. And the right image and the three-dimensional coordinates of the center point of the crosshair target in the camera's world coordinate system. ; Step 5: Based on the 3D coordinates of the corresponding marker points in the camera coordinate system obtained from two separate shots taken at different heights, calculate the corresponding direction vector in the camera coordinate system. and normalized to a unit vector. ; Step 6: Load the optical path obtained from the camera coordinate system using the hand-eye transformation matrix obtained during laboratory calibration, and transform it to the optical path direction vector in the laser coordinate system. ; Step 7, denote the rotation matrix from a point in the camera's world coordinate system to the corresponding point in the laser coordinate system as: The coordinates of the origin of the camera coordinate system in the laser coordinate system are: Construct the unitized direction vector in the camera coordinate system. With respect to the direction vector in the laser coordinate system Relationship ,Depend on A total of 10 such markers were obtained. A similar set of equations, combined with the least-squares solution, yields the rotation matrix. ; Step 8: Select two images and place them in the camera coordinate system. Points on a graph with larger coordinates Equal coordinates Points Based on the direction vector in the visual coordinate system of the corresponding point. Find the ray between corresponding points This is the optical path of the laser beam emitted in the visual coordinate system; through this The common intersection points of these rays in the camera's world coordinate system are obtained by fitting a set of rays, and the intersection points are calculated by weighted averaging of these points. ; Step nine: Load the common intersection point of the optical path obtained from the original camera coordinate system in the laboratory calibration, and transform it to the common intersection point of the optical path in the laser coordinate system according to the original transformation matrix. Based on the calculated rotation matrix and points Correspondence between Find the corresponding translation vector Finally, the rigid body transformation matrix is ​​obtained. ; Finally, the recalibrated matrix is ​​used. Will Transform to the laser coordinate system and re-calibrate to obtain Root mean square error To perform error verification.

2. The method for hand-eye correction of a three-dimensional galvanometer laser marking system and a vision system according to claim 1, characterized in that, Specifically, marking paper is a smooth piece of paper that can leave clear marking marks under the action of a laser.

3. The method for hand-eye correction of a three-dimensional galvanometer laser marking system and a vision system according to claim 1, characterized in that, The pre-made trajectory pattern mentioned in step two is a cross-shaped target pattern of appropriate size, which facilitates the extraction of marker points and the generation of three-dimensional coordinates.

4. The method for hand-eye correction of a three-dimensional galvanometer laser marking system and a vision system according to claim 1, characterized in that, The direction vector corresponding to the marker point in step five Take two shots The larger coordinate value is If we ensure the direction of the light path is downward, then the equation of the straight light path is: .

5. The method for hand-eye correction of a three-dimensional galvanometer laser marking system and a vision system according to claim 1, characterized in that, The relationship between the direction vectors in the camera coordinate system and the laser coordinate system in step seven. Construct a system of equations ,when At the same time, we received A similar system of equations, combined with this The rotation matrix is ​​obtained by solving the equations using singular value decomposition and least squares. .

6. The method for hand-eye correction of a three-dimensional galvanometer laser marking system and a vision system according to claim 1, characterized in that, In step eight, select Nearby points can eliminate the effects of distortion correction, making this The light paths at each position converge at a single point, therefore the calculated common intersection point is... The light-emitting point at that time; the light-emitting point passes through To solve; where for Matrix of order: It is a length of Column vectors: It is a length of Column vectors; Solving for the matrix yields... In This refers to the coordinates of the light-emitting point in the camera's world coordinate system, which we require. .

Images