Calibration method for position relationship between monocular line laser sensor and two-axis machine tool system

Abstract

The present application discloses a method for calibrating the positional relationship between a monocular line laser sensor and a two-axis machine tool system, comprising the following steps: obtaining the coordinates of the laser sensor and the two-axis machine tool system; exchanging the coordinates of the laser sensor and the two-axis machine tool system; calibrating the positional relationship of the exchanged coordinates between the laser sensor and the two-axis machine tool system to obtain a calibration result. The present application proposes a method for calibrating the positional relationship between a camera and a non-rotating axis displacement mechanism. The calibration process can be achieved with the aid of a calibration plate and an external distance measuring mechanism, such as a total station. After the calibration is completed, the point cloud coordinate system scanned by the sensor can be unified to facilitate subsequent processing.

Description

Technical Field

[0001] The present application belongs to the field of engineering technology, and specifically relates to a method for calibrating the position relationship between a monocular line laser sensor and a two-axis machine tool system. Background Art

[0002] Using a vision system to obtain the three-dimensional coordinates of a point in the camera coordinate system corresponding to pixel coordinates requires obtaining a coordinate transformation matrix and the line laser plane equation. Therefore, the sensor must be calibrated to obtain the camera's intrinsic parameters and the line laser plane equation. Furthermore, the coordinates of each laser line point cloud must be unified into a common coordinate system before the scanned point cloud data can be assembled into a complete 3D scan model for subsequent processing. Therefore, the positional relationship between the sensor system and the machine tool system must be known, requiring calibration of the positional relationship between the two. While the world coordinate system is often artificially defined as a calibration plate, in measurement systems, the world coordinate system is generally defined as the machine tool coordinate system or the robot arm base coordinate system. The transformation between the camera coordinate system and the machine tool coordinate system is a fixed rigid-body transformation. By solving the rigid-body transformation matrix, each laser line point cloud can be transformed from the camera coordinate system to the unified machine tool coordinate system.

[0003] When a monocular line laser sensor is connected to a robotic arm, calibrating its positional relationship is called hand-eye calibration. Based on the calibration plate image and the camera's intrinsic parameters, the relationship between the sensor and the calibration plate is calculated at each position. Combined with the robot's joint information at each position, the robotic arm moves the sensor multiple times to capture the calibration plate image, and the transformation matrix from the sensor to the robot end can be calculated. However, this method assumes that the robot can achieve six degrees of freedom. If each movement of the robot is a translational movement, multiple solutions will occur during the matrix calculation process. Two- or three-axis displacement mechanisms, such as gantry machines, are commonly used in large parts processing plants such as ships and aircraft. These mechanisms cannot rotate the sensor, so hand-eye calibration methods cannot be directly used. This paper presents calibration principles and methods for two-axis displacement mechanisms that are only capable of translation.

[0004] However, in measurement systems, the world coordinate system is typically defined as the machine tool coordinate system or the robot arm base coordinate system. The transformation between the camera coordinate system and the machine tool coordinate system is a fixed rigid-body transformation. By solving the rigid-body transformation matrix, each laser line point cloud can be converted from the camera coordinate system to a unified machine tool coordinate system. This paper proposes a calibration principle and method for two-axis displacement mechanisms that are limited to translation. Summary of the Invention

[0005] This application proposes a method for calibrating the position relationship between a monocular line laser sensor and a two-axis machine tool system. The calibration process can be achieved with the help of a calibration plate and an external distance measuring mechanism such as a total station. After the calibration is completed, the point cloud coordinate system scanned by the sensor can be unified to facilitate subsequent processing.

[0006] To achieve the above objectives, this application provides the following solutions:

[0007] A method for calibrating the position relationship between a monocular line laser sensor and a two-axis machine tool system includes the following steps:

[0008] Obtain the coordinates of the laser sensor and the two-axis machine tool system;

[0009] exchanging coordinates of the laser sensor and the two-axis machine tool system;

[0010] The position relationship of the exchanged coordinates between the laser sensor and the two-axis machine tool system is calibrated to obtain a calibration result.

[0011] Preferably, the method for exchanging the coordinates of the laser sensor and the two-axis machine tool system includes:

[0012] Matrix A is the transformation matrix between the camera coordinate system and the end coordinate system, and matrix B is the transformation matrix between the end coordinate system and the machine tool coordinate system; then P w Points can be represented as P in each coordinate system e =AP c , P w =BP e ; The A and B matrices are both rotation and translation matrices, and the formula is:

[0013]

[0014] Where, Represents the rotation matrix from the camera to the end coordinate system; Represents the translation matrix from the camera to the end coordinate system.

[0015] Preferably, the position relationship calibration method includes: calibration of a rotation matrix and calibration of a translation matrix.

[0016] Preferably, the rotation matrix calibration method includes:

[0017] Calculate the translation direction vector of the displacement mechanism;

[0018] Calculate the camera's movement direction vector;

[0019] A rotation matrix is ​​calculated based on the translation direction vector of the displacement mechanism and the movement direction vector of the camera.

[0020] Preferably, the method for calculating the camera movement direction vector includes: recording the two camera coordinate systems before and after the camera moves as P1 and P2,

[0021]

[0022] Among them, P2′ means point P2 is at O c The corresponding point in the coordinate system, point P represents the fixed coordinate in space, and the vector The unit direction vector representing the camera movement.

[0023] Preferably, based on the unit direction vector of the camera movement, the unit direction vector of the displacement machine after one displacement is set to The fixed point moves in the end coordinate system as After moving, it is recorded in the camera coordinate system as Where k is the mobility coefficient; the formula is as follows:

[0024]

[0025] Where, P w Point is a point in the machine tool coordinate system; P c is the representation of a point in the machine tool coordinate system in the camera coordinate system; P e It is the representation of a point in the machine tool coordinate system in the terminal coordinate system; and The matrix is ​​an unknown matrix and needs to be solved; is the unit direction vector that the displacement machine moves after one displacement; is the unit direction vector of movement in the camera coordinate system; k is the movement coefficient;

[0026] Based on the above formula, we can get:

[0027]

[0028] Where, P c is the representation of a point in the machine coordinate system in the camera coordinate system, Represents the rotation matrix from the camera to the end coordinate system; Represents the translation matrix from the camera to the end coordinate system; is the unit direction vector that the displacement machine moves after one displacement; is the unit direction vector of movement in the camera coordinate system; k is the movement coefficient.

[0029] Preferably, the method for calculating the rotation matrix includes:

[0030]

[0031] in, Represents three sets of linearly independent column vectors of camera directions, Represents three sets of linearly independent column vectors of the displacement mechanism, Represents a rotation matrix.

[0032] Preferably, the translation matrix calibration method includes:

[0033]

[0034] in, Represents the translation matrix from the camera to the end coordinate system, Represents the rotation matrix from the camera to the end coordinate system, Represents the rotation matrix from the end coordinate system to the base coordinate system.

[0035] The beneficial effects of this application are:

[0036] This application discloses a method for calibrating the positional relationship between a monocular line laser sensor and a two-axis machine tool system, and proposes a method for calibrating the positional relationship between a camera and a non-rotating axis displacement mechanism. This calibration process can be performed using a calibration plate and an external distance measurement mechanism, such as a total station. After calibration, the point cloud scanned by the sensor can be aligned to a coordinate system, facilitating subsequent processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solution of the present application, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0038] Figure 1 This is a flow chart of a method for calibrating the position relationship between a monocular line laser sensor and a two-axis machine tool system according to an embodiment of the present application;

[0039] Figure 2 A schematic diagram of a machine tool system according to an embodiment of the present application;

[0040] Figure 3 A schematic diagram of a camera system according to an embodiment of the present application;

[0041] Figure 4 Schematic diagram of the geometric relationship before and after translation of the camera system in an embodiment of the present application;

[0042] Figure 5 This is a schematic diagram of an image of a laboratory calibration plate taken after moving twice according to an embodiment of the present application;

[0043] Figure 6 Schematic diagram of the coordinates of the origin of the calibration plate in the camera coordinate system of an embodiment of the present application. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0045] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0046] like Figure 1 FIG. 1 is a flow chart of a method for calibrating the position relationship between a monocular line laser sensor and a two-axis machine tool system according to the present invention; the method includes the following steps:

[0047] Obtain the coordinates of the laser sensor and the two-axis machine tool system;

[0048] exchanging coordinates of the laser sensor and the two-axis machine tool system;

[0049] The displacement mechanism of the two-axis machine tool system in this embodiment uses a gantry flame cutting machine, which can only move along the X and Y axes. The relationship between the sensor system and the machine tool system is as follows Figure 2 and Figure 3 As shown in the figure, three coordinate systems are established in the measurement system. w -X w Y w Z w The world coordinate system is defined as the coordinate system of the machine tool in the measurement system, and this coordinate system remains fixed; e -X e Y e Z e The end coordinate system is defined on the torch which can move along the X-axis with the machine tool and along the Y-axis of the machine tool beam; c -X c Y c Z c The camera coordinate system is fixed on the flame spray gun through a bracket and moves with the bed. Figure 3 is a simplified schematic diagram of the model, P w Point is a point in the machine tool coordinate system, which is recorded as P in the camera coordinate system. c , recorded as P in the terminal coordinate system e Let matrix A be the transformation matrix between the camera coordinate system and the end coordinate system, and matrix B be the transformation matrix between the end coordinate system and the machine tool coordinate system. Then P w Points can be represented as P in each coordinate system e =AP c, P w =BP e . Both A and B matrices are rotation and translation matrices, so they can be expanded as:

[0050]

[0051]

[0052] Where, Represents the rotation matrix from the camera to the end coordinate system; Represents the rotation matrix from the end coordinate system to the base coordinate system; Represents the translation matrix from the camera to the end coordinate system; Represents the rotation matrix from the end coordinate system to the base coordinate system.

[0053] and The matrix is ​​unknown and needs to be solved by a calibration method similar to hand-eye calibration. Due to the special feature that the cutting machine can only perform axial translation movement, w -X w Y w Z w Coordinate system and O e -X e Y e Z e There is no rotation relationship between coordinate systems, and the axes of the coordinate systems are parallel, so The matrix is ​​a 3×3 unit matrix with all diagonals set to 1; when When the matrix is ​​the identity matrix, The matrix can be obtained through the real-time coordinates of the machine tool system.

[0054] Calibrate the position relationship of the exchanged coordinates between the laser sensor and the two-axis machine tool system to obtain the calibration result;

[0055] Position relationship calibration methods include: calibration of rotation matrix and calibration of translation matrix;

[0056] The calibration methods of the rotation matrix include:

[0057] Rotation matrix calibration essentially involves solving the rotational relationship between two coordinate systems by establishing equations, which rely on constraints. When the displacement mechanism performs a translational motion, the camera also translates. The direction vector of the displacement mechanism's translation can be easily calculated by reading the coordinates before and after the movement. However, the camera's translational direction vector cannot be directly calculated from the machine tool coordinates and must be solved using several camera matrix relationships.

[0058] Calculate the translation direction vector of the displacement mechanism by reading the coordinates before and after the movement.

[0059] Calculate the camera's movement direction vector;

[0060] like Figure 4 As shown, O c and O c ′ are the origins of the camera coordinate systems moving forward and backward respectively. Vector The direction vector of the camera movement needs to be solved. Point P is fixed in space and is recorded as P1 and P2 in the two camera coordinate systems before and after the camera moves. Since P1 and P2 are points in the same coordinate system, we can find the point P2 in O c The corresponding point P2′ in the coordinate system, it is easy to prove the direction vector of the line connecting P1 and P2′ With vector are parallel and in the same direction. That is, vector This is the unit direction vector that the camera moves in.

[0061] The rotation matrix is ​​calculated based on the translation direction vector of the displacement mechanism and the movement direction vector of the camera.

[0062] Assume that the unit direction vector of the displacement machine after one displacement is Therefore, we can get a fixed point in the end coordinate system and record it as Similarly, after moving, the point is recorded in the camera coordinate system as Where k is the mobility coefficient.

[0063] Substitute these two moved points into the formula We can get:

[0064]

[0065] Where, P w Point is a point in the machine tool coordinate system; P c is the representation of a point in the machine tool coordinate system in the camera coordinate system; P e It is the representation of a point in the machine tool coordinate system in the terminal coordinate system; and The matrix is ​​an unknown matrix and needs to be solved; is the unit direction vector that the displacement machine moves after one displacement; is the unit direction vector of movement in the camera coordinate system; k is the movement coefficient.

[0066] Based on the above formula, we can get:

[0067]

[0068] Where, P cis the representation of a point in the machine coordinate system in the camera coordinate system, Represents the rotation matrix from the camera to the end coordinate system; Represents the translation matrix from the camera to the end coordinate system; is the unit direction vector that the displacement machine moves after one displacement; is the unit direction vector of movement in the camera coordinate system; k is the movement coefficient.

[0069] Based on the above formula, we can get the vector and Relationship:

[0070]

[0071] Where, is the unit direction vector that the displacement machine moves after one displacement; is the unit direction vector of movement in the camera coordinate system; Represents the rotation matrix from the camera to the end coordinate system.

[0072] because The matrix is ​​3×3, so three sets of linearly independent column vectors are needed to establish the following matrix equation to obtain the matrix The only solution of .

[0073]

[0074] The three sets of linearly independent column vectors in the above formula can be obtained as follows: first, the displacement mechanism is controlled to move along the X axis to obtain two translation vectors and Then move along the Y axis to get two translation vectors and and is the unit direction vector of the displacement mechanism translation, and is the unit direction vector of the camera translation, which are linearly independent of each other. and and The transformation relationship satisfies the formula. Finally, the third set of linearly independent vectors can be obtained by vector cross product, that is In summary, we get three sets of linearly independent column vectors, which can be used to solve the rotation matrix between the end coordinate system and the camera coordinate system.

[0075] The calibration of the rotation matrix is ​​performed in groups of two translations and three shots, with a total of four groups. The calibration results obtained each time are shown in the table. Figure 5An image of a calibration plate captured for one set of experiments and subjected to dot detection and sorting. Because the laboratory camera's field of view is small and the displacement mechanism can only move a limited distance, the calibration plate in the image moves a relatively short distance.

[0076] The calculation results of the rotation matrix obtained through four calibration experiments are shown in Table 1, where the maximum deviation value is the largest number in the vector obtained by subtracting the calculated value from the average value.

[0077] Table 1

[0078]

[0079] From the statistical data in the table, it can be considered that it is reasonable to use the average value as the rotation matrix to be solved. for:

[0080]

[0081] The calibration methods of the translation matrix include:

[0082] The translation matrix is ​​calibrated using a laser tracker. First, the laser tracker software is used to establish the coordinate system of the displacement platform. The calibration position is then randomly placed multiple times. Each time the calibration plate is placed, the displacement platform is moved so that all points on the calibration plate are within the camera's field of view and can be clearly photographed. After the movement is complete, the current coordinates of the machine tool and the coordinates of the calibration plate's origin in the displacement platform's coordinate system are recorded. Finally, the captured photos are used to calculate the coordinates of the calibration plate's origin in the camera's coordinate system. Using these three coordinates (the calibration plate's origin in the camera's coordinate system, the calibration plate's origin in the machine tool's coordinate system, and the machine tool's current position coordinates), a T matrix can be solved. By placing the calibration plate multiple times, random errors can be reduced.

[0083] Based on the above-mentioned relationship that the machine tool end coordinate system and the machine tool base coordinate system have only a translation relationship but no rotation relationship, the above formula can be obtained:

[0084]

[0085] In the formula, we need is the translation matrix from the camera to the end coordinate system. The rotation matrix has been obtained through the calibration in the previous section. is the translation matrix from the end coordinate system to the base coordinate system. This matrix is ​​the real-time coordinate of the machine tool and can be read out by the controller. Therefore, any point P in the base coordinate system w The three-dimensional coordinates of the point and the three-dimensional coordinates of the point corresponding to the camera coordinate system can be solved by this formula to obtain the translation matrix

[0086] The calibration plate was randomly placed five times, following the calibration process described above. After each placement, photos were collected and calculated to obtain the three coordinates. After the five placements were completed, the coordinate values ​​and the calculated rotation matrix were substituted into the formula to solve the translation matrix. The results are shown in Table 2, where the maximum deviation is the largest number in the vector obtained by subtracting the calculated value from the average value.

[0087] Table 2

[0088]

[0089] It can be seen from the table that the maximum deviation value is less than 0.005, and the average value is taken as the translation vector Calibration results:

[0090]

[0091] To obtain the coordinates of a point in the camera coordinate system, you can refer to the camera calibration model. According to the camera calibration model, when each calibration plate image is obtained, the conversion matrix between the calibration plate coordinate system and the camera coordinate system is the extrinsic parameter matrix. The formula can be obtained as follows:

[0092]

[0093] Where R and t matrices are the rotation and translation extrinsic parameter matrices obtained during camera calibration.

[0094] like Figure 6 As shown, when performing camera calibration, it is usually assumed that the calibration plate coordinate system is the world coordinate system. The first point on the calibration plate is the origin, that is, the point with coordinates (0,0,0). Substituting this point into the formula, we can get [x c y c z c ] = t. Therefore, the translation matrix t of the extrinsic parameter matrix obtained during camera calibration is the coordinate of the first point on the calibration plate in the camera coordinate system. Therefore, as long as the calibration plate is placed within the camera's field of view and is kept stationary, the above method can be used to calculate the coordinates of the next point in the camera coordinate system after each movement, and these points are the same point in the world coordinate system.

[0095] Since the calibration plate is used as the target for translation matrix calibration, external instruments are required to determine the three-dimensional coordinates of the calibration plate's origin in the base coordinate system. The touch method can be used to move the machine tool to the calibration plate's origin, and then use the machine tool's spray gun to touch it. However, this method will damage the calibration plate, resulting in reduced accuracy, and some machine tools do not have a cutting spray gun for touch. Improvements to the touch method are made by fixing the laser pen to the end of the machine tool. During installation, ensure that the laser line projected by the laser pen is perpendicular to the plane of the calibration plate. Control the machine tool's movement so that the laser pen coincides with the center of the calibration plate's origin. At this point, only the machine tool's position information needs to be read to determine the two coordinates x and y of the calibration plate's origin in the base coordinate system. Since the translation machine tool does not have a z coordinate mark and the height in the z direction has no effect on the measurement results, it is possible to consider fixing z to 1 or an arbitrary number.

[0096] The embodiments described above are merely descriptions of the preferred embodiments of the present application and do not limit the scope of the present application. Without departing from the design spirit of the present application, various modifications and improvements made to the technical solutions of the present application by ordinary technicians in this field should fall within the scope of protection determined by the claims of the present application.

Claims

1. A method for calibrating the position relationship between a monocular line laser sensor and a two-axis machine tool system, characterized in that: The following steps are involved: Obtain the coordinates of the laser sensor and the two-axis machine tool system; exchanging coordinates of the laser sensor and the two-axis machine tool system; Calibrate the positional relationship of the exchanged coordinates between the laser sensor and the two-axis machine tool system to obtain a calibration result; The position relationship calibration method includes: calibration of a rotation matrix and calibration of a translation matrix; The rotation matrix calibration method includes: Calculate the translation direction vector of the displacement mechanism; Calculate the camera's movement direction vector; Calculating a rotation matrix based on the translation direction vector of the displacement mechanism and the movement direction vector of the camera; The camera's moving direction vector is calculated as follows: Point P represents a fixed coordinate in space, which is recorded as and ,make ,vector Represents the unit direction vector of the camera movement, where express Point The corresponding points in the coordinate system are: is the origin of the camera coordinate system before movement; The rotation matrix is ​​calculated as follows: Based on the unit direction vector of the camera movement, the unit direction vector of the displacement mechanism after one displacement is , the fixed point moves in the end coordinate system and is recorded as , after moving, it is recorded in the camera coordinate system as , where k is the mobility coefficient; the formula is as follows: , where It is the representation of a point in the machine coordinate system in the camera coordinate system; It is the representation of a point in the machine tool coordinate system in the terminal coordinate system; and The matrices are the rotation matrix from the camera to the end coordinate system and the translation matrix from the camera to the end coordinate system; The following transformation relationships exist between coordinate systems: ; After simplifying the above formula, we get the vector and The relationship between: ; because is a 3×3 matrix, so the matrix equation is established by three sets of linearly independent column vectors to obtain the only solution; The calibration of the translation matrix includes: ,in, Represents the rotation matrix from the end coordinate system to the base coordinate system.

2. The method for calibrating the position relationship between a monocular line laser sensor and a two-axis machine tool system according to claim 1, characterized in that: The method for exchanging the coordinates of the laser sensor and the two-axis machine tool system includes: Matrix A is the transformation matrix between the camera coordinate system and the end coordinate system, and matrix B is the transformation matrix between the end coordinate system and the machine tool coordinate system; then Points are represented in various coordinate systems as , ; The A and B matrices are both rotation and translation matrices, and the formula is: Where, Represents the rotation matrix from the camera to the end coordinate system; Represents the translation matrix from the camera to the end coordinate system.

3. The method for calibrating the position relationship between a monocular line laser sensor and a two-axis machine tool system according to claim 1, characterized in that: Methods for calculating the rotation matrix include: in, , , Represents three sets of linearly independent column vectors of camera directions, , , Represents three sets of linearly independent column vectors of the displacement mechanism, Represents a rotation matrix.

Images