Calibration method, calibration device and calibration system for external parameters of line laser sensor

Abstract

The invention discloses an external parameter calibration method, calibration device and calibration system for a line laser sensor, and the method comprises the steps: A, calculating a rotation matrix corresponding to a yaw angle, a rotation matrix corresponding to a pitch angle, and a rotation matrix corresponding to a roll angle based on a target image collected by a camera from the surface of a vertical calibration plate, calculating a first combined rotation matrix through the rotation matrix corresponding to the yaw angle, the rotation matrix corresponding to the pitch angle and the rotation matrix corresponding to the roll angle; b, performing coordinate transformation on a line laser point set formed by transmitting the line laser transmitter to the horizontal calibration plate based on the first combined rotation matrix to obtain a first horizontal calibration point set, and calculating an optimized roll angle by using the first horizontal calibration point set; and step C, performing coordinate transformation on the first horizontal calibration point set by using the optimized roll angle to obtain a second horizontal calibration point set, and marking the average height of the second horizontal calibration point set as the calibration external parameter of the line laser sensor.

Description

Technical Field

[0001] The present application relates to the field of calibration of line laser sensors, and in particular to an external parameter calibration method, a calibration device and a calibration system for line laser sensors. Background Art

[0002] In the prior art, the commonly used method for calibrating line laser sensors is to process the coordinate matching relationship between the robot coordinate system and the laser sensor coordinate system, and solve the rotation parameters and offset of the line laser sensor relative to the robot. In this calibration process, it is necessary to continuously acquire multiple sets of target images, and when determining the target coordinates, it is necessary to equip sensors or level tools other than the line laser sensor to provide positioning information, resulting in the disadvantages of large amount of calculation and large influence of platform vibration and acceleration on the measurement data; or, in this calibration process, only the line laser point is used to calculate the assembly angle of the line laser sensor, and the internal calibration error of the line laser sensor will introduce interference factors. Summary of the invention

[0003] The present application discloses an external parameter calibration method, a calibration device and a calibration system for a line laser sensor. The specific technical scheme is as follows:

[0004] A method for calibrating an external parameter of a line laser sensor, wherein the line laser sensor is fixedly mounted on a robot, and comprises a camera and a line laser transmitter; a vertical calibration plate and a horizontal calibration plate are vertically arranged outside the robot, wherein a target is arranged on the surface of the vertical calibration plate, and no target is arranged on the surface of the horizontal calibration plate; the vertical calibration plate and the horizontal calibration plate are both mounted within the detectable range of the line laser sensor; the method for calibrating an external parameter comprises: step A, based on a target image collected by a camera from the surface of the vertical calibration plate, calculating a rotation matrix corresponding to a yaw angle, a rotation matrix corresponding to a pitch angle, and a rotation matrix corresponding to a roll angle, Then calculate the first combined rotation matrix through the rotation matrix corresponding to the yaw angle, the rotation matrix corresponding to the pitch angle and the rotation matrix corresponding to the roll angle; step B, based on the first combined rotation matrix described in step A, perform coordinate transformation on the line laser point set formed by the line laser transmitter emitting to the horizontal calibration plate to obtain a first horizontal calibration point set, and then use the first horizontal calibration point set to calculate the optimized roll angle; step C, use the optimized roll angle described in step B to perform coordinate transformation on the first horizontal calibration point set to obtain a second horizontal calibration point set, and then mark the average height of the second horizontal calibration point set as the calibration external parameter of the line laser sensor.

[0005] Compared with the prior art, the present application adopts a vertical calibration plate with a target and a horizontal calibration plate without a target to collect a line laser point set, and on this basis, the target image is used to calculate a first combined rotation matrix formed by combining the pitch angle, the yaw angle, and the roll angle, and the first combined rotation matrix is ​​used to perform coordinate transformation on the line laser point set formed by the horizontal calibration plate and then optimize the roll angle accordingly, which is equivalent to performing four rotation operations around the axis on the line laser point set formed by the horizontal calibration plate, providing a roll angle reference for the horizontal correction of the line laser sensor, so as to achieve the horizontal calibration plate formed by the horizontal calibration plate. The effect of horizontal plane correction by line laser point set is achieved, and the second horizontal calibration point set obtained by rotating the line laser point set formed by the horizontal calibration plate four times around the axis is used to calculate the calibration height of the line laser sensor, so as to use the optimized roll angle to perform installation height correction of the line laser sensor to ensure the reliability of external parameter calibration. There is no need to make the target setting too complicated, and the production requirements for the calibration plate are reduced. There is no need to continuously collect multiple groups of line laser point sets, nor is there a need to continuously collect multiple groups of target images. The calibration process is simplified as a whole, and the efficiency of external parameter calibration of the line laser sensor is improved.

[0006] Furthermore, the step C also includes: controlling the rotation matrix corresponding to the optimized roll angle to be multiplied by the first combined rotation matrix on the left, the product obtained is the second combined rotation matrix, and then marking the second combined rotation matrix as the calibration external parameter of the line laser sensor, so that the calibration height of the line laser sensor and the second combined rotation matrix constitute the calibration result of the line laser sensor by the external parameter calibration method; the rotation matrix corresponding to the optimized roll angle is configured as the rotation matrix required for coordinate transformation of the first horizontal calibration point set. Therefore, the technical solution obtains the rotation matrix corresponding to the optimized roll angle according to the line laser point set of the horizontal calibration plate and the first combined rotation matrix calculated from the target image, and then combines it into a second combined rotation matrix, thereby realizing the use of the second combined rotation matrix to transform the coordinates of the line laser point set of the horizontal calibration plate into the second horizontal calibration point set as the final calibrated horizontal plane point set, so that the final calibrated horizontal plane point set is close to the ideal calibration value.

[0007] On the other hand, the rotation matrix corresponding to the optimized roll angle required for the coordinate transformation of step C in the external parameter calibration method is multiplied by the first combined rotation matrix to obtain a second combined rotation matrix, and then the pixel point set in the target image captured by the camera is corrected by using the second combined rotation matrix. The normal vector of the target image can be corrected to the same direction as the horizontal plane. In this embodiment, without using relevant correction instruments such as a level, there is no need to limit the horizontal direction of the target on the surface of the vertical calibration plate, which makes the target production and use of the vertical calibration plate more convenient.

[0008] The present application also discloses a calibration device, which includes a calibration plate control module, a sensor fixing module, a horizontal calibration plate with no target set on the surface, a vertical calibration plate with a target set on the surface, a point set acquisition module, and an external parameter calculation module; the sensor fixing module is used to fix the position of the line laser sensor; the calibration plate control module is used to set the position of the horizontal calibration plate and the position of the vertical calibration plate to control the position of the horizontal calibration plate and the position of the vertical calibration plate to be within the detection range of the line laser sensor; the point set acquisition module is used to acquire the line laser point set formed by the horizontal calibration plate being emitted by the line laser sensor; the point set acquisition module is also used to acquire the target image on the surface of the vertical calibration plate; the external parameter calculation module is used to calculate the calibration external parameters of the line laser sensor using the target image and the line laser point set acquired by the point set acquisition module according to the external parameter calibration method.

[0009] In summary, the calibration device uses a vertical calibration plate with a target and a horizontal calibration plate without a target to collect a line laser point set, and on this basis, uses the target image to calculate a first combined rotation matrix formed by combining a rotation matrix corresponding to the pitch angle, a rotation matrix corresponding to the yaw angle, and a rotation matrix corresponding to the roll angle, and uses the first combined rotation matrix to perform coordinate transformation on the line laser point set formed by the horizontal calibration plate and then optimize the roll angle accordingly, providing a roll angle reference for the horizontal correction of the line laser sensor, so as to achieve the effect of using the line laser point set formed by the horizontal calibration plate to perform horizontal plane correction, and also calculates the calibration height of the line laser sensor based on the optimized roll angle, taking into account the installation height correction of the line laser sensor, thereby ensuring the reliability of the external parameter calibration, without the need to overly complicate the target setting, reducing the requirements for the production of the calibration plate, and not needing to continuously collect multiple sets of line laser point sets, nor to continuously collect multiple sets of target images, thereby simplifying the calibration process as a whole and improving the external parameter calibration efficiency of the line laser sensor.

[0010] The present application also discloses a calibration system, which includes a robot and an external parameter calibration device, wherein the external parameter calibration device includes a memory and a processor; the memory has a wired laser external parameter calibration program, and the processor executes the external parameter calibration method when running the wire laser external parameter calibration program, and can obtain a first combined rotation matrix according to the angle information calculated from steps A to C, and construct a constraint relationship between the target coordinate system and the camera coordinate system, which also corresponds to the constraint relationship between the straight line direction vector of the laser point set on the horizontal plane scanned by the wire laser sensor and the coordinate axis of the camera coordinate system, and obtain the optimized roll angle and its corresponding rotation matrix, and then a second combined rotation matrix can be combined, which is equivalent to performing a coordinate transformation on the wire laser point set output by the wire laser sensor to the horizontal calibration plate through the first combined rotation matrix and the second combined rotation matrix, so that the final point set result is close to the ideal calibration value, and the overall calibration process of the wire laser sensor is completed. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 It is a flowchart of an external parameter calibration method of a line laser sensor disclosed in an embodiment of the present application.

[0012] Figure 2 It is a schematic diagram of a line laser sensor emitting a line laser toward a horizontal calibration plate according to an embodiment of the present application.

[0013] Figure 3 It is a schematic diagram of an embodiment of the present application that discloses a line laser sensor collecting targets (black and white cells) on the surface of a vertical calibration plate.

[0014] Figure 4 It is a flow chart of a method for calculating an optimized roll angle disclosed in one embodiment of the present application.

[0015] Figure 5 It is a schematic diagram of the module distribution of a calibration device disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0016] The technical solutions in the embodiments of the present invention will be described in detail below in conjunction with the drawings in the embodiments of the present invention. To further illustrate the embodiments, the present invention provides drawings. These drawings are part of the disclosure of the present invention, which are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these contents, ordinary technicians in this field should be able to understand other possible implementations and advantages of the present invention.

[0017] As an embodiment, a method for calibrating the external parameters of a line laser sensor is disclosed, which is used to calibrate and calculate the external parameters of the line laser sensor; the control subject of the external parameter calibration method can be a controller used to calibrate and calculate the external parameters of the line laser sensor, and the controller can be installed inside or outside the robot. The line laser sensor is fixedly installed on the robot, wherein the line laser sensor includes a camera and a line laser transmitter, the line laser transmitter is used to emit a line laser to the outside and reflect it on the surface of the object to be measured to form a laser point set, the laser point set can be regarded as being located in the intersection line or near the intersection line between the emitted laser plane and the surface of the object to be measured, the camera inside the line laser sensor can be an infrared camera, the infrared camera is used to collect the line laser point set, and can also obtain image data containing line laser light strip information, the line laser point set is obtained by calculating the internal calibration parameters of the line laser sensor, and it is selected according to actual needs without limitation. The line laser point set disclosed in the present application is a description form of a laser point cloud or a laser point set.

[0018] In this embodiment, there are a vertical calibration plate and a horizontal calibration plate arranged vertically on the outside of the robot; a target is arranged on the surface of the vertical calibration plate, and the target image can be captured by the camera, generally imaged as a black and white rectangular pattern; no target is arranged on the horizontal calibration plate, and a solid color calibration plate pattern can be captured by the camera. In this embodiment, a horizontal solid color flat plate without a pattern can be used for line laser data collection, and a vertical flat plate with a pattern can be used for target posture information collection.

[0019] The vertical calibration plate and the horizontal calibration plate are both installed within the detectable range of the camera. Specifically, the line laser emitter is configured to emit to the horizontal calibration plate to form a line laser point set. The line laser point set formed by the horizontal calibration plate is collected by the camera but does not carry target image information. The line laser points formed by the line laser emitter are located in a laser plane. In addition, the line laser emitter is also configured not to emit line laser to the vertical calibration plate. Only the target image is collected by the camera without the laser point. At this time, the robot can be located on the ground parallel to the horizontal calibration plate and the line laser emitter can emit oblique laser to the horizontal calibration plate. There is no restriction on whether the horizontal calibration plate and the vertical calibration plate need to be moved. The line laser point set is as follows: Figure 2 The coordinates of each corner point of the target are expressed in the camera coordinate system XYZ as shown in Figure 3 The camera coordinate system XYZ shown in the figure all becomes the coordinate system data of the camera. In summary, the calibration environment is constructed for the external parameter calibration method of the line laser sensor.

[0020] In this embodiment, if Figure 1 As shown, the external parameter calibration method includes the following steps:

[0021] Step A: Based on the target image collected by the camera from the surface of the vertical calibration plate, the rotation matrix corresponding to the yaw angle, the rotation matrix corresponding to the pitch angle and the rotation matrix corresponding to the roll angle are calculated, and the specific values ​​of the yaw angle, the pitch angle and the roll angle can be calculated at the same time; then the first combined rotation matrix is ​​calculated by the rotation matrix corresponding to the yaw angle, the rotation matrix corresponding to the pitch angle and the rotation matrix corresponding to the roll angle, so as to construct a combined rotation matrix from the rotation parameters of the three degrees of freedom of the yaw angle, the pitch angle and the roll angle in the camera coordinate system.

[0022] The values ​​of the yaw angle, pitch angle and roll angle are generally limited to angle values ​​between 0 and 90 degrees.

[0023] The target can be a checkerboard, an asymmetric dot target, etc., which can be selected according to the needs without limitation. Generally, the appropriate target size is selected according to the camera lens parameters (focal length, inner and outer diameters of the lens, lens thickness, mirror curvature, etc.) adopted by the line laser sensor. The target is attached to the surface of the vertical calibration plate. Preferably, the normal of the plane where the target is located is parallel to the optical axis of the camera, so that the target is facing the camera, which facilitates the clear imaging of the target image in the camera.

[0024] It should be noted that, from a bird's-eye view, the plane where the target is located can be considered perpendicular to the robot's body axis. When calibrating the position of the line laser relative to the robot, the target should be set as a reference in the direction of the robot's body axis. In actual use, the body is clamped by a fixture so that the body axis is perpendicular to the vertical calibration plate.

[0025] Among them, the chessboard can be found in Figure 3 The black and white cells arranged on the surface of the vertical calibration plate are within the detection range of the line laser sensor. From the two dotted lines presented from the longitudinal perspective, the line laser sensor can capture the complete pattern of the target.

[0026] Preferably, the camera is equipped with an infrared filter, and a fill light device can be added to fill light for the target when acquiring the target image, so as to facilitate subsequent image processing.

[0027] Step B, based on the first combined rotation matrix described in step A, coordinate transformation is performed on the line laser point set formed by the line laser transmitter emitting to the horizontal calibration plate to obtain a first horizontal calibration point set, and then the optimized roll angle is calculated using the first horizontal calibration point set; the roll angle calculated using the target image in the aforementioned step A and its corresponding rotation matrix may not necessarily be the required calibration angle, so it is necessary to calculate the roll angle again using the line laser point set on the horizontal calibration plate to approach the desired roll angle and achieve the purpose of optimizing the roll angle.

[0028] In some embodiments, the prerequisite for obtaining the first horizontal calibration point set in step B is to place the robot flat on a horizontal plane or a horizontal calibration board, obtain the line laser point set emitted by the line laser transmitter to the horizontal calibration board and confirm that it is a valid line laser point set, then obtain the initial laser point set, and then control the first combined rotation matrix to multiply the initial line laser point set on the left to obtain the first horizontal calibration point set. Then, the first horizontal calibration point set is used to fit and calculate the optimized roll angle, specifically, the angle formed by the projection vector of the first horizontal calibration point set in the plane perpendicular to the horizontal forward direction of the robot and the width direction of the robot's body is calculated, and then the calculated result is marked as the optimized roll angle. The optimized roll angle represents the roll angle deviation formed by the direction vector of the line laser point set in the horizontal calibration board relative to the width direction of the robot's body.

[0029] It should be noted that the aforementioned yaw angle, pitch angle, and roll angle constitute the assembly angle information of the line laser sensor, and there is a constraint relationship or a corresponding projection relationship between the coordinate systems between the target formed by the corresponding rotation matrix and the robot. Therefore, by executing step B, this embodiment uses the pure target image information on the surface of the vertical calibration plate without line laser information to calculate the aforementioned optimized roll angle, and obtains the optimized line laser sensor assembly angle parameters, thereby overcoming the disadvantage of introducing interference factors due to the calibration error of the line laser sensor internal parameters in the method of calculating the assembly angle using only the line laser point set.

[0030] Step C, use the optimized roll angle described in step B to perform coordinate transformation on the first horizontal calibration point set to obtain a second horizontal calibration point set. Specifically, the first horizontal calibration point set is transformed into the second horizontal calibration point set by left-multiplying the rotation matrix corresponding to the optimized roll angle as the final calibrated horizontal plane point set, and then the average height of the second horizontal calibration point set is marked as the calibration height of the line laser sensor, and the calibration height of the line laser sensor is marked as the calibration external parameter of the line laser sensor, thereby overcoming the influence of the assembly angle error corresponding to the three degrees of freedom of yaw angle, pitch angle, and roll angle, so as to be close to the expected calibration result, and also allow the line laser point set on the surface of the horizontal calibration plate to play a real horizontal correction role.

[0031] Compared with the prior art, this embodiment uses a vertical calibration plate with a target and a horizontal calibration plate without a target to collect a line laser point set, and on this basis, the target image is used to calculate a first combined rotation matrix formed by combining the pitch angle, the yaw angle, and the roll angle, and the first combined rotation matrix is ​​used to perform coordinate transformation on the line laser point set formed by the horizontal calibration plate and then optimize the roll angle accordingly, which is equivalent to performing four rotation operations around the axis on the line laser point set formed by the horizontal calibration plate, providing a roll angle reference for the horizontal correction of the line laser sensor, so as to achieve the result formed by the horizontal calibration plate. The effect of horizontal plane correction by line laser point set is achieved, and the second horizontal calibration point set obtained by rotating the line laser point set formed by the horizontal calibration plate four times around the axis is used to calculate the calibration height of the line laser sensor, so as to use the optimized roll angle to perform installation height correction of the line laser sensor to ensure the reliability of external parameter calibration. There is no need to make the target setting too complicated, and the production requirements for the calibration plate are reduced. There is no need to continuously collect multiple groups of line laser point sets, nor is there a need to continuously collect multiple groups of target images. The calibration process is simplified as a whole, and the efficiency of external parameter calibration of the line laser sensor is improved.

[0032] In the above embodiment, the step C also includes controlling the rotation matrix corresponding to the optimized roll angle to be multiplied by the first combined rotation matrix on the left, and the product obtained is the second combined rotation matrix, and then marking the second combined rotation matrix as the calibration external parameter of the line laser sensor, so that the calibration height of the line laser sensor and the second combined rotation matrix constitute the calibration result of the line laser sensor by the external parameter calibration method, wherein, step C configures the rotation matrix corresponding to the optimized roll angle as the rotation matrix required for coordinate transformation of the first horizontal calibration point set, which has the same meaning as the rotation parameter generated by the roll angle, and is the optimization of the rotation parameter in the camera coordinate system after conversion from the target coordinate system to the camera coordinate system. Therefore, this embodiment obtains the rotation matrix corresponding to the optimized roll angle according to the line laser point set of the horizontal calibration plate and the first combined rotation matrix calculated from the target image, and then combines it into a second combined rotation matrix, thereby realizing the use of the second combined rotation matrix to transform the coordinates of the line laser point set of the horizontal calibration plate into the second horizontal calibration point set as the final calibrated horizontal plane point set, so that the final calibrated horizontal plane point set is close to the ideal calibration value.

[0033] On the other hand, the rotation matrix corresponding to the optimized roll angle required for the coordinate transformation of step C in the external parameter calibration method is multiplied by the first combined rotation matrix to obtain a second combined rotation matrix, and then the pixel point set in the target image captured by the camera is corrected by using the second combined rotation matrix. The normal vector of the target image can be corrected to the same direction as the horizontal plane. In this embodiment, without using relevant correction instruments such as a level, there is no need to limit the horizontal direction of the target on the surface of the vertical calibration plate, which makes the target production and use of the vertical calibration plate more convenient.

[0034] Preferably, the rotation parameter corresponding to one degree of freedom can be expressed by a 3×3 orthogonal rotation matrix; at the same time, the yaw angle can be recorded as the yaw angle of the line laser sensor, the pitch angle can be recorded as the pitch angle of the line laser sensor, and the roll angle can be recorded as the roll angle of the line laser sensor, thereby forming the calibration external parameters of the line laser sensor to realize the calibration of the assembly angle of the line laser sensor.

[0035] It should be noted that the laser line emitted by the line laser transmitter is structured light. For example, line laser can be regarded as a laser plane composed of countless laser lines emitted by the same line laser transmitter, corresponding to Figure 2 The light plane is composed of multiple dotted lines shown in the figure. The intersection of the laser plane and the calibration plate forms a light strip, such as a projection spot of a straight line laser in the calibration plate. It can be extracted from the calibration plate image captured by the camera to perform triangulation operation (construct similar triangles for distance calculation). Similarly, Figure 3 The corner positions of each cell in the target shown can be extracted from the target image captured by the camera to perform triangulation operation.

[0036] As an embodiment, before executing step A, the line laser transmitter does not emit a line laser, the optical axis of the camera approximately points to the target of the vertical calibration plate, the target of the vertical calibration plate is within the detection range of the line laser sensor, the camera captures the target image, obtains the edge and corner information in the target image, the edge and corner information includes the edge and corner positions of each cell, each vertex of the cell, and can be expressed in the target coordinate system, and the camera coordinate system can be used to calculate the relative pose of the target after the coordinate system conversion, and the target image does not contain light strip information formed by the reflection of the line laser; therefore, in combination with the aforementioned embodiment, the pure target image information without line laser information on the surface of the vertical marking plate is used to calculate the corresponding assembly angle parameters on the relevant degrees of freedom of the line laser sensor, which can overcome the interference factors introduced by the camera intrinsic calibration error of the line laser sensor in the prior art method of calculating the assembly angle using only line laser point sets.

[0037] Before executing step B, the robot is fixed on a horizontal plane, the line laser transmitter emits a line laser toward the horizontal calibration plate, the line laser transmitter set inside the line laser sensor emits a line laser toward the surface of the horizontal calibration plate, the line laser is reflected on the measured surface of the horizontal calibration plate to form laser points, forming a line laser point set, the camera collects the line laser point set formed by the line laser reflecting on the surface of the horizontal calibration plate, thereby obtaining the line laser point set formed by the line laser transmitter emitting to the horizontal calibration plate, providing the line laser point set required for assembly angle error calibration for the coordinate transformation required to be performed in step B.

[0038] It should be noted that there is no restriction on the order in which the camera captures the target image and the camera captures the line laser point set formed by the surface reflection of the horizontal calibration plate. Specifically, the line laser emitter emits to the horizontal calibration plate to form a line laser point set after the camera captures the target image from the surface of the vertical calibration plate, or it can be selected to be executed before the camera captures the target image from the surface of the vertical calibration plate. However, the line laser point set formed in step B and the first horizontal calibration point set do not include the pose information of the target image, the information collected by the camera from the vertical calibration plate and the information collected from the horizontal calibration plate are unrelated, and the information sources of the two calibration plate surfaces are different.

[0039] In this embodiment, the horizontal calibration plate is parallel to the horizontal plane; in the line laser sensor, a camera coordinate system is set in the camera, and the line laser point set is located under the camera coordinate system. The line laser point set is located on the surface of the horizontal calibration plate, and the laser points form a scanning line on the measured surface of the horizontal calibration plate to enclose a laser plane, or the line laser is reflected on the measured surface of the horizontal calibration plate to form a laser plane, and the line laser point set is located in the laser plane. Figure 2 The horizontal calibration plate is shown parallel to the coordinate plane XOZ.

[0040] In some calibration scenarios, the camera of the line laser sensor preferably uses a small-angle camera to meet the pinhole imaging principle, and there is a certain angle between the forward emission direction of the line laser transmitter and the optical axis of the camera to form the detection range, and in the optical path system, the line laser is mostly oblique relative to the surface of the incident object. When the robot is fixed or placed on a horizontal plane, the angle between the forward emission direction of the line laser transmitter and the optical axis of the camera is fixed.

[0041] In the above embodiments, combined with Figure 2 and Figure 3 It can be seen that in the camera coordinate system XYZ, the coordinate axis of the normal vector parallel to the surface of the vertical calibration plate is set as the Z axis; wherein the positive direction of the Z axis represents the horizontal forward direction of the robot, so as to point to the front of the line laser transmitter.

[0042] In the camera coordinate system XYZ, the coordinate axis parallel to the width direction of the robot's body is set as the X-axis; wherein the horizontal forward direction of the robot is parallel to the normal vector of the vertical calibration plate, and the width direction of the robot's body is perpendicular to the horizontal forward direction of the robot; the width direction of the robot's body uses the direction in which the robot's wheel axle crosses the body.

[0043] In the camera coordinate system XYZ, the plane formed by the intersection of the X axis and the Z axis is set as the XOZ coordinate plane, and then the XOZ coordinate plane is set as the robot's horizontal coverage plane, where the robot's horizontal coverage plane is parallel to the horizontal plane and the surface of the horizontal calibration plate. If the robot needs to move, the XOZ coordinate plane is configured as the robot's walking plane, and the positive direction of the Z axis represents the robot's walking direction and the detection direction of the line laser sensor.

[0044] In the camera coordinate system XYZ, set the coordinate axis perpendicular to the XOZ coordinate plane as the Y axis, and set the plane formed by the intersection of the X axis and the Y axis as the XOY coordinate plane.

[0045] Among them, the YOZ coordinate plane, the XOZ coordinate plane and the XOY coordinate plane intersect each other perpendicularly at the origin O of the camera coordinate system XYZ.

[0046] In summary, the coordinate axes and coordinate planes of the camera coordinate system in various directions are constructed through the horizontal forward direction of the robot, the width direction of the robot body, the horizontal plane, and the vertical calibration plate, thereby providing coordinate parameter and angle parameter measurement standards for the coordinate transformation and fitting calculation of the line laser point set in the camera coordinate system and the relative position relationship between the target coordinate system and the camera coordinate system. The height calibration accuracy and horizontal correction accuracy of the line laser sensor are guaranteed.

[0047] Combination Figure 2 and Figure 3It can be seen that the camera coordinate system XYZ conforms to the right-handed coordinate system. The right-handed coordinate system is one of the methods for defining a rectangular coordinate system in space. In this coordinate system, the positive directions of the X-axis, Y-axis, and Z-axis are defined as follows: place your right hand at the origin, with your thumb, index finger, and middle finger at right angles to each other, and point your thumb in the positive direction of the X-axis, your index finger in the positive direction of the Y-axis, and the direction pointed by the middle finger is the positive direction of the Z-axis.

[0048] As an embodiment, in step A, the method for calculating the rotation matrix corresponding to the yaw angle, the rotation matrix corresponding to the pitch angle, and the rotation matrix corresponding to the roll angle based on the target image captured by the camera from the surface of the vertical calibration plate includes: controlling the camera to capture the target image on the vertical calibration plate in front of it, and then extracting the corner points displayed by the target image, and then calculating the relative pose between the target coordinate system and the camera coordinate system in combination with the camera lens parameters and the corner points, and then extracting the rotation matrix corresponding to the yaw angle, the rotation matrix corresponding to the pitch angle, and the rotation matrix corresponding to the roll angle from the relative pose. It is also possible to first use the corner points of the target image to calculate the rotation parameters corresponding to the three degrees of freedom of the yaw angle, pitch angle, and roll angle, and obtain the corresponding rotation matrix (the rotation matrix is ​​composed of 3 rows and 3 columns of elements) to form the relative pose relationship between the target coordinate system and the camera coordinate system.

[0049] It should be noted that the method of calculating the relative pose between the target coordinate system and the camera coordinate system by combining the lens parameters and corner points of the camera is used to realize the target pose estimation in the camera coordinate system. This method can use the OpenCV open source algorithm library or Matlab and other tools to complete such target pose estimation, which will not be described here. Then the calculated relative pose between the target coordinate system and the camera coordinate system includes the basic rotation matrix and translation vector required for the conversion of the target coordinate system to the camera coordinate system, which can be expressed by the rotation parameters of the three degrees of freedom: yaw angle, pitch angle and roll angle.

[0050] In some embodiments, the detectable range of the line laser sensor is the distance range between the minimum detection distance of the camera and the maximum detection distance of the camera, so that the camera can capture the complete pattern of the target in the vertical calibration plate within the detectable range. The detection range of the line laser sensor in the marking scene is a preset distance in front of the line laser sensor. Since the detection range of the line laser sensor includes the distance range between the minimum detection distance of the camera and the maximum detection distance of the camera, the preset distance is set to be between the minimum detection distance of the camera and the maximum detection distance of the camera, which can be selected according to actual conditions without special restrictions.

[0051] Among them, the appropriate target pattern size is selected according to the camera lens parameters of the line laser sensor. If the camera of the line laser sensor is equipped with an infrared filter, a fill light device can be added to fill light for the target when acquiring the target image, which is convenient for subsequent image processing. There is no restriction on the horizontal direction of the target in the vertical calibration plate (the normal vector of the plane where the target is located), because the point set on the horizontal plane plays a real horizontal correction role in the operation of step B and step C, which also simplifies the production of the calibration plate and facilitates production.

[0052] Specifically, by Figure 3 The camera coordinate system XYZ can be deduced that the angle between the projection vector of the normal line of the target plane on the XOZ coordinate plane of the camera coordinate system and the X axis is the yaw angle θ Y , forming the angle required for the target coordinate system to rotate around the Y axis; the projection vector of the normal line of the target plane on the XOZ coordinate plane of the camera coordinate system and its angle with the X axis are calculated according to trigonometric functions in the camera coordinate system, and based on the properties of trigonometric functions, the value of the yaw angle is converted to be between 0 and 90 degrees, so that the angle between the projection vector of the normal line of the target plane on the XOZ coordinate plane of the camera coordinate system and the positive direction of the X axis is the yaw angle.

[0053] The angle between the projection vector of the normal of the target plane on the YOZ coordinate plane of the camera coordinate system and the Z axis is the pitch angle θ P , forming the angle of rotation of the target coordinate system around the X axis; the projection vector of the normal line of the target plane on the YOZ coordinate plane of the camera coordinate system and its angle with the Y axis are calculated according to trigonometric functions in the camera coordinate system, and based on the properties of trigonometric functions, the pitch angle θ P The value of is converted to be between 0 and 90 degrees.

[0054] The angle between the projection vector of the coordinate axis with the smallest angle with the X axis of the camera coordinate system in the two-dimensional coordinate system of the target plane and the X axis of the camera coordinate system on the XOY coordinate plane of the camera coordinate system is the roll angle θ. R , forming the angle of rotation of the target coordinate system around the Z axis; the angle between the two-dimensional coordinate system of the target plane and the X axis of the camera coordinate system, the projection vector of the coordinate axis with the smallest angle between the two-dimensional coordinate system of the target plane and the X axis of the camera coordinate system in the XOY coordinate plane of the camera coordinate system and its angle with the X axis of the camera coordinate system are all calculated according to trigonometric functions in the camera coordinate system, and based on the properties of trigonometric functions, the roll angle θ R The value of is converted to be between 0 and 90 degrees.

[0055] It should be noted that the two-dimensional coordinate system of the plane where the target is located is set according to the boundary direction of the target pattern, and the two-dimensional coordinate system of the plane where the target is located is composed of two coordinate axes located on the target plane in the target coordinate system; one coordinate axis of the two-dimensional coordinate system of the plane where the target is located can be parallel to the boundary of the target pattern; the coordinate axis with the smallest angle with the X-axis of the camera coordinate system in the two-dimensional coordinate system of the plane where the target is located belongs to one of the two coordinate axes located on the target plane in the target coordinate system, so that the corresponding coordinate axis required for calculating the roll angle is as close to the horizontal direction as possible.

[0056] Further, in step A, the rotation matrix corresponding to the pitch angle, the rotation matrix corresponding to the yaw angle, and the rotation matrix corresponding to the roll angle are multiplied in sequence, and the product obtained is the first combined rotation matrix. In the prior art, the combined rotation matrix corresponding to the Euler angle according to the ZXY rule is obtained by multiplying the rotation matrix corresponding to the roll angle, the rotation matrix corresponding to the yaw angle, and the rotation matrix corresponding to the pitch angle in sequence, which is used to transform the camera coordinate system; in this embodiment, the rotation matrix corresponding to the pitch angle, the rotation matrix corresponding to the yaw angle, and the rotation matrix corresponding to the roll angle are multiplied in sequence to obtain the first combined rotation matrix. The first combined rotation matrix can be understood as a combination of the rotation matrices required in each coordinate axis direction of the target coordinate system, which constitutes the rotation transformation relationship between the target coordinate system and the camera coordinate system. Therefore, the first combined rotation matrix is ​​used to transform the target coordinate system to convert the target posture information from the target coordinate system to the camera coordinate system.

[0057] It should be noted that the roll angle θ obtained in the above embodiment is R , pitch angle θ P , yaw angle θ Y In the camera coordinate system, they correspond to the rotation parameters / rotation matrices of three different degrees of freedom. It can be understood that the roll angle θ R , pitch angle θ P , yaw angle θ Y These correspond to rotation transformations with three different degrees of freedom, which respectively represent the position information of the target or the normal vector of the plane where the target is located, rotating the coordinate system around the Z axis, X axis, and Y axis.

[0058] If expressed in the form of a rotation matrix, the yaw angle θ Y Corresponding to the second rotation matrix R2, the pitch angle θ P Corresponding to the first rotation matrix R1, the rolling angle θ R Corresponding to the third rotation matrix R3.

[0059] Among them, the first rotation matrix R1 is The second rotation matrix R2 is The third rotation matrix R3 is

[0060] Therefore, the first combined rotation matrix is ​​equal to (R1*R2*R3), forming a corresponding combined rotation matrix in the right-hand coordinate system in the YXZ order, which directly represents the rotation transformation relationship between the target coordinate system and the camera coordinate system.

[0061] As an example, Figure 4 As shown, in step B, based on the first combined rotation matrix described in step A, coordinate transformation is performed on the line laser point set formed by the line laser transmitter emitting to the horizontal calibration plate to obtain a first horizontal calibration point set, and then the method of calculating the optimized roll angle using the first horizontal calibration point set includes:

[0062] Step B1: After confirming that the line laser point set formed by the line laser transmitter emitting to the horizontal calibration plate is valid, the valid line laser point set is marked as a valid laser point set. The three-dimensional coordinates of a laser point can be obtained from the valid laser point set. Then execute step B2.

[0063] In step B1, the second line laser point set is located on the surface of the horizontal calibration plate. The line laser point set formed by the line laser transmitter emitting to the horizontal calibration plate is a line laser point set formed by reflection from the horizontal calibration plate. At least some laser points in the line laser point set are ensured to be on the same straight line before the part of the laser points are formed into the effective laser point set.

[0064] Step B2, using the first combined rotation matrix described in step A to control the effective laser point set to perform coordinate transformation, and obtain the first horizontal calibration point set; then execute step B2. Based on the above embodiment, the three-dimensional coordinates of a laser point in the line laser point set formed by the line laser emitter emitting to the horizontal calibration plate are determined based on the first combined rotation matrix. After coordinate transformation, the three-dimensional coordinates of the corresponding points in the first horizontal calibration point set are equal to As the three-dimensional coordinate point after one correction, (R1*R2*R3) can be regarded as the correction matrix calculated based on the target image.

[0065] Step B3, obtain the direction vector (a, b, c) of the first horizontal calibration point set in the camera coordinate system by linear fitting, and use the direction vector (a, b, c) as the fitting result to determine the direction of the first horizontal calibration point set in the camera coordinate system, and then execute step B4. The direction vector (a, b, c) can be expressed as the direction of the line laser point set from the horizontal calibration plate after the coordinate transformation in step A, which is determined based on the straight line parallel to the direction vector (a, b, c) fitted by linear fitting.

[0066] Step B4: Calculate the angle between the projection vector of the direction vector (a, b, c) in the XOY coordinate plane and the X coordinate axis to obtain the optimized roll angle, and obtain the rotation matrix corresponding to the optimized roll angle, where the optimized roll angle The corresponding rotation matrix is ​​marked as R'. Specifically, the projection vector of the direction vector (a, b, c) in the XOY coordinate plane is the vector (a, b), the coordinate component of the vector (a, b) on the X axis is a, and the coordinate component of the vector (a, b) on the Y axis is b. From the trigonometric function relationship, we can see that the angle between the vector (a, b) and the X axis is

[0067] The aforementioned step B uses the angle formed by the direction of the line laser point set from the horizontal calibration plate and after the coordinate transformation corresponding to the first combined rotation matrix and the width direction of the robot's body (parallel to the X-axis) to represent the optimized roll angle, and further represents the angular deviation formed by the direction of the first horizontal calibration point set relative to the width direction of the robot's body. It can be understood as a more accurate angular assembly error generated on the roll angle of the line laser sensor, which serves as the roll angle required for horizontal correction when the target image is not corrected with a spirit level.

[0068] In this embodiment, the rotation in the three-dimensional space has three degrees of freedom, corresponding to rotation parameters / rotation matrices in three directions; by executing the aforementioned steps A to B, the external parameter calibration method first uses the target in the vertical calibration plate to obtain the first combined rotation matrix described in step A, and then uses the first combined rotation matrix to perform a corresponding rotation transformation on the effective laser point set to determine the completion of the rotation transformation of the three degrees of freedom, and then obtains a more accurate optimized roll angle than the roll angle obtained by the target image (the roll angle decomposed from the first combined rotation matrix), providing a horizontal reference of reasonable accuracy for the line laser sensor, which is convenient for step C to perform calibration and leveling of the line laser sensor.

[0069] On the basis of the above embodiment, in step C, the method of using the optimized roll angle described in step B to perform coordinate transformation on the first horizontal calibration point set to obtain a second horizontal calibration point set, and then marking the average height of the second horizontal calibration point set as the calibration extrinsic parameter of the line laser sensor includes:

[0070] The rotation matrix corresponding to the optimized roll angle is used to multiply the first horizontal calibration point set by the left multiplication to obtain the second horizontal calibration point set, so as to perform a new coordinate transformation on the first horizontal calibration point set, thereby overcoming the assembly angle error from the roll angle of the line laser sensor. The result of (R'*R1*R2*R3) is used as a calibration external parameter of the line laser sensor, and is multiplied by the left multiplication on the effective laser point set to obtain the point set after horizontal correction, i.e., the second horizontal calibration point set.

[0071] Then, in the second horizontal calibration point set, the average value of the coordinate values ​​of all points on the Y axis is calculated to obtain the average height of the second horizontal calibration point set, which can be used as the installation height of the line laser sensor, that is, the height of the probe position of the line laser sensor from the horizontal plane / the surface of the horizontal calibration plate; then the average height of the second horizontal calibration point set is marked as the calibration height of the line laser sensor to suppress the influence of the assembly height error of the line laser sensor. The calibration height of the line laser sensor is then marked as the calibration external parameter of the line laser sensor. Therefore, in this embodiment, the product of the rotation matrix corresponding to the optimized roll angle and the first combined rotation matrix and the calibration height of the line laser sensor are configured as the calibration result of the line laser sensor by the external parameter calibration method.

[0072] In summary, this embodiment uses the angle and height information calculated based on the relevant steps, and uses the line laser point set on the surface of the horizontal calibration plate to calculate the installation height of the line laser sensor and take into account the horizontal correction, thereby avoiding excessive complexity of the target, thereby simplifying the production of the calibration plate and facilitating production without affecting the accuracy of the external parameters, so that the height information of the second horizontal calibration point set is close to the ideal installation height value.

[0073] As an embodiment, the method of obtaining the direction vector by straight line fitting includes:

[0074] Step B31, extract the three-dimensional coordinates (xi, yi, zi) of at least three non-collinear points in the camera coordinate system XYZ from the first horizontal calibration point set, where i=1, 2, ..., n, and n is greater than or equal to 3; then execute step B32.

[0075] Step B32: Set the linear parameter equation of the straight line to be fitted in the camera coordinate system XYZ Then execute step B33. In the straight line parameter equation, (x0, y0, z0) is the three-dimensional coordinate of the fixed point through which the straight line to be fitted passes. The straight line to be fitted is preferably a straight line used to fit the direction of the first horizontal calibration point set. The straight line parameter equation is the fitting equation required for fitting the straight line to fit the direction of the first horizontal calibration point set. In the straight line parameter equation, t represents the distance moved by a point in the fitted straight line on the direction vector, x, y, and z are all dependent variables, and t is the independent variable. Different values ​​of the dependent variable on the same straight line are obtained by taking different values ​​of t.

[0076] Step B33, substitute the three-dimensional coordinates (xi, yi, zi) into the straight line parametric equation, solve to obtain the coordinate coefficients a, b and c, and then construct the direction vector (a, b, c) to determine the direction of the straight line to be fitted in the camera coordinate system, that is, the direction vector (a, b, c) of the point set of the effective laser point set of the horizontal calibration plate after the coordinate transformation in the camera coordinate system.

[0077] In step B33, after substituting the three-dimensional coordinates (xi, yi, zi) into the straight line parameter equation, the solution method of the straight line parameter equation, that is, the straight line fitting method, can adopt a variety of methods such as least squares method and RANSAC, which can be selected according to actual needs without limitation.

[0078] In this embodiment, the three coordinate coefficients obtained by fitting in the aforementioned straight line parameter equation can be regarded as the moving speeds of the straight line to be fitted on the X, Y, and Z coordinate axes of the camera coordinate system, so the three coordinate coefficients obtained by fitting jointly determine the direction of the straight line to be fitted.

[0079] In summary, the three-dimensional coordinates of the laser point in the camera coordinate system XYZ disclosed in this embodiment itself contain three sets of corresponding information, and the straight line direction vector of the horizontal calibration plate plane in the camera coordinate system can be established by fitting, which can be solved directly. The assembly angle error corresponding to the optimized roll angle calculated in step C is obtained from the fitting straight line information, which provides a roll angle reference for the horizontal correction of the line laser sensor. However, the error will not be accumulated and amplified due to repeated data acquisition, and the final calibration error is smaller than that of the prior art.

[0080] Preferably, the method for determining whether the line laser point set formed by the line laser transmitter is valid includes: determining whether the ratio of the number of laser points in the same straight line to the total number of laser points in the line laser point set is greater than a preset ratio threshold in the line laser point set, and if so, confirming that the line laser point set is valid. The line laser point set is the surface of the measured surface reflected from the horizontal calibration plate, and the preset ratio threshold is preferably 60%.

[0081] Preferably, the method for determining whether the line laser point set formed by the line laser transmitter is valid also includes: determining one by one whether each laser point in the line laser point set conforms to a preset standard data format, and if so, confirming that the line laser point set is valid; and detecting whether the data structure constituting the line laser point set is normal, and if there is garbled code or noise, the line laser point set is invalid.

[0082] The present application also discloses a calibration device, such as Figure 5 As shown, the calibration device includes a calibration plate control module, a sensor fixing module, a horizontal calibration plate with no target set on the surface, a vertical calibration plate with a target set on the surface, a point set acquisition module, and an external parameter calculation module; the sensor fixing module is used to fix the position of the line laser sensor, which is generally a fixture that can fix the robot or the line laser sensor.

[0083] The calibration plate control module is used to set the position of the horizontal calibration plate and the position of the vertical calibration plate to control the position of the horizontal calibration plate and the position of the vertical calibration plate to be within the detection range of the line laser sensor; the point set acquisition module is used to acquire the line laser point set formed by the horizontal calibration plate being emitted by the line laser sensor, and the line laser point set reflected by the horizontal calibration plate can be stored in the form of image pixels; the point set acquisition module is also used to acquire the target image on the surface of the vertical calibration plate. While ensuring the reliability of the calibration result, the point set acquisition module only needs to collect a set of line laser point sets and a set of target images, the process is simple, and the external parameter calibration efficiency of the calibration device is improved.

[0084] The extrinsic parameter calculation module is used to calculate the calibration extrinsic parameters of the line laser sensor according to the extrinsic parameter calibration method (including step A, step B and step C) using the target image and the line laser point set acquired by the point set acquisition module to obtain the aforementioned second combined rotation matrix and the calibration height of the line laser sensor.

[0085] In summary, the calibration device uses a vertical calibration plate with a target and a horizontal calibration plate without a target to collect a line laser point set, and on this basis, uses the target image to calculate a first combined rotation matrix formed by combining a rotation matrix corresponding to the pitch angle, a rotation matrix corresponding to the yaw angle, and a rotation matrix corresponding to the roll angle, and uses the first combined rotation matrix to perform coordinate transformation on the line laser point set formed by the horizontal calibration plate and then optimize the roll angle accordingly, providing a roll angle reference for the horizontal correction of the line laser sensor, so as to achieve the effect of using the line laser point set formed by the horizontal calibration plate to perform horizontal plane correction, and also calculates the calibration height of the line laser sensor based on the optimized roll angle, taking into account the installation height correction of the line laser sensor, thereby ensuring the reliability of the external parameter calibration, without the need to overly complicate the target setting, reducing the requirements for the production of the calibration plate, and not needing to continuously collect multiple sets of line laser point sets, nor to continuously collect multiple sets of target images, thereby simplifying the calibration process as a whole and improving the external parameter calibration efficiency of the line laser sensor.

[0086] Preferably, the calibration device also includes a storage module for writing the calibration external parameters calculated above into a corresponding storage unit, wherein the storage unit for storing the external parameters may be located in a memory on the robot or in a memory in the line laser sensor, without limitation.

[0087] Based on the above embodiments, the present application also discloses a calibration system, the calibration system includes a robot and an external parameter calibration device, the external parameter calibration device includes a memory and a processor; the memory has a line laser external parameter calibration program, and the processor executes the external parameter calibration method when running the line laser external parameter calibration program. When the external parameter calibration device preferably includes a calibration device, the processor includes an external parameter calculation module. When the processor executes the external parameter calibration method disclosed in the above embodiments when running the line laser external parameter calibration program, the first combined rotation matrix can be obtained according to the angle information calculated from step A to step C, and the constraint relationship between the target coordinate system and the camera coordinate system can be constructed, which also corresponds to the constraint relationship between the straight line direction vector of the laser point set of the horizontal plane scanned by the line laser sensor and the coordinate axis of the camera coordinate system, and the optimized roll angle and its corresponding rotation matrix can be obtained, and then the second combined rotation matrix can be combined, which is equivalent to the coordinate transformation of the line laser point set output by the line laser sensor to the horizontal calibration plate through the first combined rotation matrix and the second combined rotation matrix, so that the final point set result is close to the ideal calibration value, and the overall calibration process of the line laser sensor is completed.

[0088] Since some components included in the calibration device of the present application may be known components individually, the details of the known components will be omitted in the following description without affecting the full disclosure and feasibility of the invention of the device. In addition, some or all steps of the external parameter calibration method of the present application may be in the form of software and / or hardware entities, and may be executed by the calibration device of the present invention or its equivalent device. Without affecting the full disclosure and feasibility of the present application, the following description will focus on the content of the steps rather than the hardware.

[0089] The above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable technicians familiar with the technical field to understand the content of the present invention and implement it accordingly, and they cannot be used to limit the protection scope of the present invention. Any equivalent changes or modifications made according to the spirit of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for calibrating external parameters of a line laser sensor, wherein the line laser sensor is fixedly mounted on a robot, and the line laser sensor includes a camera and a line laser transmitter; characterized in that: There are a vertical calibration plate and a horizontal calibration plate arranged vertically on the outside of the robot, wherein a target is arranged on the surface of the vertical calibration plate, and no target is arranged on the surface of the horizontal calibration plate; both the vertical calibration plate and the horizontal calibration plate are installed within the detectable range of the line laser sensor; The external parameter calibration method comprises: Step A, based on the target image collected by the camera from the surface of the vertical calibration plate, calculate the rotation matrix corresponding to the yaw angle, the rotation matrix corresponding to the pitch angle and the rotation matrix corresponding to the roll angle, and then calculate the first combined rotation matrix through the rotation matrix corresponding to the yaw angle, the rotation matrix corresponding to the pitch angle and the rotation matrix corresponding to the roll angle; Step B: Based on the first combined rotation matrix described in step A, coordinate transformation is performed on the line laser point set formed by the line laser transmitter emitting to the horizontal calibration plate to obtain a first horizontal calibration point set, and then the optimized roll angle is calculated using the first horizontal calibration point set; Step C, using the optimized roll angle described in step B to perform coordinate transformation on the first horizontal calibration point set to obtain a second horizontal calibration point set, and then marking the average height of the second horizontal calibration point set as the calibration external parameter of the line laser sensor.

2. The external parameter calibration method according to claim 1, characterized in that: The step C also includes: controlling the rotation matrix corresponding to the optimized roll angle to be multiplied by the first combined rotation matrix, the product obtained is a second combined rotation matrix, and then marking the second combined rotation matrix as a calibration external parameter of the line laser sensor, so that the calibration height of the line laser sensor and the second combined rotation matrix constitute the calibration result of the line laser sensor by the external parameter calibration method; The rotation matrix corresponding to the optimized roll angle is configured to be the rotation matrix required for coordinate transformation of the first horizontal calibration point set.

3. The external parameter calibration method according to claim 2, characterized in that: Before executing step A, the line laser transmitter does not emit the line laser, the camera collects the target image, and obtains the target image, and the target image does not contain light strip information formed by the reflection of the line laser; Before executing step B, the line laser transmitter emits a line laser toward the horizontal calibration plate, and the camera collects a line laser point set formed by the line laser reflecting on the surface of the horizontal calibration plate, thereby obtaining a line laser point set formed by the line laser transmitter emitting toward the horizontal calibration plate; wherein the horizontal calibration plate is parallel to the horizontal plane; in the line laser sensor, a camera coordinate system is set inside the camera, and the line laser point set is located under the camera coordinate system.

4. The external parameter calibration method according to claim 3, characterized in that: In the camera coordinate system, the coordinate axis of the normal vector parallel to the surface of the vertical calibration plate is set as the Z axis; wherein the positive direction of the Z axis represents the horizontal forward direction of the robot or the central axis of the body, so as to point to the front of the line laser transmitter; In the camera coordinate system, a coordinate axis parallel to the width direction of the robot body is set as the X axis; wherein the width direction of the robot body is perpendicular to the horizontal forward direction of the robot; In the camera coordinate system, set the plane formed by the intersection of the X axis and the Z axis as the XOZ coordinate plane, and then set the XOZ coordinate plane to be parallel to the horizontal plane; In the camera coordinate system, set the coordinate axis perpendicular to the XOZ coordinate plane as the Y axis, and set the plane formed by the intersection of the X axis and the Y axis as the XOY coordinate plane; Among them, the YOZ coordinate plane, the XOZ coordinate plane and the XOY coordinate plane intersect perpendicularly at the origin O of the camera coordinate system.

5. The external parameter calibration method according to claim 4, characterized in that: In step A, the method of calculating the rotation matrix corresponding to the yaw angle, the rotation matrix corresponding to the pitch angle, and the rotation matrix corresponding to the roll angle based on the target image collected by the camera from the surface of the vertical calibration plate includes: The camera is controlled to collect the target image on the vertical calibration plate in front of it, and then the corner points displayed in the target image are extracted. Then, the relative pose relationship between the target coordinate system and the camera coordinate system is calculated based on the camera lens parameters and the corner points. Then, the rotation matrix corresponding to the yaw angle, the rotation matrix corresponding to the pitch angle, and the rotation matrix corresponding to the roll angle are extracted from the relative pose relationship.

6. The external parameter calibration method according to claim 5, characterized in that: The angle between the projection vector of the normal line of the target plane on the XOZ coordinate plane of the camera coordinate system and the X axis is the yaw angle; The angle between the projection vector of the normal line of the target plane on the YOZ coordinate plane of the camera coordinate system and the Z axis is the pitch angle; The angle between the projection vector of the coordinate axis with the smallest angle with the X axis of the camera coordinate system in the two-dimensional coordinate system of the target plane and the X axis of the camera coordinate system on the XOY coordinate plane of the camera coordinate system is the roll angle; In step A, the rotation matrix corresponding to the pitch angle, the rotation matrix corresponding to the yaw angle, and the rotation matrix corresponding to the roll angle are multiplied in sequence, and the product obtained is a first combined rotation matrix.

7. The external parameter calibration method according to claim 6, characterized in that: In step B, based on the first combined rotation matrix described in step A, coordinate transformation is performed on the line laser point set formed by the line laser transmitter emitting to the horizontal calibration plate to obtain a first horizontal calibration point set, and then the method of calculating the optimized roll angle using the first horizontal calibration point set includes: Step B1, after confirming that the line laser point set formed by the line laser transmitter emitting to the horizontal calibration plate is valid, marking the valid line laser point set as a valid laser point set; Step B2, using the first combined rotation matrix described in step A to control the effective laser point set to perform coordinate transformation, to obtain a first horizontal calibration point set; Step B3, obtaining the direction vector (a, b, c) of the first horizontal calibration point set in the camera coordinate system by straight line fitting; Step B4, calculate the angle between the projection vector of the direction vector (a, b, c) in the XOY coordinate plane and the X coordinate axis, obtain the optimized roll angle, and obtain the rotation matrix corresponding to the optimized roll angle.

8. The external parameter calibration method according to claim 7, characterized in that: In step C, the method of using the optimized roll angle described in step B to transform the coordinates of the first horizontal calibration point set to obtain a second horizontal calibration point set, and then marking the average height of the second horizontal calibration point set as the calibration external parameter of the line laser sensor includes: Multiplying the first horizontal calibration point set by the rotation matrix corresponding to the optimized roll angle to obtain a second horizontal calibration point set; Then, in the second horizontal calibration point set, the average value of the coordinate values ​​of all points on the Y-axis is calculated to obtain the average height of the second horizontal calibration point set, and then the average height of the second horizontal calibration point set is marked as the calibration height of the line laser sensor, and then the calibration height of the line laser sensor is marked as the calibration external parameter of the line laser sensor.

9. The external parameter calibration method according to claim 7, characterized in that: Methods for obtaining direction vectors by straight line fitting include: Step B31, extracting the three-dimensional coordinates (xi, yi, zi) of at least three non-collinear points in the camera coordinate system XYZ from the first horizontal calibration point set, where i=1, 2, ..., n, and n is greater than or equal to 3; Step B32: Set the linear parameter equation of the straight line to be fitted in the camera coordinate system XYZ Wherein, (x0, y0, z0) is the three-dimensional coordinate of the fixed point through which the straight line to be fitted passes; the straight line to be fitted is the straight line used to fit the direction of the first horizontal calibration point set; t represents the distance that a point in the fitted straight line moves on the direction vector; Step B33, substitute the three-dimensional coordinates (xi, yi, zi) into the linear parametric equation, solve to obtain the coordinate coefficients a, b and c, and then construct the direction vector (a, b, c).

10. The external parameter calibration method according to claim 9, characterized in that: Methods for determining whether the line laser point set formed by the line laser transmitter is valid include: In the line laser point set, it is determined whether the ratio of the number of laser points in the same straight line to the total number of laser points in the line laser point set is greater than a preset ratio threshold, and if so, it is confirmed that the line laser point set is valid.

11. The calibration device is characterized in that: The calibration device includes a calibration plate control module, a sensor fixing module, a horizontal calibration plate with no target set on the surface, a vertical calibration plate with a target set on the surface, a point set acquisition module, and an external parameter calculation module; A sensor fixing module, used for fixing the position of the line laser sensor; A calibration plate control module, used to set the position of the horizontal calibration plate and the position of the vertical calibration plate, so as to control the position of the horizontal calibration plate and the position of the vertical calibration plate to be within the detection range of the line laser sensor; A point set acquisition module, used to acquire a line laser point set formed by the horizontal calibration plate being emitted by the line laser sensor; The point set acquisition module is also used to acquire the target image on the surface of the vertical calibration plate; An extrinsic parameter calculation module is used to calculate the calibration extrinsic parameters of the line laser sensor using the target image and the line laser point set acquired by the point set acquisition module according to the extrinsic parameter calibration method according to any one of claims 1 to 10.

12. The calibration system is characterized in that The calibration system includes a robot and an external parameter calibration device, wherein the external parameter calibration device includes a memory and a processor; The memory has a line laser external parameter calibration program, and the processor executes the external parameter calibration method according to any one of claims 1 to 10 when running the line laser external parameter calibration program.

Images