Calibration method, calibration device and calibration system for external parameters of line laser module

Abstract

The invention discloses an external parameter calibration method, calibration device and calibration system of a line laser module, and the external parameter calibration method comprises the following steps: A, calculating a yaw angle based on a line laser point set formed by transmitting a line laser transmitter to a vertical calibration plate; step B, calculating a pitch angle under the condition that the vertical calibration plate displaces relative to the line laser module; c, on the basis of the yaw angle and the pitch angle, coordinate transformation is conducted on a line laser point set formed by transmitting the line laser transmitter to the horizontal calibration plate, and a laser mapping point set is obtained; then calculating a rolling angle by using the laser mapping point set; and D, performing coordinate transformation on the laser mapping point set by using the rolling angle to obtain a calibration horizontal plane point set, and marking the average height of the calibration horizontal plane point set as the calibration external parameter of the line laser module. The calibration requirement of the line laser module in a scene with a low precision requirement is met, and the calculation amount is small.

Description

Technical Field

[0001] The present application relates to the field of calibration of line laser sensors, and in particular to an external parameter calibration method, a calibration device and a calibration system for a line laser module. Background Art

[0002] When the structured light sensor is deployed on a robot, there may be certain assembly errors. The assembly errors will cause the obstacle position information obtained by the robot from the built-in infrared camera to deviate from the actual information, resulting in reduced measurement accuracy. To solve this type of problem, the solution in the prior art is to calibrate the rotation and offset parameters of the line laser sensor relative to the robot. The commonly used method is to use the line laser sensor to collect target information at a specific position. First, the first coordinate of the target mapped to the machine coordinate system and the second coordinate of the target mapped to the laser coordinate system are obtained, and then the calibration parameters are solved using the first coordinate and the second coordinate. In this method, obtaining the target coordinates in the machine coordinate system requires the use of tools other than line laser sensors to assist in positioning, such as levels, dynamic inclinometers, etc. However, the measurement data is greatly affected by platform vibration and acceleration, and the image processing of the target and the matching and solving of the first coordinate and the second coordinate both require a large amount of calculation; moreover, in some application scenarios, the external parameter accuracy is not required to be very high to meet the application requirements, and the use of some high-precision gyroscopes to provide three different degrees of freedom of rotation angles will instead increase the production cost, and the complex operations such as alignment and synchronization are not conducive to improving production efficiency. Summary of the invention

[0003] The present application discloses an external parameter calibration method, a calibration device and a calibration system for a line laser module. The specific technical scheme is as follows:

[0004] A method for calibrating an external parameter of a line laser module, wherein the line laser module is fixedly mounted on a robot, and the line laser module comprises a camera and a line laser transmitter; a vertical calibration plate and a horizontal calibration plate are vertically arranged outside the robot, wherein neither the vertical calibration plate nor the horizontal calibration plate is provided with a target pattern; the vertical calibration plate and the horizontal calibration plate are both mounted within the detectable range of the line laser module; the method for calibrating an external parameter comprises the following steps: step A, calculating a yaw angle based on a line laser point set formed by the line laser transmitter emitting to the vertical calibration plate; step B , calculate the pitch angle based on the displacement of the vertical calibration plate relative to the line laser module; step C, based on the yaw angle described in step A and the pitch angle described in step B, perform coordinate transformation on the line laser point set formed by the line laser transmitter emitting to the horizontal calibration plate to obtain a laser mapping point set, and then use the laser mapping point set to calculate the roll angle; step D, use the roll angle to perform coordinate transformation on the laser mapping point set to obtain a calibration horizontal plane point set, and then mark the average height of the calibration horizontal plane point set as the calibration external parameter of the line laser module.

[0005] Compared with the prior art, the technical solution only needs to collect the line laser point set emitted by the line laser transmitter in the vertical calibration plate and the horizontal calibration plate, without introducing additional target image processing and calculation of target coordinates in the robot coordinate system; by executing steps A to D defined by the external parameter calibration method, the line laser point set formed by the line laser transmitter emitting to the vertical calibration plate is used to calculate the yaw angle and the pitch angle, and the assembly angle error of the line laser module in the two dimensions of yaw angle and pitch angle is obtained, and then the roll angle is calculated based on the corresponding constraint relationship using the line laser point set formed by the line laser transmitter emitting to the horizontal calibration plate, so as to obtain the assembly angle error information of the line laser module in three degrees of freedom, and provide a rotation angle reference in three degrees of freedom for the correction of the line laser module.

[0006] Since the direction vector and projection vector of the line laser point set are used for angle calculation, there is no need to use complex image optimization algorithms to optimize the results multiple times, including no need to perform distortion correction, difference and binarization processing on the calibration scene image. Therefore, it is possible to obtain the rotation angle information on the three degrees of freedom in the camera space coordinates in succession while consuming less pose calculation. This makes it easy to overcome the angle error caused by the assembly error of the line laser module by performing coordinate transformation on the line laser point set, which is sufficient to solve the calibration needs of the line laser module in scenes with low precision requirements. The calibration process is simpler than that of the target pattern and requires less calculation.

[0007] The present application also discloses a calibration device, which includes a calibration plate control module, a sensor fixing module, a horizontal calibration plate, a vertical calibration plate, a point set acquisition module, and an external parameter calculation module; the sensor fixing module is used to fix the position of the line laser module; the calibration plate control module is used to set the position of the horizontal calibration plate and the position of the vertical calibration plate to control the position of the horizontal calibration plate and the position of the vertical calibration plate to be within the detection range of the line laser module; the point set acquisition module is used to acquire the line laser point set formed by the horizontal calibration plate being emitted by the line laser module; the calibration plate control module is also used to control the vertical calibration plate according to step B in the external parameter calibration method, The vertical calibration plate moves through at least two calibration positions relative to the fixed line laser module, so that the line laser point set formed by the line laser module emitting to the vertical calibration plate is within its detection range; the point set acquisition module is used to obtain the line laser point set formed by the line laser module when the vertical calibration plate is at different calibration positions according to the step B; the external parameter calculation module is used to calculate the calibration external parameters of the line laser module using the line laser point set obtained by the point set acquisition module according to the external parameter calibration method; wherein the calibration device also includes a slide rail, the calibration plate control module controls the vertical calibration plate to translate along the slide rail, and the sensor fixing module fixes the line laser module at one end of the slide rail.

[0008] In summary, the calibration device only needs to collect the line laser point set emitted by the line laser transmitter in the vertical calibration plate and the horizontal calibration plate, without introducing additional target image processing and calculation of target coordinates in the robot coordinate system. The line laser point set formed by the line laser transmitter emitting to the vertical calibration plate is used to calculate the yaw angle and pitch angle, and the assembly angle error of the line laser module in the two dimensions of yaw angle and pitch angle is obtained. Then, based on the corresponding constraint relationship, the line laser point set formed by the line laser transmitter emitting to the horizontal calibration plate is used to calculate the roll angle, and thus the assembly angle error information of the line laser module in three degrees of freedom is obtained, providing a rotation angle reference in three degrees of freedom for the correction of the line laser module. On this basis, the angle error caused by the assembly error of the line laser module is overcome by performing coordinate transformation on the line laser point set, which is sufficient to solve the calibration requirements of the line laser module in scenarios where the precision requirements are not high. The calibration process is simpler than the calibration process of the target pattern, and the amount of calculation is small.

[0009] The calibration system includes a robot and an external parameter calibration device, and the external parameter calibration device includes a memory and a processor; the memory includes a control program for necessary peripheral devices and a line laser external parameter calibration program, and the processor executes the external parameter calibration method when running the line laser external parameter calibration program. Thus, the rotation matrix corresponding to the yaw angle, the rotation matrix corresponding to the pitch angle, and the rotation matrix corresponding to the roll angle can be obtained according to the angle information calculated from step A to step C, and the constraint relationship or corresponding projection relationship between the coordinate systems of the line laser point set formed by the line laser transmitter emitting to the vertical calibration plate and the line laser point set formed by the line laser transmitter emitting to the horizontal calibration plate is constructed, which also corresponds to the constraint relationship between the straight line direction vectors of different planes scanned by the line laser module, and then the line laser point set output by the line laser module can be transformed by the rotation matrix corresponding to the yaw angle, the rotation matrix corresponding to the pitch angle, and the rotation matrix corresponding to the roll angle, so that the final point set result is close to the ideal calibration value, and the overall calibration process of the line laser module is completed. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 It is a flowchart of an external parameter calibration method of a line laser module disclosed in an embodiment of the present application.

[0011] Figure 2 It is a schematic diagram of an embodiment of the present application that discloses a line laser module emitting a line laser into a measured surface of a horizontal calibration plate.

[0012] Figure 3 It is a schematic diagram of an embodiment of the present application that discloses a line laser module emitting a line laser into a measured surface of a vertical calibration plate, wherein the vertical calibration plate is configured to translate relative to the line laser module.

[0013] Figure 4It is a flow chart of a method for calculating a yaw angle disclosed in one embodiment of the present application.

[0014] Figure 5 It is a flow chart of a method for calculating a roll angle disclosed in one embodiment of the present application.

[0015] Figure 6 It is a schematic diagram of the module distribution of a calibration device disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0016] The technical solutions in the embodiments of the present invention will be described in detail below in conjunction with the drawings in the embodiments of the present invention. To further illustrate the embodiments, the present invention provides drawings. These drawings are part of the disclosure of the present invention, which are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these contents, ordinary technicians in this field should be able to understand other possible implementations and advantages of the present invention.

[0017] As an embodiment, a method for calibrating the external parameters of a line laser module is disclosed, which is used to calibrate and calculate the external parameters of the line laser module; the control subject of the external parameter calibration method can be a controller used to calibrate and calculate the external parameters of the line laser module. The line laser module is fixedly mounted on the robot, wherein the line laser module includes a camera and a line laser transmitter. The line laser module can be a line laser sensor, which is a type of structured light sensor. The line laser transmitter is used to emit a line laser externally and reflect it on the surface of the object to be measured to form a laser point set. The laser point set can be regarded as being located in the intersection of the emitted laser plane and the surface of the object to be measured or in the area near the intersection; the camera included in the line laser module can be an infrared camera, which is used to collect the line laser point set, or it can obtain image data containing line laser light strip information, and the line laser point set is obtained by calculating the internal calibration parameters of the line laser module, which can be selected according to actual needs without limitation. The line laser point set disclosed in the present application is a description form of a laser point cloud or a laser point set.

[0018] In this embodiment, there are vertically arranged vertical calibration plates and horizontal calibration plates on the outside of the robot, and neither of the vertical calibration plates nor the horizontal calibration plates is provided with a target pattern, so that horizontal and vertical solid-color plates without patterns are used for line laser data collection; the vertical calibration plate and the horizontal calibration plate are both installed within the detectable range of the camera, and specifically, the line laser transmitter is configured to emit to the vertical calibration plate to form a line laser point set, and the line laser point set formed by the vertical calibration plate is collected by the camera, and further, under the control command issued by the aforementioned controller, the vertical calibration plate moves relative to the line laser module During the process, the line laser emitter emits a line laser point set formed by emitting to the moving vertical calibration plate, which can be captured by the camera and transmitted to the aforementioned controller. The line laser point formed by the line laser emitter is located in a laser plane. In addition, the line laser emitter is also configured to emit to the horizontal calibration plate to form a line laser point set. The line laser point set formed by the horizontal calibration plate is captured by the camera and transmitted to the aforementioned controller. At this time, the robot can be located on the ground parallel to the horizontal calibration plate and the line laser emitter emits an oblique laser to the horizontal calibration plate. There is no restriction on whether the horizontal calibration plate needs to be moved. The line laser point set is as follows: Figure 2 and 3 The camera coordinate system XYZ is expressed as the coordinate system data of the camera. In summary, the calibration scene of the line laser module is constructed.

[0019] In this embodiment, if Figure 1 As shown, the external parameter calibration method includes the following steps:

[0020] Step A, calculating the yaw angle based on the line laser point set formed by the line laser transmitter emitting to the vertical calibration plate, specifically calculating the angle formed by the projection vector of the direction vector of the line laser point set in the vertical calibration plate in the horizontal coverage plane of the robot and the body width direction of the robot, and then marking the calculated angle as the yaw angle; the yaw angle represents the angular deviation formed by the direction vector of the line laser point set in the vertical calibration plate relative to the body width direction of the robot, which can be understood as the rotation angle error corresponding to the yaw angle, that is, a part of the assembly angle error of the line laser module.

[0021] The width direction of the robot body is perpendicular to the horizontal forward direction of the robot, and the horizontal forward direction of the robot can be regarded as the forward direction of the line laser module or the front of the body.

[0022] In step A, first confirm that the line laser point set formed by the line laser transmitter emitting to the vertical calibration plate has been collected and is a valid line laser point set, and then specifically decomposed into first calculating the direction vector L1 of the line laser point set in the vertical calibration plate based on the line laser point set formed by the line laser transmitter emitting to the vertical calibration plate, wherein the direction vector L1 is a straight line fitting result to reduce the influence of the acquisition error; then calculate the projection vector S1 of the direction vector L1 in the horizontal coverage plane of the robot (parallel to the horizontal plane, that is, the plane where the robot is placed), and then calculate the angle formed by the projection vector S1 and the body width direction of the robot (specifically expressed by a coordinate axis direction). The body width direction of the robot can use the positive direction of the coordinate axis of the camera coordinate system or the negative direction of the coordinate axis. For the value of the angle, for the convenience of trigonometric function calculation, it is generally converted into an angle value between 0 and 90 degrees. For example, if the value of the angle is between 90 and 180 degrees, 180 degrees minus the value of the angle is used, and the difference obtained is used as the yaw angle.

[0023] Step B, calculate the pitch angle based on the displacement of the vertical calibration plate relative to the line laser module; specifically, when the vertical calibration plate is displaced relative to the line laser module, calculate the relative offset angle of the laser plane formed by the line laser transmitter relative to the horizontal forward direction of the robot, and then mark the calculated relative offset angle as the pitch angle. The pitch angle represents the angular deviation formed by the direction vector of the line laser point set in the vertical calibration plate relative to the horizontal forward direction of the robot in the vertical direction, which can be understood as the rotation angle error corresponding to the pitch angle, that is, a part of the assembly angle error of the line laser module.

[0024] In order to execute step B, it is necessary to control the displacement of the vertical calibration plate relative to the line laser module, wherein the position of the line laser module remains fixed, and the vertical calibration plate can move along the direction of its normal vector and is within the detectable range of the line laser module, then at least 2 sets of line laser point sets of the measured surface of the vertical calibration plate are obtained within the preset distance range in front of the line laser module. Step B is decomposed into: calculating the height change value of the line laser point set formed by the line laser transmitter during the movement of the vertical calibration plate, and calculating the detection angle of the line laser module in combination with the distance moved by the vertical calibration plate, and then controlling the detection angle of the line laser module to subtract the emission angle of the line laser transmitter to obtain the relative offset angle.

[0025] Step C, based on the yaw angle described in step A and the pitch angle described in step B, coordinate transformation is performed on the line laser point set formed by the line laser transmitter emitting to the horizontal calibration plate to obtain a laser mapping point set; the prerequisite for obtaining the laser mapping point set here is to place the robot flat on a horizontal plane or a horizontal calibration plate, obtain the line laser point set emitted by the line laser transmitter to the horizontal calibration plate and confirm that it is a valid line laser point set, then obtain the initial laser point set, and then control the product between the rotation matrix corresponding to the yaw angle and the rotation matrix corresponding to the pitch angle to multiply the initial line laser point set on the left to obtain the laser mapping point set. Then, the roll angle is calculated using the laser mapping point set, specifically, the angle formed by the projection vector of the direction vector of the laser mapping point set in the plane perpendicular to the horizontal forward direction of the robot and the width direction of the robot's body, and then the calculated result is marked as the roll angle. The roll angle represents the angle deviation formed by the direction vector of the line laser point set in the horizontal calibration plate relative to the width direction of the robot's body, which can be understood as the rotation angle error corresponding to the roll angle, that is, a part of the assembly angle error of the line laser module.

[0026] It should be noted that the aforementioned yaw angle, pitch angle, and roll angle constitute the assembly angle error of the line laser module. There is a constraint relationship or a corresponding projection relationship between coordinate systems between the yaw angle described in step A, the pitch angle described in step B, and the line laser point set formed by the line laser transmitter emitting to the horizontal calibration plate, which also corresponds to the constraint relationship between the straight line direction vectors of different planes scanned by the line laser module. Therefore, by executing step C, based on the yaw angle described in step A and the pitch angle described in step B, the line laser point set formed by the line laser transmitter emitting to the horizontal calibration plate is transformed into a coordinate to obtain a laser mapping point set, which can overcome the influence of the aforementioned yaw angle and pitch angle, that is, overcome the influence of the assembly angle errors of the two dimensions of the yaw angle and pitch angle of the line laser module.

[0027] Step C is specifically decomposed into first calculating the direction vector L2 of the laser mapping point set, wherein the direction vector L2 is a straight line fitting result to reduce the influence of the sensor assembly error, then calculating the projection vector S2 of the direction vector L2 in the plane perpendicular to the horizontal forward direction of the robot, and then calculating the angle formed by the projection vector S2 and the width direction of the robot (specifically, it is expressed by a coordinate axis direction). The width direction of the robot can be the positive direction of the coordinate axis of the camera coordinate system or the negative direction of the coordinate axis. For the value of the angle, for the convenience of trigonometric function calculation, it is generally transformed into an angle value between 0 and 90 degrees. For example, if the value of the angle is between 90 and 180 degrees, then 180 degrees minus the value of the angle is used to obtain the difference as the roll angle. Thus, step C is equivalent to first using the point set on the measured surface of the vertical calibration plate to perform the rotation operation required for the two coordinate axes of the camera coordinate system, and then using the laser mapping point set to calculate the angle required to rotate in the last direction on the basis of completing the rotation operation required for the two coordinate axes of the camera coordinate system.

[0028] Step D, using the roll angle to transform the laser mapping point set to obtain the calibration horizontal plane point set, and then mark the average height of the calibration horizontal plane point set as the calibration external parameter of the line laser module. The average height of the calibration horizontal plane point set represents the calibration height of the line laser module, and the calibration height of the line laser module is used to represent the height of the line laser module from the ground as a whole, which can be understood as the height of the line laser module from the ground when the robot is placed flat on the ground. In step D, in the process of coordinate transformation of the laser mapping point set, the rotation matrix corresponding to the roll angle is multiplied by the laser mapping point set on the left, and the laser mapping point set is transformed into the calibration horizontal plane point set as the final horizontal plane point set; thereby, on the basis of overcoming the influence of the aforementioned yaw angle and pitch angle, the influence of the assembly angle error corresponding to the aforementioned roll angle is continued to be overcome, so that the calibration horizontal plane point set is closer to the ideal value, but the external parameter accuracy is not required to be very high, which can meet the calibration requirements of the line laser module in the navigation and positioning scenario with low accuracy requirements. The ideal value here can be set according to the actual needs of horizontal calibration, including the corresponding coordinate axis of the camera coordinate system being parallel to a certain direction.

[0029] Compared with the prior art, the above-mentioned embodiment only needs to collect the line laser point set emitted by the line laser transmitter in the vertical calibration plate and the horizontal calibration plate, without introducing additional target image processing and calculation of target coordinates in the robot coordinate system; by executing steps A to D defined by the external parameter calibration method, the yaw angle and pitch angle are calculated using the line laser point set formed by the line laser transmitter emitting to the vertical calibration plate, and the assembly angle error of the line laser module in the two dimensions of yaw angle and pitch angle is obtained, and then the roll angle is calculated using the line laser point set formed by the line laser transmitter emitting to the horizontal calibration plate on the basis of the corresponding constraint relationship, so as to obtain the assembly angle error information of the line laser module in three degrees of freedom, and provide a rotation angle reference in three degrees of freedom for the correction of the line laser module.

[0030] Since the direction vector and projection vector of the line laser point set are used for angle calculation, there is no need to use complex image optimization algorithms to optimize the results multiple times, including no need to perform distortion correction, difference and binarization processing on the calibration scene image. Therefore, it is possible to obtain the rotation angle information on the three degrees of freedom in the camera space coordinates in succession while consuming less pose calculation. This makes it easy to overcome the angle error caused by the assembly error of the line laser module by performing coordinate transformation on the line laser point set, which is sufficient to solve the calibration needs of the line laser module in scenes with low precision requirements. The calibration process is simpler than that of the target pattern and requires less calculation.

[0031] In the above embodiment, the external parameter calibration method also includes: when the step C transforms to obtain the calibration horizontal plane point set, the rotation matrix corresponding to the yaw angle, the rotation matrix corresponding to the pitch angle, and the rotation matrix corresponding to the roll angle are all marked as the calibration external parameters of the line laser module, which can also be understood as the rotation transformation parameters required for the laser point set in each coordinate axis direction of the camera coordinate system, forming the rotation relationship between the laser point set in the camera coordinate system and the laser coordinate system. In the line laser module, the camera coordinate system is set in the camera, the line laser transmitter emits to form a laser plane, and the laser point set is configured to be located in the laser plane. Therefore, this embodiment obtains the rotation matrix corresponding to the yaw angle, the rotation matrix corresponding to the pitch angle, and the rotation matrix corresponding to the roll angle according to the calculated angle information, and constructs the constraint relationship or corresponding projection relationship between the coordinate systems of the line laser point set formed by the line laser transmitter emitting to the vertical calibration plate and the line laser point set formed by the line laser transmitter emitting to the horizontal calibration plate, which also corresponds to the constraint relationship between the straight line direction vectors of different planes scanned by the line laser module. Then, the line laser point set output by the line laser module can be transformed by the rotation matrix corresponding to the yaw angle, the rotation matrix corresponding to the pitch angle, and the rotation matrix corresponding to the roll angle, so that the final point set result is close to the ideal calibration value, thereby completing the overall calibration process of the line laser module.

[0032] Preferably, the rotation relationship can be expressed by a 3×3 orthogonal rotation matrix; of course, the yaw angle can be recorded as the yaw angle of the line laser module, the pitch angle can be recorded as the pitch angle of the line laser module, and the roll angle can be recorded as the roll angle of the line laser module, thereby forming the calibration external parameters of the line laser module to realize the calibration of the assembly angle of the line laser module.

[0033] It should be noted that the laser line emitted by the line laser transmitter is structured light. For example, line laser can be regarded as a laser plane composed of countless laser lines emitted by the same line laser transmitter, corresponding to Figure 2 and Figure 3 The light plane is composed of multiple dotted lines shown, and the intersection of the laser plane and the calibration plate forms a light strip, such as a projection spot of a straight line laser in the calibration plate; it can be extracted from the calibration plate image captured by the camera to perform triangulation operations (construct similar triangles for distance calculation).

[0034] As an embodiment, before executing step C of the above embodiment, the robot is fixedly placed on a horizontal surface, such as Figure 2 As shown, the line laser transmitter set inside the line laser module emits a line laser into the measured surface of the horizontal calibration plate. The line laser is reflected on the measured surface of the horizontal calibration plate to form laser points, which form a line laser point set. The line laser point set is located in the measured surface of the horizontal calibration plate; the laser points form scanning lines on the measured surface of the horizontal calibration plate to form a laser plane; or, the line laser is reflected on the measured surface of the horizontal calibration plate to form a laser plane, and the line laser point set is located in the laser plane. In this embodiment, the measured surface of the horizontal calibration plate is parallel to the horizontal plane. Figure 2 The horizontal plane shown is parallel to the coordinate plane XOZ. Then the camera collects the line laser point set formed by the reflection of the line laser on the measured surface of the horizontal calibration plate, obtains the line laser point set formed by the line laser transmitter emitting to the horizontal calibration plate, and provides the line laser point set for the coordinate transformation required to be performed in step C.

[0035] In the calibration scenario, the camera built into the line laser module preferably uses a small-angle camera to meet the pinhole imaging principle, and there is a certain angle between the forward emission direction of the line laser transmitter and the optical axis of the camera, and in the optical path system, the line laser is mostly oblique relative to the surface of the incident object. When the robot is fixed or placed on a horizontal plane, the angle between the forward emission direction of the line laser transmitter and the optical axis of the camera is fixed.

[0036] Before executing step A of the above embodiment, when the robot is stationary, Figure 3As shown, the line laser transmitter emits a line laser toward the vertical calibration plate. The line laser is reflected on the measured surface of the vertical calibration plate to form laser points, which constitute a line laser point set. The line laser point set is located within the measured surface of the vertical calibration plate. The laser points form scanning lines on the measured surface of the vertical calibration plate to enclose a laser plane. Alternatively, the line laser is reflected on the measured surface of the vertical calibration plate to form a laser plane, and the line laser point set is located within the laser plane. In this embodiment, the horizontal forward direction of the robot is parallel to the normal vector of the measured surface of the vertical calibration plate, the horizontal forward direction of the robot is parallel to the Z axis in the illustrated coordinate system XYZ, the measured surface of the vertical calibration plate is parallel to the coordinate plane YOX, and the vertical calibration plate is at Figure 3 The scene shown is configured to be translated relative to the line laser module, corresponding to a coordinate offset relative to the origin O. Then the camera collects the line laser point set formed by the reflection of the line laser in the vertical calibration plate, obtains the line laser point set formed by the line laser transmitter emitting to the vertical calibration plate, and provides the line laser point set for the direction vector calculation and yaw angle calculation required to be performed in step A. When the vertical calibration plate is translated, the line laser point set is provided for the calculation of the pitch angle in step B.

[0037] In the aforementioned embodiment, in the camera coordinate system, the first coordinate axis is set to be parallel to the normal vector of the measured surface of the vertical calibration plate; wherein the positive direction of the first coordinate axis represents the horizontal forward direction of the robot, so as to point to the front of the line laser transmitter; the second coordinate axis is parallel to the width direction of the robot's body, and the width direction of the robot's body uses the direction in which the robot's wheel axle crosses the body, then the first coordinate axis is perpendicular to the second coordinate axis.

[0038] In the camera coordinate system, the first coordinate plane is set as the horizontal coverage plane of the robot, wherein the horizontal coverage plane of the robot is parallel to the horizontal plane and also parallel to the measured surface of the horizontal calibration plate; the coordinate plane formed by the intersection of the first coordinate axis and the second coordinate axis is the first coordinate plane; if the robot needs to move, the first coordinate plane is configured as the walking plane of the robot, and the positive direction of the first coordinate axis represents the walking direction of the robot and the detection direction of the line laser module.

[0039] In the camera coordinate system, the second coordinate plane is set to a plane perpendicular to the horizontal forward direction of the robot, wherein the second coordinate plane is parallel to the measured surface of the vertical calibration plate, the coordinate plane formed by the intersection of the second coordinate axis and the third coordinate axis is the second coordinate plane, the second coordinate axis is perpendicular to the third coordinate axis, and the third coordinate axis is parallel to the normal vector of the first coordinate plane; when the vertical calibration plate is translated relative to the robot, the second coordinate plane does not translate, wherein the second coordinate plane is perpendicular to the first coordinate plane.

[0040] In the camera coordinate system, the coordinate plane formed by the intersection of the first coordinate axis and the third coordinate axis is the third coordinate plane; the height information of the laser point can be described in the third coordinate plane; the first coordinate plane, the second coordinate plane and the third coordinate plane intersect perpendicularly at the origin of the camera coordinate system.

[0041] In summary, the coordinate axes and coordinate planes of the camera coordinate system in various directions are constructed through the horizontal forward direction of the robot, the width direction of the robot's body, the horizontal plane, and the vertical calibration plate, thereby providing a spatial coordinate environment for the coordinate transformation of the line laser point set, the direction vector of the line laser point set in the laser plane in the camera coordinate system, the direction vector of the normal vector of the laser plane in the camera coordinate system, and the related projection vector.

[0042] Combination Figure 2 and Figure 3 It can be seen that in the camera coordinate system XYZ, the first coordinate axis is the Z axis, the second coordinate axis is the X axis, the third coordinate axis is the Y axis, the first coordinate plane is the XOZ coordinate plane, the second coordinate plane is the XOY coordinate plane, and the third coordinate plane is the YOZ coordinate plane, which conforms to the pointing rule of the right-hand coordinate system for the rectangular coordinate system.

[0043] The right-hand coordinate system is one of the methods of defining a rectangular coordinate system in space. In this coordinate system, the positive directions of the X-axis, Y-axis, and Z-axis are defined as follows: place your right hand at the origin, with your thumb, index finger, and middle finger at right angles to each other, and point your thumb in the positive direction of the X-axis, your index finger in the positive direction of the Y-axis, and the direction pointed by your middle finger is the positive direction of the Z-axis.

[0044] As an example, combining Figure 3 and Figure 4 It can be seen that in step A, the method for calculating the yaw angle based on the line laser point set formed by the line laser transmitter emitting to the vertical calibration plate includes:

[0045] Step A1: after confirming that the line laser point set formed by the line laser transmitter emitting to the vertical calibration plate is valid, mark the valid line laser point set as the first line laser point set, wherein the first line laser point set is located on the measured surface of the vertical calibration plate, and the line laser point set formed by the line laser transmitter emitting to the vertical calibration plate is a line laser point set formed by reflection of the vertical calibration plate, and at least some laser points in the line laser point set are ensured to be on the same straight line before the part of the laser points are formed into the first line laser point set, i.e., a valid line laser point set is obtained. Then, step A2 is executed.

[0046] Step A2: Obtain the direction vector (a1, b1, c1) of the first line laser point set in the camera coordinate system by fitting. The direction vector (a1, b1, c1) is used as the fitting result to determine the direction of the first line laser point set in the camera coordinate system. In this embodiment, a straight line parallel to the direction vector (a1, b1, c1) is fitted by straight line fitting to obtain the straight line equation in the camera coordinate system. Then, step A3 is executed.

[0047] Step A3: Calculate the angle between the projection vector of the direction vector (a1, b1, c1) in the first coordinate plane and the second coordinate axis. Then the angle θ Y Marked as the yaw angle; where the projection vector of the direction vector of the line laser point set in the vertical calibration plate in the horizontal coverage plane of the robot is the projection vector of the direction vector (a1, b1, c1) in the first coordinate plane, that is, the projection vector (a1, c1) of the direction vector (a1, b1, c1) in the XOZ coordinate plane, corresponding to Figure 3 It is the projection of the direction vector (a1, b1, c1) in the XOZ coordinate plane, so as to use the angle between the direction of the line laser point set in the vertical calibration plate and the width direction of the robot body (parallel to the second coordinate axis). Represents the yaw angle, and further represents the angle deviation formed by the direction of the line laser point set in the vertical calibration plate relative to the width direction of the robot body, which can be understood as the angle assembly error caused by the yaw angle of the line laser module. Further, after obtaining the yaw angle, the line laser point set (which can be the laser point set in the vertical calibration plate or the laser point set in the horizontal calibration plate) can be transformed by the rotation matrix corresponding to the yaw angle, so that the assembly angle error can be overcome from the degree of freedom of the yaw angle of the line laser module.

[0048] As an example, combining Figure 3 It can be seen that in step B, the method for calculating the pitch angle based on the displacement of the vertical calibration plate relative to the line laser module includes: when the line laser module and the robot are both fixed, controlling the line laser transmitter to emit a line laser to the vertical calibration plate, controlling the vertical calibration plate to translate relative to the line laser module in sequence within the detection range of the line laser module, and whenever the vertical calibration plate translates a preset distance Δd to a calibration position, the camera collects a set of line laser point sets formed by the reflection of the line laser in the vertical calibration plate, and extracts the height change value Δh generated by the corresponding line laser point set in the vertical calibration plate before and after the translation, which corresponds to the line laser emitted by the line laser transmitter or the laser plane in Figure 3 The height difference between the light strips formed on the measured surface of the two vertical calibration plates.

[0049] It should be noted that at least two sets of line laser point sets on the vertical calibration plate formed by the line laser sensor are obtained within the detection range of the line laser module, wherein the detection range of the line laser module is the preset distance in front of the line laser module in the marking scene. Since the detection range of the line laser module includes the distance range between the minimum detection distance of the camera and the maximum detection distance of the camera, the preset distance is set to be between the minimum detection distance of the camera and the maximum detection distance of the camera, which can be selected according to actual conditions without special restrictions. Since what is required here is the relative distance between the vertical calibration plates before and after the translation, and no absolute distance is required, the implementation difficulty is reduced.

[0050] In the calibration scenario, the vertical calibration plate is configured to translate along the slide rail, and the robot is configured to be fixed at one end of the slide rail, that is, the line laser module is fixed at one end of the slide rail, and the vertical calibration plate translates along the slide rail to the other end relative to the line laser module, wherein the slide rail is parallel to the first coordinate axis. During the translation of the vertical calibration plate along the slide rail, the line laser transmitter continues to emit line lasers to the vertical calibration plate, and the camera collects the line laser point set formed by the reflection of the line laser on the measured surface of the horizontal calibration plate every time the vertical calibration plate translates a preset distance Δd to a calibration position, so that the laser point collection process does not need to manually move the target, and it is easy to realize automation, thereby improving the external parameter calibration efficiency of the line laser module.

[0051] Then calculate the relative offset angle of the laser plane formed by the line laser transmitter relative to the horizontal forward direction of the robot. Then the relative offset angle θ P Marked as the pitch angle; wherein θ0 is the emission angle of the line laser emitter in the camera coordinate system, which is used to represent the angle formed by the intersection line between the laser plane and the third coordinate plane and the first coordinate axis; θ0 is the emission angle pre-measured in the line laser module; the preset distance Δd is pre-measured.

[0052] In this embodiment, the relative offset angle θ P As the pitch angle, it represents the angle deviation formed by the direction of the line laser point set in the vertical calibration plate relative to the horizontal forward direction of the robot in the vertical plane, which can be understood as the angle assembly error caused by the pitch angle of the line laser module. Further, after obtaining the pitch angle, the line laser point set (which can be the line laser point set in the vertical calibration plate or the line laser point set in the horizontal calibration plate) can be transformed by the rotation matrix corresponding to the pitch angle, so that the assembly angle error can be overcome from the degree of freedom of the pitch angle of the line laser module.

[0053] As an example, combining Figure 5 and Figure 2It can be seen that in step C, based on the yaw angle described in step A and the pitch angle described in step B, the laser point set formed by the line laser transmitter emitting to the horizontal calibration plate is transformed into a laser mapping point set, and then the method of calculating the roll angle using the laser mapping point set includes:

[0054] Step C1: after confirming that the line laser point set formed by the line laser transmitter emitting to the horizontal calibration plate is valid, mark the valid line laser point set as the second line laser point set; wherein the second line laser point set is located on the measured surface of the horizontal calibration plate, and the line laser point set formed by the line laser transmitter emitting to the horizontal calibration plate is a line laser point set formed by reflection of the horizontal calibration plate, and at least some laser points in the line laser point set are ensured to be on the same straight line before the part of the laser points are formed into the second line laser point set, i.e., a valid line laser point set is obtained. Then, step C2 is executed.

[0055] Step C2, using the rotation matrix corresponding to the yaw angle described in step A and the rotation matrix corresponding to the pitch angle described in step B to control the coordinate transformation of the second line laser point set to obtain a laser mapping point set; then executing step C3. In step C2, firstly, the product of the rotation matrix corresponding to the yaw angle and the rotation matrix corresponding to the pitch angle is used as a correction matrix, and then the correction matrix is ​​controlled to be left-multiplied by the second line laser point set to obtain a laser mapping point set, and the second line laser point set is determined to complete the corresponding coordinate transformation, which can overcome the influence of the aforementioned yaw angle and pitch angle, that is, overcome the influence of the assembly angle errors of the two dimensions of the yaw angle and pitch angle of the line laser module.

[0056] Step C3, obtain the direction vector (a2, b2, c2) of the laser mapping point set in the camera coordinate system by fitting; the direction vector (a2, b2, c2) is used as the fitting result to determine the direction of the laser mapping point set in the camera coordinate system, and the direction vector (a2, b2, c2) can be expressed as the direction of the line laser point set from the horizontal calibration plate after coordinate transformation; in this embodiment, a straight line parallel to the direction vector (a2, b2, c2) is fitted by straight line fitting to obtain the straight line equation in the camera coordinate system; then execute step C4.

[0057] Step C4: Calculate the angle between the projection vector of the direction vector (a2, b2, c2) in the second coordinate plane and the second coordinate axis. Then the angle θ Ris the rolling angle; in the camera coordinate system, the plane perpendicular to the horizontal forward direction of the robot is the second coordinate plane; when the horizontal forward direction of the robot is the Z axis, the second coordinate plane is the XOY coordinate plane; the projection vector of the direction vector of the laser mapping point set in the plane perpendicular to the horizontal forward direction of the robot is the projection vector of the direction vector (a2, b2, c2) in the second coordinate plane, recorded as vector (a2, b2), the coordinate value of the vector (a2, b2) on the second coordinate axis is a2, the coordinate value of the vector (a2, b2) on the third coordinate axis is b2, then it can be seen from the trigonometric function relationship that the angle between the vector (a2, b2) and the second coordinate axis is

[0058] In summary, step C uses the angle formed by the direction of the line laser point set from the horizontal calibration plate and after coordinate transformation and the width direction of the robot's body (parallel to the second coordinate axis) to represent the roll angle, and then represents the angle deviation formed by the direction of the laser mapping point set relative to the width direction of the robot's body, which can be understood as the angle assembly error generated on the roll angle of the line laser module. Further, after obtaining the roll angle, the rotation matrix corresponding to the roll angle can be controlled by executing step D to perform coordinate transformation on the laser mapping point set, so as to achieve a new coordinate transformation of the line laser point set of the horizontal calibration plate, thereby overcoming the assembly angle error from the roll angle of the line laser module.

[0059] In this embodiment, the rotation in three-dimensional space has three degrees of freedom, corresponding to rotation parameters in three directions; by executing the aforementioned steps A to C, the external parameter calibration method first uses the line laser point set on the vertical calibration plate to obtain the yaw angle described in step A and the pitch angle described in step B, and obtains rotation parameters in two directions; then the line laser point set on the horizontal calibration plate is subjected to corresponding rotation transformation using the yaw angle described in step A and the pitch angle described in step B, and the rotation transformation in two directions is determined, and then the angle required for the rotation transformation in the last direction, i.e., the pitch angle described in step C, is calculated to obtain a more accurate roll angle. A horizontal reference with reasonable accuracy is provided for the line laser module, which facilitates the calibration and leveling of the line laser module in step D, reduces the complexity of calibration and improves calibration efficiency.

[0060] It should be noted that the roll angle θ obtained in the above embodiment is R , pitch angle θ P , yaw angle θ Y In the camera coordinate system, they correspond to the rotation parameters of three different degrees of freedom, namely the rotation matrix. It can be understood that the roll angle θ R , pitch angle θ P , yaw angle θ YCorresponding to the rotation transformation of three different degrees of freedom, the rolling angle θ R It represents the angle required to rotate the coordinate system around the Z axis, the pitch angle θ P Indicates the angle required to rotate the coordinate system around the X-axis, the yaw angle θ Y Indicates the angle required to rotate the coordinate system around the Y axis.

[0061] If expressed in the form of a rotation matrix, the yaw angle θ Y Corresponding to the first rotation matrix, the pitch angle θ P Corresponding to the second rotation matrix, the rolling angle θ R Corresponding to the third rotation matrix.

[0062] Among them, the first rotation matrix is The second rotation matrix is The third rotation matrix is

[0063] As an embodiment, in step D, the method of using the roll angle to transform the coordinates of the laser mapping point set to obtain a calibration horizontal plane point set, and then marking the average height of the calibration horizontal plane point set as a calibration external parameter of the line laser module includes:

[0064] The rotation matrix corresponding to the roll angle is used to multiply the laser mapping point set in step C to obtain the calibration horizontal plane point set, and determine that the laser mapping point set completes the coordinate transformation; then in the calibration horizontal plane point set, the average value of the coordinate values ​​of all points on the third coordinate axis is calculated to obtain the average height of the calibration horizontal plane point set, and then the average height of the calibration horizontal plane point set is marked as the calibration height of the line laser module, and then the calibration height of the line laser module is marked as the calibration external parameter of the line laser module to suppress the influence of the assembly height error of the line laser module. In this embodiment, the first rotation matrix, the second rotation matrix, the third rotation matrix and the average height disclosed in the above embodiment are configured as the calibration result of the line laser module by the external parameter calibration method. In summary, this embodiment uses the roll angle that overcomes the assembly error influence of the remaining degrees of freedom (yaw angle and pitch angle) to calculate the new rotation direction angle, obtain the corresponding rotation matrix and apply it to the laser mapping point set, that is, to perform the last coordinate transformation, and obtain the optimized horizontal plane point set, that is, the calibration horizontal plane point set, without the need to use a complex optimization algorithm to perform secondary optimization on the result.

[0065] The method for obtaining the direction vector by fitting in the above embodiment includes:

[0066] Step P1, obtain the three-dimensional coordinates (x1i, y1i, z1i) of at least three non-collinear laser points on the measured surface of the vertical calibration plate in the camera coordinate system XYZ, preferably the three-dimensional coordinates of each laser point in the first line laser point set marked in step A1. Alternatively, obtain the three-dimensional coordinates (x2i, y2i, z2i) of at least three non-collinear laser points on the measured surface of the horizontal calibration plate after the coordinate transformation of the rotation matrix corresponding to the yaw angle and the rotation matrix corresponding to the pitch angle, preferably the three-dimensional coordinates of each laser point in the second line laser point set marked in step C1. Wherein, i = 1, 2, ..., n, n is greater than or equal to 3.

[0067] Step P2: Set the linear parameter equation of the laser plane in the camera coordinate system XYZ Among them, (x0, y0, z0) are the three-dimensional coordinates of the fixed point through which the straight line to be fitted passes, which are known parameters; t represents the distance moved by a point in the straight line to be fitted on the direction vector; the straight line parameter equation represents the fitting equation required for the straight line to be fitted in the laser plane; in the straight line parameter equation, x, y, and z are all dependent variables, and t is the independent variable. By taking different values ​​of t, different dependent variable values ​​on the same straight line are obtained.

[0068] Step P3, substitute the three-dimensional coordinates (x1i, y1i, z1i) into the linear parametric equation, solve to obtain the coordinate coefficients a1, b1 and c1, and then construct the direction vector (a1, b1, c1) to determine the direction of the straight line to be fitted in the camera coordinate system, that is, the direction vector (a1, b1, c1) of the first line laser point set in the camera coordinate system disclosed in step A2. Alternatively, substitute the three-dimensional coordinates (x2i, y2i, z2i) into the linear parametric equation, solve to obtain the coordinate coefficients a2, b2 and c2, and then construct the direction vector (a2, b2, c2) to determine the direction of the straight line to be fitted in the camera coordinate system, that is, the direction vector (a2, b2, c2) of the laser mapping point set in the camera coordinate system disclosed in step C3.

[0069] In step P3, the three-dimensional coordinates are substituted into the straight line parameter equation. The solution method of the straight line parameter equation, that is, the straight line fitting method, can adopt a variety of methods such as least squares method and RANSAC, which can be selected according to actual needs without limitation.

[0070] In this embodiment, the three coordinate coefficients obtained by fitting in the aforementioned straight line parameter equation can be regarded as the moving speeds of the straight line to be fitted on the X, Y, and Z coordinate axes of the camera coordinate system, so the three coordinate coefficients obtained by fitting jointly determine the direction of the straight line to be fitted.

[0071] In summary, the three-dimensional coordinates of the laser point in the camera coordinate system XYZ disclosed in this embodiment itself contain three sets of corresponding information, and the straight line direction vectors of different calibration plate planes in the camera coordinate system can be established by fitting, which can be directly solved. The assembly angle error corresponding to the yaw angle calculated in step A and the roll angle calculated in step C comes from the acquisition of the fitting straight line information, but the error will not be accumulated and amplified due to repeated data acquisition, and the final calibration error is smaller than the prior art.

[0072] Preferably, the method for determining whether a line laser point set formed by a line laser transmitter is valid comprises: determining within the line laser point set whether a ratio of the number of laser points in the same straight line to the total number of laser points in the line laser point set is greater than a preset ratio threshold, and if so, confirming that the line laser point set is valid, wherein the line laser point set is reflected from the measured surface of a horizontal calibration plate or the measured surface of a vertical calibration plate, and the preset ratio threshold is preferably 60%.

[0073] Preferably, the method for determining whether the line laser point set formed by the line laser transmitter is valid also includes: determining one by one whether each laser point in the line laser point set conforms to a preset standard data format, and if so, confirming that the line laser point set is valid; and detecting whether the data structure constituting the line laser point set is normal, and if there is garbled code or noise, the line laser point set is invalid.

[0074] The present application also discloses a calibration device, such as Figure 6 As shown, the calibration device includes a calibration board control module, a sensor fixing module, a horizontal calibration board, a vertical calibration board, a point set acquisition module, and an external parameter calculation module. The sensor fixing module is used to fix the position of the line laser module, which is generally a fixture that can fix the robot or the line laser module.

[0075] The calibration plate control module is used to set the position of the horizontal calibration plate and the position of the vertical calibration plate to control the position of the horizontal calibration plate and the position of the vertical calibration plate to be within the detection range of the line laser module. The point set acquisition module can collect the line laser point set of the measured surface of the horizontal calibration plate and / or the line laser point set within the measured surface of the vertical calibration plate.

[0076] The point set acquisition module is used to acquire the line laser point set formed by the horizontal calibration plate being emitted by the line laser module. The line laser point set reflected back by the horizontal calibration plate can be stored in the form of image pixels.

[0077] The calibration plate control module is further used to control the vertical calibration plate to move relative to the fixed line laser module through at least two calibration positions according to step B in the external parameter calibration method, so that the line laser point set formed by the line laser module emitting to the vertical calibration plate is within its detection range. The point set acquisition module is used to acquire the line laser point set formed by the line laser module emitting to the vertical calibration plate at different calibration positions according to step B.

[0078] The calibration plate control module includes a slide rail and a motor that drives the vertical calibration plate movement ( Figure 6 (not shown), the sensor fixing module fixes the line laser module at one end of the slide rail, and the motor drives the vertical calibration plate to move along the slide rail relative to the fixed line laser module through at least two calibration positions, and the point set acquisition module collects at least two groups of line laser point sets formed by the line laser reflected in the vertical calibration plate.

[0079] The external parameter calculation module is used to calculate the calibration external parameters of the line laser module according to the external parameter calibration method (including step A, step B and step C), using the line laser point set obtained by the point set acquisition module (including the line laser point set from the vertical calibration plate and the line laser point set from the horizontal calibration plate), including the rotation matrix corresponding to the yaw angle, the rotation matrix corresponding to the pitch angle, the rotation matrix corresponding to the roll angle and the calibration height of the line laser module.

[0080] Preferably, the calibration device also includes a storage module for writing the calibration external parameters calculated above into a corresponding storage unit, wherein the storage unit for storing the external parameters may be located in a memory on the robot or in a memory in the line laser module, without limitation.

[0081] In summary, the calibration device only needs to collect the line laser point set emitted by the line laser transmitter in the vertical calibration plate and the horizontal calibration plate, without introducing additional target image processing and calculation of target coordinates in the robot coordinate system. The line laser point set formed by the line laser transmitter emitting to the vertical calibration plate is used to calculate the yaw angle and pitch angle, and the assembly angle error of the line laser module in the two dimensions of yaw angle and pitch angle is obtained. Then, based on the corresponding constraint relationship, the line laser point set formed by the line laser transmitter emitting to the horizontal calibration plate is used to calculate the roll angle, and thus the assembly angle error information of the line laser module in three degrees of freedom is obtained, providing a rotation angle reference in three degrees of freedom for the correction of the line laser module. On this basis, the angle error caused by the assembly error of the line laser module is overcome by performing coordinate transformation on the line laser point set, which is sufficient to solve the calibration requirements of the line laser module in scenarios where the precision requirements are not high. The calibration process is simpler than the calibration process of the target pattern, and the amount of calculation is small.

[0082] Based on the foregoing embodiments, the present application also discloses a calibration system, which includes a robot and an external parameter calibration device, wherein the external parameter calibration device includes a memory and a processor; the memory has a control program for necessary peripheral devices and a line laser external parameter calibration program, and the external parameter calibration device preferably includes a calibration device, and the processor includes an external parameter calculation module. When the processor runs the line laser external parameter calibration program, it executes the external parameter calibration method disclosed in the above-mentioned embodiment, and can obtain the rotation matrix corresponding to the yaw angle, the rotation matrix corresponding to the pitch angle, and the rotation matrix corresponding to the roll angle according to the angle information calculated in steps A to C, and construct the constraint relationship or corresponding projection relationship between the coordinate systems of the line laser point set formed by the line laser transmitter emitting to the vertical calibration plate and the line laser point set formed by the line laser transmitter emitting to the horizontal calibration plate, which also corresponds to the constraint relationship between the straight line direction vectors of different planes scanned by the line laser module. Then, the line laser point set output by the line laser module can be transformed in coordinates through the rotation matrix corresponding to the yaw angle, the rotation matrix corresponding to the pitch angle, and the rotation matrix corresponding to the roll angle, so that the final point set result is close to the ideal calibration value, thereby completing the overall calibration process of the line laser module.

[0083] The above embodiments are only for illustrating the technical concept and features of the present invention, and their purpose is to enable technicians familiar with the technical field to understand the content of the present invention and implement it accordingly, and they cannot be used to limit the protection scope of the present invention. Any equivalent changes or modifications made according to the spirit of the present invention should be included in the protection scope of the present invention.

Claims

1. An external parameter calibration method for a line laser module, wherein the line laser module is fixedly mounted on a robot, and the line laser module includes a camera and a line laser transmitter; the method is characterized in that: There are vertically arranged vertical calibration plates and horizontal calibration plates on the outside of the robot, wherein neither the vertical calibration plates nor the horizontal calibration plates are provided with target patterns; both the vertical calibration plates and the horizontal calibration plates are installed within the detectable range of the line laser module; The external parameter calibration method comprises the following steps: Step A, calculating the yaw angle based on a line laser point set formed by a line laser transmitter emitting light toward a vertical calibration plate; Step B, calculating the pitch angle based on the displacement of the vertical calibration plate relative to the line laser module; Step C, based on the yaw angle described in step A and the pitch angle described in step B, coordinate transformation is performed on the line laser point set formed by the line laser transmitter emitting to the horizontal calibration plate to obtain a laser mapping point set, and then the roll angle is calculated using the laser mapping point set; Step D: Use the roll angle to transform the coordinates of the laser mapping point set to obtain a calibration horizontal plane point set, and then mark the average height of the calibration horizontal plane point set as the calibration external parameter of the line laser module.

2. The external parameter calibration method according to claim 1, characterized in that: The external parameter calibration method further includes: when the calibration horizontal plane point set is obtained by transformation in step C, the rotation matrix corresponding to the yaw angle, the rotation matrix corresponding to the pitch angle and the rotation matrix corresponding to the roll angle are all marked as calibration external parameters of the line laser module; In the line laser module, a camera coordinate system is set inside the camera, a line laser transmitter emits to form a laser plane, and a laser point set is configured to be located in the laser plane.

3. The external parameter calibration method according to claim 2, characterized in that: Before executing step C, when the robot is fixedly placed on a horizontal plane, the line laser transmitter emits a line laser into the measured surface of the horizontal calibration plate, and the camera collects a line laser point set formed by the line laser reflected on the measured surface of the horizontal calibration plate, and obtains a line laser point set formed by the line laser transmitter emitting to the horizontal calibration plate; wherein the measured surface of the horizontal calibration plate is parallel to the horizontal plane; Before executing step A, with the robot fixed, the line laser transmitter transmits a line laser toward the vertical calibration plate, the camera collects a line laser point set formed by the line laser being reflected in the vertical calibration plate, and obtains a line laser point set formed by the line laser transmitter transmitting toward the vertical calibration plate; wherein, the horizontal forward direction of the robot is parallel to the normal vector of the measured surface of the vertical calibration plate, and the body width direction of the robot is perpendicular to the horizontal forward direction of the robot.

4. The external parameter calibration method according to claim 3, characterized in that: In the camera coordinate system, the first coordinate axis is set to be parallel to the normal vector of the measured surface of the vertical calibration plate; wherein the positive direction of the first coordinate axis represents the horizontal forward direction of the robot, so as to point to the front of the line laser transmitter; the second coordinate axis is parallel to the width direction of the robot body, and the width direction of the robot body uses the direction in which the wheel axle of the robot crosses the body; In the camera coordinate system, the first coordinate plane is set as the robot horizontal coverage plane, wherein the robot horizontal coverage plane is parallel to the horizontal plane, and the coordinate plane formed by the intersection of the first coordinate axis and the second coordinate axis is the first coordinate plane; In the camera coordinate system, the second coordinate plane is set as a plane perpendicular to the horizontal forward direction of the robot, wherein the coordinate plane formed by the intersection of the second coordinate axis and the third coordinate axis is the second coordinate plane; The coordinate plane formed by the intersection of the first coordinate axis and the third coordinate axis is the third coordinate plane; the first coordinate plane, the second coordinate plane and the third coordinate plane intersect perpendicularly at the origin of the camera coordinate system.

5. The external parameter calibration method according to claim 4, characterized in that: In the camera coordinate system XYZ, the first coordinate axis is the Z axis, the second coordinate axis is the X axis, the third coordinate axis is the Y axis, the first coordinate plane is the XOZ coordinate plane, the second coordinate plane is the XOY coordinate plane, and the third coordinate plane is the YOZ coordinate plane.

6. The external parameter calibration method according to claim 4, characterized in that: In step A, the method for calculating the yaw angle based on the line laser point set formed by the line laser transmitter emitting to the vertical calibration plate includes: Step A1, after confirming that the line laser point set formed by the line laser transmitter emitting to the vertical calibration plate is valid, marking the valid line laser point set as the first line laser point set; Step A2, obtaining the direction vector (a1, b1, c1) of the first line laser point set in the camera coordinate system by fitting; Step A3: Calculate the angle between the projection vector of the direction vector (a1, b1, c1) in the first coordinate plane and the second coordinate axis. Then the angle θ Y Labeled as yaw angle.

7. The external parameter calibration method according to claim 5, characterized in that: In step B, the method for calculating the pitch angle based on the displacement of the vertical calibration plate relative to the line laser module includes: When the line laser module and the robot are both fixed, the line laser emitter is controlled to emit the line laser toward the vertical calibration plate, and the vertical calibration plate is controlled to translate relative to the line laser module in the detection range of the line laser module in sequence, and whenever the vertical calibration plate translates a preset distance Δd to a calibration position, the camera collects a set of line laser point sets formed by the reflection of the line laser in the vertical calibration plate, and extracts the height change value Δh generated by the corresponding line laser point set in the vertical calibration plate before and after the translation; Then calculate the relative offset angle of the laser plane formed by the line laser transmitter relative to the horizontal forward direction of the robot. Then the relative offset angle θ P Marked as the pitch angle; where θ0 is the emission angle of the line laser transmitter in the camera coordinate system, which is used to represent the angle between the intersection line between the laser plane and the third coordinate plane and the first coordinate axis; The detection range of the line laser module includes the distance range between the minimum detection distance of the camera and the maximum detection distance of the camera; The vertical calibration plate is configured to translate along the slide rail, and the robot is configured to be fixed at one end of the slide rail.

8. The external parameter calibration method according to claim 5, characterized in that: In step C, based on the yaw angle described in step A and the pitch angle described in step B, coordinate transformation is performed on the laser point set formed by the line laser transmitter emitting to the horizontal calibration plate to obtain a laser mapping point set, and then the method of calculating the roll angle using the laser mapping point set includes: Step C1, after confirming that the line laser point set formed by the line laser transmitter emitting to the horizontal calibration plate is valid, marking the valid line laser point set as the second line laser point set; Step C2, using the rotation matrix corresponding to the yaw angle described in step A and the rotation matrix corresponding to the pitch angle described in step B to control the coordinate transformation of the second line laser point set to obtain a laser mapping point set; Step C3, obtaining the direction vector (a2, b2, c2) of the laser mapping point set in the camera coordinate system by fitting; Step C4: Calculate the angle between the projection vector of the direction vector (a2, b2, c2) in the second coordinate plane and the second coordinate axis. Then the angle θ R Labeled as roll angle.

9. The external parameter calibration method according to claim 8, characterized in that: In step D, the method of using the roll angle to transform the coordinates of the laser mapping point set to obtain a calibration horizontal plane point set, and then marking the average height of the calibration horizontal plane point set as the calibration external parameter of the line laser module includes: Multiplying the laser mapping point set by the rotation matrix corresponding to the roll angle to obtain a calibration horizontal plane point set, and determining that the laser mapping point set completes the coordinate transformation; Then, in the calibration horizontal plane point set, the average coordinate values ​​of all points on the third coordinate axis are calculated to obtain the average height of the calibration horizontal plane point set, and then the average height of the calibration horizontal plane point set is marked as the calibration height of the line laser module, and then the calibration height of the line laser module is marked as the calibration external parameter of the line laser module.

10. The external parameter calibration method according to claim 6 or 8, characterized in that: Methods for obtaining direction vectors by fitting include: Step P1, obtaining the three-dimensional coordinates (x1i, y1i, z1i) of at least three non-collinear laser points on the measured surface of the vertical calibration plate in the camera coordinate system XYZ; or obtaining the three-dimensional coordinates (x2i, y2i, z2i) of at least three non-collinear laser points on the measured surface of the horizontal calibration plate after the coordinate transformation of the rotation matrix corresponding to the yaw angle and the rotation matrix corresponding to the pitch angle; wherein i=1, 2, ..., n, and n is greater than or equal to 3; Step P2: Set the linear parameter equation of the laser plane in the camera coordinate system XYZ Where (x0, y0, z0) is the three-dimensional coordinate of the fixed point through which the straight line to be fitted passes; t represents the distance that a point in the straight line to be fitted moves on the direction vector; the straight line parameter equation represents the fitting equation required for the straight line to be fitted in the laser plane; Step P3, substitute the three-dimensional coordinates (x1i, y1i, z1i) into the straight line parametric equation, solve to obtain the coordinate coefficients a1, b1 and c1, and then construct the direction vector (a1, b1, c1) to determine the direction of the straight line to be fitted in the camera coordinate system; or substitute the three-dimensional coordinates (x2i, y2i, z2i) into the straight line parametric equation, solve to obtain the coordinate coefficients a2, b2 and c2, and then construct the direction vector (a2, b2, c2) to determine the direction of the straight line to be fitted in the camera coordinate system.

11. The external parameter calibration method according to claim 6 or 8, characterized in that: Methods for determining whether the line laser point set formed by the line laser transmitter is valid include: In the line laser point set, it is determined whether the ratio of the number of laser points in the same straight line to the total number of laser points in the line laser point set is greater than a preset ratio threshold, and if so, it is confirmed that the line laser point set is valid.

12. The calibration device is characterized in that: The calibration device includes a calibration plate control module, a sensor fixing module, a horizontal calibration plate, a vertical calibration plate, a point set acquisition module, and an external parameter calculation module; A sensor fixing module is used to fix the position of the line laser module; A calibration plate control module is used to set the position of the horizontal calibration plate and the position of the vertical calibration plate to control the position of the horizontal calibration plate and the position of the vertical calibration plate to be within the detection range of the line laser module; A point set acquisition module, used to acquire a line laser point set formed by the horizontal calibration plate being emitted by the line laser module; The calibration plate control module is further used to control the vertical calibration plate to move through at least two calibration positions relative to the fixed line laser module according to step B of the external parameter calibration method according to any one of claims 1 to 9, so that the line laser module emits a line laser point set formed by the vertical calibration plate within its detection range; A point set acquisition module, used to acquire, according to step B, a line laser point set formed by the vertical calibration plate being emitted by the line laser module at different calibration positions; An external parameter calculation module, used to calculate the calibration external parameters of the line laser module using the line laser point set acquired by the point set acquisition module according to the external parameter calibration method according to any one of claims 1 to 9; The calibration device further includes a slide rail, the calibration plate control module controls the vertical calibration plate to translate along the slide rail, and the sensor fixing module fixes the line laser module at one end of the slide rail.

13. The calibration system is characterized in that The calibration system includes a robot and an external parameter calibration device, wherein the external parameter calibration device includes a memory and a processor; The memory has a control program for necessary peripheral devices and a line laser external parameter calibration program, and the processor executes the external parameter calibration method according to any one of claims 1 to 9 when running the line laser external parameter calibration program.

Images