Multi-sensor pose calibration method, device and system

Abstract

The application discloses a multi-sensor pose calibration method, device and system. The method comprises the following steps: for a plurality of plane marker plates with different positions and directions, determining the position information of laser points projected onto each plane marker plate in a radar coordinate system; for each laser point in the laser points, establishing a first correlation between the position information of the laser point in the radar coordinate system and the position information of the laser point in a vision sensor coordinate system through the extrinsic parameters of the vision sensor and the laser radar sensor; for each laser point of the laser points, constructing a distance residual model of the laser point to the corresponding plane marker plate according to the first correlation; and weighting the distance residuals of the laser points, so as to obtain the extrinsic parameters of the vision sensor and the laser radar sensor when the weighting result is the minimum. The technical scheme provided by the application can save the pose calibration time of the sensor, is simple and easy to operate, and is high in precision and stability.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a multi-sensor posture calibration method, device and system. Background Art

[0002] In robotic applications, multi-sensor fusion positioning technology is often used for robot navigation and positioning. However, the position and posture relationships between different sensors need to be known in advance, which requires calibration of the position and posture relationships between different sensors.

[0003] The calibration method used in related technologies requires manual or automatic selection of matching points between the image and the laser, and then uses the distance between the points and the plane to minimize the position and posture relationship between the camera and the lidar sensor. However, the above calibration method has the following problems:

[0004] 1. Point-on-a-plane methods require manual selection of laser points projected onto the image, making the process complex and cumbersome. Even when manual selection of laser points is not required, the calibration environment often has very strict requirements. For example, during calibration, the calibration must be performed with no objects other than the calibration plate in a specific area, otherwise the laser points projected onto the image may be misdetected.

[0005] 2. The method based on image feature and laser feature matching requires manual selection of image feature points corresponding to the laser points projected onto the image, which is tedious and complicated.

[0006] 3. The position and angle of the marking plate need to be moved and changed multiple times, which is difficult to implement in mass production.

[0007] 4. The image captured by the camera may be blurred or have ghosting during the process of moving the marking plate, which may cause inaccurate image data and affect the accuracy.

[0008] Therefore, in response to the above problems, there is an urgent need to provide a new calibration solution for the position and posture relationship between the lidar sensor and the visual sensor (for example, a binocular camera) to solve at least one of the above problems. Summary of the Invention

[0009] The main purpose of the present invention is to disclose a multi-sensor posture calibration method, device and system to at least solve the problems of the calibration scheme of the position and posture relationship between the lidar sensor and the binocular camera in the related art, which is cumbersome and complicated to operate, has strict requirements on the calibration environment, and the observed image data is inaccurate, thereby affecting the accuracy.

[0010] According to one aspect of the present invention, a multi-sensor pose calibration method is provided.

[0011] The multi-sensor posture calibration method according to the present invention includes: for a plurality of plane marker plates with different position and direction settings, determining the position information of the laser points projected onto each of the above plane marker plates in the radar coordinate system; for each of the above laser points, establishing a first association relationship between the position information of the laser point in the radar coordinate system and the position information of the laser point in the visual sensor coordinate system through the external parameters of the visual sensor and the laser radar sensor; for each of the above laser points, constructing a model of the distance residual from the laser point to the corresponding plane marker plate according to the above first association relationship; weighting the distance residual corresponding to each of the above laser points, and when the weighted result is the minimum, obtaining the external parameters of the above visual sensor and the above laser radar sensor.

[0012] According to another aspect of the present invention, a multi-sensor posture calibration device is provided.

[0013] The multi-sensor posture calibration device according to the present invention includes: a determination module, which is used to determine the position information of the laser point projected onto each of the plane marker plates with different position and direction settings in the radar coordinate system; an association module, which is used to establish, for each of the above laser points, a first association relationship between the position information of the laser point in the radar coordinate system and the position information of the laser point in the visual sensor coordinate system through the external parameters of the visual sensor and the lidar sensor; a construction module, which is used to construct, for each of the above laser points, a model of the distance residual from the laser point to the corresponding plane marker plate according to the above first association relationship; a calibration module, which is used to weight the distance residual corresponding to each of the above laser points, and obtain the external parameters of the above visual sensor and the above lidar sensor when the weighted result is minimum.

[0014] According to yet another aspect of the present invention, a multi-sensor posture calibration system is provided.

[0015] The multi-sensor posture calibration system according to the present invention includes: a posture calibration device as described in any one of the above items, and also includes: a plurality of planar marker plates, the positions and directions of the plurality of planar marker plates are all set differently; a lidar sensor for emitting lasers to the plurality of planar marker plates and collecting laser data including distance information and angle information; a visual sensor for collecting image information, wherein the image information includes relevant image information of marker points on the plurality of planar marker plates.

[0016] According to the present invention, multiple planar marker plates with different positions and orientations are set up, and the position information of the laser points projected onto each of the planar marker plates in the radar coordinate system is determined. For each of the laser points, a first correlation relationship is established between the position information of the laser point in the radar coordinate system and the position information of the laser point in the visual sensor coordinate system using the external parameters of the visual sensor and the lidar sensor. For each of the laser points, a model of the distance residual from the laser point to the corresponding planar marker plate is constructed based on the first correlation relationship. The distance residuals corresponding to each of the laser points are weighted, and when the weighted result is minimized, the external parameters of the visual sensor and the lidar sensor are obtained. Using the above method, there is no need to manually select matching points between the image and the laser, and there is no need to move the marker plate or the reference system. This method can greatly save the time of sensor posture calibration, is simple and easy to operate, and has high accuracy and stability. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a flowchart of a multi-sensor pose calibration method according to an embodiment of the present invention;

[0018] Figure 2 is a flow chart of a multi-sensor pose calibration method according to a preferred embodiment of the present invention;

[0019] Figure 3 is a structural block diagram of a multi-sensor posture calibration device according to an embodiment of the present invention;

[0020] Figure 4 1 is a structural block diagram of a multi-sensor posture calibration device according to a preferred embodiment of the present invention;

[0021] Figure 5 4 is a structural block diagram of a multi-sensor posture calibration system according to a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0022] The specific implementation of the present invention is described in detail below with reference to the accompanying drawings.

[0023] According to an embodiment of the present invention, a multi-sensor pose calibration method is provided.

[0024] Figure 1 FIG. 1 is a flow chart of a multi-sensor pose calibration method according to an embodiment of the present invention. Figure 1 As shown, the multi-sensor pose calibration method includes:

[0025] Step S101: for a plurality of planar marker plates having different positions and directions, determining the position information of the laser points projected onto each of the planar marker plates in the radar coordinate system;

[0026] Step S102: For each of the laser points, establish a first association relationship between position information of the laser point in the radar coordinate system and position information of the laser point in the visual sensor coordinate system using external parameters of the visual sensor and the lidar sensor;

[0027] Step S103: for each of the laser points, construct a model of the distance residual between the laser point and the corresponding plane marking plate according to the first association relationship;

[0028] Step S104: weighting the distance residuals corresponding to the respective laser points in the laser points, and obtaining the external parameters of the visual sensor and the lidar sensor when the weighted result is the smallest.

[0029] In related technologies, it is necessary to manually / automatically select matching points between the image and the laser, and then use the distance from the point to the plane to minimize the position and posture relationship between the camera and the lidar sensor. The operation is tedious and complicated, and the calibration environment is strictly required. The observed image data is inaccurate, which affects the accuracy. Figure 1 The method shown sets up multiple planar markers with different positions and orientations, determines the position information of the laser points projected onto each of the planar markers in the radar coordinate system, and for each of the laser points, establishes a first correlation between the position information of the laser point in the radar coordinate system and the position information of the laser point in the visual sensor coordinate system using the external parameters of the visual sensor and the lidar sensor. For each of the laser points, a model of the distance residual from the laser point to the corresponding planar marker is constructed based on the first correlation. The distance residuals corresponding to each of the laser points are weighted, and when the weighted result is minimized, the external parameters of the visual sensor and the lidar sensor are obtained. Therefore, there is no need to manually select matching points between the image and the laser, and there is no need to move the marker or reference system, which can greatly save sensor pose calibration time. The method is simple and easy to operate, and has high accuracy and stability.

[0030] Of course, the marker on the above-mentioned plane marking plate can be an April tag or other types of markers.

[0031] Preferably, in the above step S101, determining the position information of the laser points projected onto each of the above-mentioned plane marking plates in the radar coordinate system may further include: for each of the above-mentioned plane marking plates, determining the boundary laser points projected onto the plane marking plate and the position information (for example, coordinate information) of the above-mentioned boundary laser points in the radar coordinate system; determining other laser points projected onto the plane marking plate except the above-mentioned boundary laser points based on the above-mentioned boundary laser points, as well as the position information of the above-mentioned other laser points in the radar coordinate system.

[0032] For example, a 2D LiDAR sensor emits laser light onto a flat marker board. The laser points projected onto the board form a line from left to right (the spacing between adjacent laser points is small, the laser points are densely packed, and can be considered a continuous line). The leftmost and rightmost laser points of the line are the boundary laser points on the board. Among the laser points projected onto the board, excluding the leftmost and rightmost laser points, the remaining laser points are located midway between the left and right boundary laser points.

[0033] In the preferred implementation process, the 2D lidar-scan in the radar coordinate system is converted into a corresponding laser point cloud, and the laser point cloud projected onto the plane marking plate is determined. The laser points projected onto the marking plate are determined based on the boundary laser points identified on the marking plate. For example, among the linear laser points projected onto the plane marking plate, the leftmost laser point A is represented by (100, p1, θ1), and the rightmost laser point B is represented by (150, p n ,θ n ), then the laser points between these two points include: (101, p2, θ2), (102, p3, θ3), ... (149, p n-1 ,θ n-1 ). For the laser radar sensor, based on the polar coordinate information of these known points, the position information of the laser point projected onto the plane marker board in the radar coordinate system can be calculated.

[0034] Preferably, the first association relationship includes but is not limited to:

[0035] P i c =T cl *P i l (1)

[0036] Among them, the above T cl are the external parameters of the above-mentioned visual sensor and the above-mentioned lidar sensor, and the above-mentioned P i l is the position information of the laser point in the radar coordinate system, the above P i c is the position information of the laser point in the visual sensor coordinate system.

[0037] Preferably, in the above step S103, for each of the above laser points, constructing a model of the distance residual from the laser point to the corresponding plane marker plate according to the above first association relationship can further include: establishing a second association relationship between the first plane equation of the plane marker plate corresponding to the laser point in the plane marker plate coordinate system and the second plane equation of the plane marker plate in the visual sensor coordinate system; and constructing a model of the distance residual from the laser point to the above corresponding plane marker plate according to the above first association relationship and the above second association relationship.

[0038] Preferably, the above-mentioned second association relationship includes but is not limited to:

[0039] n^c=(T ca -1 ) T *n^a (2)

[0040] Wherein, the above n^a is the first plane equation of the plane marking plate corresponding to the laser point in the plane marking plate coordinate system, the above n^c is the second plane equation of the plane marking plate corresponding to the laser point in the visual sensor coordinate system, T ca Represents the transformation relationship between the marker point on the plane marker plate and the camera coordinate system, (T ca -1 ) T Indicates T ca The transposed matrix of the inverse matrix of .

[0041] In the preferred implementation process, in order to obtain the external parameter Tcl of the visual sensor and the lidar sensor (i.e., the position and posture relationship between the lidar sensor and the binocular camera), a first association relationship between the position information of the laser point in the radar coordinate system and the position information of the laser point in the visual sensor coordinate system can be established: P i c =T cl *P i l , the laser points (point cloud) projected onto the plane marker board are passed through T from the radar coordinate system cl Transform to the camera coordinates. Then, a second correlation relationship can be established between the first plane equation of the plane marker corresponding to the laser point in the plane marker coordinate system and the second plane equation of the plane marker in the visual sensor coordinate system: n^c=(T ca -1 )T*n^a, where T ca Represents the transformation relationship between the marker point on the plane marker plate and the camera coordinate system, T caThe method in the related art can be used to obtain it. For example, the four outer corner points of the marker boundary on the plane marker plate can be extracted as the initial corner points for pose estimation. The pose information between the marker and the visual sensor (for example, a camera) can be estimated based on the homography mapping relationship. Assuming that the normal vector n^a of the plane marker plate corresponding to the current laser point is represented by [0,0,1,0] (that is, the first plane equation mentioned above), then according to the second association relationship mentioned above, the second plane equation n^c of the plane marker plate corresponding to the laser point in the visual sensor coordinate system can be calculated.

[0042] Preferably, in step S104, the external parameters of the visual sensor and the lidar sensor can be obtained by the following formula:

[0043]

[0044] Wherein, the above j represents the number of the above multiple plane marking plates, the above N is the number of the above multiple plane marking plates, the above i represents the number of the laser points projected onto the current plane marking plate, the above M is the number of the laser points projected onto the current plane marking plate, and d ij (n^c,P i c ) represents the model of the distance residual from the laser point to the corresponding plane marking plate, and the external parameters T of the above-mentioned visual sensor and the above-mentioned lidar sensor are cl is (R, t), where T represents the objective function.

[0045] In the process of optimal implementation, we can first construct the error equation and set the distance residual r i is the distance from the current laser point to the plane marking plate, r i =d(n^c,P i c ), where d ij (n^c,P i c )=(P i c ) T *n^c; According to the first association relationship (e.g., formula (1)) and the second association relationship (e.g., formula (2)), formula (4) can be obtained:

[0046] d ij (n^c,P i c )=(T cl *P i l ) T *(T ca -1 ) T *n^a (4)

[0047] It should be noted that for each laser point on each plane marking plate, an error equation can be constructed, and then the distances in all the constructed error equations can be accumulated and weighted. According to equations (3) and (4), when the weighted sum (objective function T) in equation (3) is minimized, the external parameter T of the visual sensor and the above-mentioned lidar sensor can be calculated. cl .

[0048] The following combination Figure 2 The above preferred embodiments are further described.

[0049] Figure 2 FIG. 1 is a flow chart of a multi-sensor posture calibration method according to a preferred embodiment of the present invention. Figure 2 As shown, the multi-sensor pose calibration method includes:

[0050] Step S201: converting the 2D lidar-scan in the radar coordinate system into the corresponding laser point cloud;

[0051] Step S202: Identify the laser point cloud (multiple laser points) projected onto the current plane marking plate, and determine other laser points projected onto the plane marking plate based on the identified boundary laser points on the current marking plate;

[0052] Step S203: The laser point cloud (multiple laser points) projected onto the plane marker plate is moved from the radar coordinate system through T cl Transform to camera coordinates.

[0053] Step S204: The normal vector n^a of the current plane marker plate in the plane marker plate coordinate system is represented by [0, 0, 1, 0] (i.e., the plane equation of the plane marker plate), and n^a is converted to the camera coordinate system and represented as n^c.

[0054] Step S205: Construct an error equation, where the distance residual r i is the distance from the laser point to the plane of the planar marker plate, which can be represented by the following distance residual model: i =d(n^c,P i c )

[0055] It should be noted that if a total of N planar marker plates with different positions and directions are set in the pose calibration system, the above steps S201 to S205 need to be executed for each marker plate.

[0056] Step S206: For all the above-mentioned plane marking plates, weight the distance residual corresponding to each laser point among all the laser points projected onto all the plane marking plates. When the weighted result is the minimum, the external parameter T of the above-mentioned visual sensor and the above-mentioned lidar sensor is obtained. cl .

[0057] According to an embodiment of the present invention, a multi-sensor posture calibration device is also provided.

[0058] Figure 3 : is a structural block diagram of a multi-sensor posture calibration device according to an embodiment of the present invention. Figure 3 As shown, the multi-sensor posture calibration device includes: a determination module 30, which is used to determine the position information of the laser point projected onto each of the plane marker plates with different position and direction settings in the radar coordinate system; an association module 32, which is used to establish, for each of the above laser points, a first association relationship between the position information of the laser point in the radar coordinate system and the position information of the laser point in the visual sensor coordinate system through the external parameters of the visual sensor and the lidar sensor; a construction module 34, which is used to construct, for each of the above laser points, a model of the distance residual from the laser point to the corresponding plane marker plate according to the above first association relationship; a calibration module 36, which is used to weight the distance residual corresponding to each of the above laser points, and when the weighted result is the minimum, obtain the external parameters of the above visual sensor and the above lidar sensor.

[0059] For multiple plane marking plates with different settings and directions, the determination module 30 determines the position information of the laser points projected onto each of the above plane marking plates in the radar coordinate system. The association module 32 establishes a first association relationship between the position information of the laser point in the radar coordinate system and the position information of the laser point in the visual sensor coordinate system for each of the above laser points through the external parameters of the visual sensor and the lidar sensor. The construction module 34 constructs a model of the distance residual from the laser point to the corresponding plane marking plate according to the first association relationship for each of the above laser points. The calibration module 36 weights the distance residual corresponding to each of the above laser points. When the weighted result is the smallest, the external parameters of the above visual sensor and the above lidar sensor are obtained. Figure 3 The device shown does not require manual selection of matching points between the image and the laser, and does not require moving the marking plate or the reference system, which can greatly save the sensor pose calibration time. It is simple and easy to operate, and has high accuracy and stability.

[0060] Preferably, if Figure 4As shown, the determination module 30 may further include: a first determination unit 300, for determining, for each of the above-mentioned plane marking plates, the boundary laser points projected onto the plane marking plate and the position information of the above-mentioned boundary laser points in the radar coordinate system; a second determination unit 302, for determining, based on the above-mentioned boundary laser points, other laser points projected onto the plane marking plate except the above-mentioned boundary laser points, and the position information of the above-mentioned other laser points in the radar coordinate system.

[0061] Preferably, if Figure 4 As shown, the above-mentioned construction module 34 may further include: an association unit 340, used to establish a second association relationship between the first plane equation of the plane marker plate corresponding to the laser point in the plane marker plate coordinate system and the second plane equation of the plane marker plate in the visual sensor coordinate system; a construction unit 342, used to construct a model of the distance residual from the laser point to the corresponding plane marker plate based on the above-mentioned first association relationship and the above-mentioned second association relationship.

[0062] It should be noted that the preferred embodiment of the combination of each module and each unit in the above multi-sensor posture calibration device can be specifically referred to Figures 1 to 2 The description is not repeated here.

[0063] According to an embodiment of the present invention, a multi-sensor posture calibration system is also provided.

[0064] Figure 5 : is a structural block diagram of a multi-sensor posture calibration system according to a preferred embodiment of the present invention. Figure 5 As shown, the multi-sensor posture calibration system includes, in addition to the multi-sensor posture calibration device 50 described in any one of the above items (the posture calibration device 50 is coupled with the lidar sensor 54 and the visual sensor 56 respectively), a plurality of plane marker plates 52, the positions and directions of which are different; a lidar sensor 54 for emitting lasers to the plurality of plane marker plates and collecting laser data including distance information and angle information; a visual sensor 56 for collecting image information, wherein the image information includes relevant image information of the marker points on the plurality of plane marker plates.

[0065] In the preferred implementation process, before performing the posture calibration of the sensor, it is necessary to build a posture calibration system. In addition to the posture calibration device of the multi-sensor mentioned above, the system also needs to be configured with the following devices: a laser radar sensor (e.g., a two-dimensional (2D) laser radar sensor) and a visual sensor (e.g., a camera, a multi-eye module, etc.). The laser radar sensor and the visual sensor can be installed on the vehicle, and the laser radar sensor and the visual sensor are provided with a large common viewing area. In addition, it is necessary to arrange multiple (e.g., 6) plane markers in different positions and directions within a predetermined range in front of the vehicle (e.g., within a range of 1 to 3 meters). For example, two plane markers in different directions are set at different positions at 1 meter, two plane markers in different directions are set at different positions at 2 meters, and two plane markers in different directions are set at different positions at 3 meters. The plane markers are provided with markers, and the markers include but are not limited to: April tag markers, random point markers, etc.

[0066] The above-mentioned posture calibration device 50 can be provided with a plurality of operation buttons, for example, an acquisition button and a calibration button. When the user triggers the acquisition button, the posture calibration device 50 can collect data observed by the lidar sensor and the visual sensor, for example, relevant image information of marker points on multiple plane marking plates, including laser data of distance information and angle information, etc. When the user triggers the calibration button, the posture calibration device 50 can execute the above-mentioned posture calibration method according to the collected information to achieve the purpose of "one-button" calibration. The posture calibration system using the above-mentioned multi-sensor does not require manual selection of matching points between the image and the laser, does not require moving the marking plate or the reference system (such as a robot), and is suitable for the calibration of factory robots (which require the position and posture relationship between the laser and the camera). It is easy to operate, has high precision, is stable, and is convenient for mass production.

[0067] To sum up, with the help of the above-mentioned embodiments provided by the present invention, there is no need to manually select the laser points projected onto the image, and the laser points projected onto the image can be automatically detected, which is not subject to harsh environmental calibration requirements; there is no need to manually or mechanically move the plane marking plate, and after the multi-sensor posture calibration system is arranged, data collection and sensor posture calibration can be efficiently realized; and the calibration environment is simply arranged, which is suitable for large-scale calibration in factory workshops; in addition, the multi-sensor posture calibration system provided by the present invention is highly accurate and stable.

[0068] The above disclosures are only a few specific embodiments of the present invention. However, the present invention is not limited thereto. Any changes that can be conceived by those skilled in the art should fall within the scope of protection of the present invention.

Claims

1. A multi-sensor pose calibration method, characterized in that: include: For a plurality of planar marker plates having different positions and orientations, determining position information of a laser point projected onto each of the planar marker plates in a radar coordinate system; For each of the laser points, establishing a first association relationship between position information of the laser point in the radar coordinate system and position information of the laser point in the visual sensor coordinate system by using external parameters of the visual sensor and the lidar sensor; A second association relationship is established between a first plane equation of the plane marker plate corresponding to the laser point in the plane marker plate coordinate system and a second plane equation of the plane marker plate in the visual sensor coordinate system. Based on the first association relationship and the second association relationship, a model of the distance residual from the laser point to the corresponding plane marker plate plane is constructed. The model of the distance residual from the laser point to the corresponding plane marker plate plane is constructed based on the first association relationship and the second association relationship. The error equation is first constructed, and the distance residual ri is set as the distance from the current laser point to the plane marker plate, and r i =d(n^c,P i c ), where d ij (n^c,P i c )=(P i c ) T *n^c, according to the first association relationship P i c =T cl *P i l and the second association relationship n^c=(T ca -1 ) T *n^a, get the model d of the distance residual ij (n ^ c, P i c )=(T cl *P i l ) T *(T ca -1 ) T *n ^ a, the T cl are the external parameters of the visual sensor and the lidar sensor, and the P i l is the position information of the laser point in the radar coordinate system, the P i c is the position information of the laser point in the visual sensor coordinate system, n^a is the first plane equation of the plane marking plate corresponding to the laser point in the plane marking plate coordinate system, n^c is the second plane equation of the plane marking plate corresponding to the laser point in the visual sensor coordinate system, T ca Represents the transformation relationship between the marker point on the plane marker plate and the visual sensor coordinate system, (T ca -1 ) T Indicates T ca The transposed matrix of the inverse matrix of ; The distance residuals corresponding to the respective laser points in the laser points are weighted, and when the weighted result is the minimum, the external parameters of the visual sensor and the lidar sensor are obtained.

2. The method according to claim 1, characterized in that Determining the position information of the laser points projected onto each of the planar marking plates in the radar coordinate system includes: For each of the planar marker plates, determining a boundary laser point projected onto the planar marker plate and position information of the boundary laser point in a radar coordinate system; Other laser points other than the boundary laser point projected onto the plane marking plate, as well as position information of the other laser points in a radar coordinate system, are determined based on the boundary laser point.

3. The method according to claim 1, characterized in that The external parameters of the visual sensor and the lidar sensor are obtained by the following formula: Wherein, j represents the number of the plurality of plane marking plates, N is the number of the plurality of plane marking plates, i represents the number of the laser points projected onto the current plane marking plate, M is the number of the laser points projected onto the current plane marking plate, and d ij (nc, P i c ) represents the model of the distance residual from the laser point to the corresponding plane marker plane, and the external parameter T of the visual sensor and the lidar sensor cl is (R, t), where T represents the objective function.

4. A multi-sensor posture calibration device, characterized in that: include: A determination module is configured to determine, for a plurality of planar marker plates having different positions and directions, position information of a laser point projected onto each of the planar marker plates in a radar coordinate system; an association module for establishing, for each of the laser points, a first association relationship between position information of the laser point in the radar coordinate system and position information of the laser point in the visual sensor coordinate system by using external parameters of the visual sensor and the lidar sensor; The construction module includes: an association unit for establishing a second association relationship between a first plane equation of the plane marker plate corresponding to the laser point in the plane marker plate coordinate system and a second plane equation of the plane marker plate in the visual sensor coordinate system, and a construction unit for constructing a model of the distance residual from the laser point to the corresponding plane marker plate based on the first association relationship and the second association relationship, wherein the construction unit is used to first construct an error equation, assuming that the distance residual ri is the distance from the current laser point to the plane marker plate, r i =d(n^c,P i c ), where d ij (n^c,P i c )=(P i c ) T *n^c, according to the first association relationship P i c =T cl *P i l And the second association relationship n^c=(T ca -1 ) T *n^a, get the model d of the distance residual ij (n ^ c, P i c )=(T cl *P i l ) T *(T ca -1 ) T *n ^ a, the T cl are the external parameters of the visual sensor and the lidar sensor, and the P i l is the position information of the laser point in the radar coordinate system, the P i c is the position information of the laser point in the visual sensor coordinate system, n^a is the first plane equation of the plane marking plate corresponding to the laser point in the plane marking plate coordinate system, n^c is the second plane equation of the plane marking plate corresponding to the laser point in the visual sensor coordinate system, T ca Represents the transformation relationship between the marker point on the plane marker plate and the visual sensor coordinate system, (T ca -1 ) T Indicates T ca The transposed matrix of the inverse matrix of ; The calibration module is used to weight the distance residuals corresponding to each laser point in the laser points, and obtain the external parameters of the visual sensor and the lidar sensor when the weighted result is minimum.

5. The device according to claim 4, characterized in that The determination module includes: a first determining unit, configured to determine, for each of the planar marker plates, a boundary laser point projected onto the planar marker plate and position information of the boundary laser point in a radar coordinate system; The second determining unit is configured to determine other laser points projected onto the plane marking plate except the boundary laser point according to the boundary laser point, as well as position information of the other laser points in a radar coordinate system.

6. A multi-sensor posture calibration system comprising: The posture calibration device according to claim 4 or 5, further comprising: A plurality of planar marking plates, wherein the positions and orientations of the plurality of planar marking plates are all different; a laser radar sensor for emitting laser light toward the plurality of planar marker plates and collecting laser data including distance information and angle information; The visual sensor is used to collect image information, wherein the image information includes relevant image information of the marker points on the multiple planar marking plates.

Images