Sensing device calibration method and apparatus, storage medium, and electronic device

Abstract

The present disclosure relates to a sensor device calibration method, device, storage medium and electronic device. The sensor device calibration method comprises: performing position detection based on image information of each target calibration plate to obtain first position information of each target calibration plate in a first coordinate system, the first coordinate system being a three-dimensional coordinate system corresponding to a first sensing device; performing fitting based on point cloud data of each target calibration plate to obtain second position information of each target calibration plate in a second coordinate system, the second coordinate system being a three-dimensional coordinate system corresponding to a second sensing device, the second position information being obtained based on position calibration of edge information of the calibration plate fitted based on the point cloud data and plane information of the calibration plate fitted based on the point cloud data; and performing coordinate system conversion processing based on the first position information and the second position information of each target calibration plate in a plurality of calibration plates to obtain target conversion parameters between the two coordinate systems. The present disclosure can improve the efficiency and robustness of sensor device calibration.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of sensor calibration, and in particular to a sensor device calibration method, device, storage medium and electronic device. BACKGROUND

[0002] With the rapid development of the automotive industry, the types and quantities of sensors on vehicles are increasing. In order to better play the performance of the sensors, it is necessary to effectively fuse the information collected by multiple sensors. Therefore, a unified coordinate system needs to be obtained through calibration. Camera and lidar are the most important sensors of autonomous vehicles, and their calibration has always been a concern. The existing calibration methods mainly extract feature points of calibration objects for lidar and camera respectively, which requires manual selection or placement of multiple sets of calibration objects, and is inefficient. Moreover, the robustness of the external parameters obtained by the lidar through the feature points cannot be guaranteed. SUMMARY

[0003] To solve at least one of the above technical problems, the present disclosure provides a sensor device calibration method, device, storage medium and electronic device.

[0004] According to an aspect of the present disclosure, a sensor device calibration method is provided, which comprises:

[0005] obtaining image information of each target calibration board in a plurality of calibration boards collected by a first sensing device and point cloud data of the each target calibration board in the plurality of calibration boards collected by a second sensing device;

[0006] detecting the position based on the image information of the each target calibration board to obtain first position information of the each target calibration board in a first coordinate system, the first coordinate system being a three-dimensional coordinate system corresponding to the first sensing device;

[0007] fitting based on the point cloud data of the each target calibration board to obtain second position information of the each target calibration board in a second coordinate system, the second coordinate system being a three-dimensional coordinate system corresponding to the second sensing device, the second position information being obtained by position calibration based on fitted calibration board edge information and fitted calibration board plane information based on the point cloud data;

[0008] performing coordinate system conversion processing based on the first position information and the second position information of each target calibration board in the plurality of calibration boards to obtain target conversion parameters between the first coordinate system and the second coordinate system.

[0009] In some possible implementations, the first position information includes first center point information and first normal vector information, the second position information includes second center point information and second normal vector information, and the target conversion parameters include rotation parameters and translation parameters.

[0010] The coordinate system conversion processing is performed based on the first position information and the second position information of each target calibration board in the plurality of calibration boards, to obtain a target conversion parameter between the first coordinate system and the second coordinate system, including:

[0011] The first center point information of each target calibration board and the first normal vector information of each target calibration board in the plurality of calibration boards are integrated respectively, to obtain a first center point matrix corresponding to the first center point information and a first normal vector matrix corresponding to the first normal vector information;

[0012] The second center point information of each target calibration board and the second normal vector information of each target calibration board in the plurality of calibration boards are integrated respectively, to obtain a second center point matrix corresponding to the second center point information and a second normal vector matrix corresponding to the second normal vector information;

[0013] The coordinate system rotation calculation is performed based on the first normal vector matrix and the second normal vector matrix, to obtain the rotation parameter;

[0014] The coordinate system translation calculation is performed based on the first center point matrix, the second center point matrix and the rotation parameter, to obtain the translation parameter.

[0015] In some possible implementation manners, the coordinate system rotation calculation is performed based on the first normal vector matrix and the second normal vector matrix, to obtain the rotation parameter, including:

[0016] The second normal vector matrix and a transpose matrix of the second normal vector matrix are multiplied, to obtain a first matrix;

[0017] An inverse matrix of the first matrix is obtained;

[0018] The second normal vector matrix and a transpose matrix of the first normal vector matrix are multiplied, to obtain a second matrix;

[0019] A transpose matrix of the second matrix is obtained;

[0020] The inverse matrix of the first matrix and the transpose matrix of the second matrix are multiplied, to obtain the rotation parameter.

[0021] In some possible implementation manners, the coordinate system translation calculation is performed based on the first center point matrix, the second center point matrix and the rotation parameter, to obtain the translation parameter, including:

[0022] The second center point matrix and the rotation parameter are multiplied, to obtain a third matrix;

[0023] subtracting the first center point matrix from the third matrix to obtain a fourth matrix;

[0024] averaging the fourth matrix to obtain the translation parameter.

[0025] In some possible implementation manners, the fitting based on the point cloud data of each target calibration board to obtain the second position information of the target calibration board in the second coordinate system comprises:

[0026] fitting the point cloud data of each target calibration board by using a preset algorithm to obtain calibration board plane information of the target calibration board;

[0027] performing normal vector analysis based on the calibration board plane information of each target calibration board to obtain the second normal vector information.

[0028] In some possible implementation manners, the fitting based on the point cloud data of each target calibration board to obtain the second position information of the target calibration board in the second coordinate system comprises:

[0029] fitting the point cloud data corresponding to the edges of each target calibration board to obtain calibration board edge information of the target calibration board;

[0030] performing edge connection processing based on the calibration board edge information of each target calibration board to obtain vertex coordinate information of the target calibration board;

[0031] performing line connection processing based on the vertex coordinate information of each target calibration board to obtain second center point information of the target calibration board.

[0032] In some possible implementation manners, the position detection based on the image information of each target calibration board to obtain the first position information of the target calibration board in the first coordinate system comprises:

[0033] performing image recognition processing based on the image information of each target calibration board to obtain corner point information corresponding to the target calibration board;

[0034] performing coordinate calculation processing based on the corner point information corresponding to each target calibration board to obtain first normal vector information and first center point information corresponding to the target calibration board.

[0035] According to a second aspect of the present disclosure, a calibration device for a sensing device is provided, and the device comprises:

[0036] An initial information acquisition module is configured to acquire image information of each target calibration plate in a plurality of calibration plates collected by a first sensing device and point cloud data of the each target calibration plate collected by a second sensing device;

[0037] A first coordinate system calculation module is configured to perform position detection based on the image information of the each target calibration plate to obtain first position information of the each target calibration plate in a first coordinate system, the first coordinate system being a three-dimensional coordinate system corresponding to the first sensing device;

[0038] A second coordinate system calculation module is configured to perform fitting based on the point cloud data of the each target calibration plate to obtain second position information of the each target calibration plate in a second coordinate system, the second coordinate system being a three-dimensional coordinate system corresponding to the second sensing device, the second position information being obtained by position calibration based on fitted calibration plate edge information and fitted calibration plate plane information based on the point cloud data;

[0039] A target conversion parameter determination module is configured to perform coordinate system conversion processing based on the first position information and the second position information of each target calibration plate in the plurality of calibration plates to obtain target conversion parameters between the first coordinate system and the second coordinate system.

[0040] According to a third aspect of the present disclosure, an electronic device is provided, including at least one processor, and a memory connected with the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the at least one processor implements the sensing device calibration method according to any one of the first aspect by executing the instructions stored in the memory.

[0041] According to a fourth aspect of the present disclosure, a computer readable storage medium is provided, and the computer readable storage medium stores at least one instruction or at least one program, the at least one instruction or at least one program is loaded and executed by a processor to implement the sensing device calibration method according to any one of the first aspect.

[0042] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, but not limiting the present disclosure.

[0043] The present disclosure has the following beneficial effects:

[0044] The data detected by the first sensing device and the data monitored by the second sensing device are obtained by acquiring image information of each target calibration plate in the plurality of calibration plates collected by the first sensing device and point cloud data of each target calibration plate in the plurality of calibration plates collected by the second sensing device. The first position information of each target calibration plate in the first coordinate system is obtained by performing position detection on the image information of each target calibration plate, the first coordinate system is a three-dimensional coordinate system corresponding to the first sensing device, and the position information of each target calibration plate in the first coordinate system is determined. The second position information of each target calibration plate in the second coordinate system is obtained by fitting the point cloud data of each target calibration plate, the second coordinate system is a three-dimensional coordinate system corresponding to the second sensing device, the second position information is obtained by position calibration based on the edge information of the calibration plate fitted based on the point cloud data and the plane information of the calibration plate fitted based on the point cloud data, the robustness and accuracy of calibration are improved; the target conversion parameters between the first coordinate system and the second coordinate system are obtained by performing coordinate system conversion processing on the first position information and the second position information of each target calibration plate in the plurality of calibration plates, and the external parameters between the first sensing device and the second sensing device are obtained only by one frame of data, thereby improving the efficiency and automation degree of calibration.

[0045] Other features and aspects of the present disclosure will become apparent from the following detailed description of exemplary embodiments with reference to the drawings. BRIEF DESCRIPTION OF DRAWINGS

[0046] In order to more clearly illustrate the technical solutions and advantages of the embodiments or prior art in the specification, the drawings needed in the embodiment or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the specification, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0047] Figure 1 A flowchart of a sensing device calibration method according to an embodiment of the present disclosure is shown;

[0048] Figure 2 A flowchart of a rotation parameter and translation parameter solving method according to an embodiment of the present disclosure is shown;

[0049] Figure 3 A flowchart of a rotation parameter solving method according to an embodiment of the present disclosure is shown;

[0050] Figure 4 A flowchart of a translation parameter solving method according to an embodiment of the present disclosure is shown;

[0051] Figure 5 A flowchart of a second normal vector information solving method according to an embodiment of the present disclosure is shown;

[0052] Figure 6 FIG. 2 shows a flow diagram of a second center point information solving method according to an embodiment of the present disclosure;

[0053] Figure 7 FIG. 3 shows a flow diagram of a first normal vector information and first center point information solving method according to an embodiment of the present disclosure;

[0054] Figure 8 FIG. 4 shows a block diagram of an electronic device according to an embodiment of the present disclosure;

[0055] Figure 9 FIG. 5 shows a block diagram of another electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0056] The technical solutions in the embodiments of the present disclosure will be described clearly and completely below with reference to the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only some of the embodiments of the present disclosure, but not all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work fall within the scope of the present disclosure.

[0057] It should be noted that the terms "first", "second", and the like in the description and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present disclosure described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or server including a series of steps or units does not necessarily have to include those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to the process, method, product, or device.

[0058] Various exemplary embodiments, features, and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numbers in the drawings represent functionally similar or the same elements. Although various aspects of the embodiments are illustrated in the drawings, the drawings are not necessarily drawn to scale unless specifically indicated.

[0059] The word "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any implementation described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other implementations.

[0060] The term "and / or", as used herein, merely describes association between associated objects, and can indicate that there are three cases, for example, A and / or B can indicate that there are three cases of A alone, A and B, and B alone. In addition, the term "at least one" herein indicates any one of a plurality or any combination of at least two of a plurality, for example, at least one of A, B and C can indicate any one or more elements selected from the set consisting of A, B and C.

[0061] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the specific embodiments below. Those skilled in the art should understand that the present disclosure can also be implemented without some specific details. In some examples, methods, means, elements and circuits well known to those skilled in the art are not described in detail, in order to highlight the main ideas of the present disclosure.

[0062] Figure 1 A flowchart of a sensor device calibration method according to an embodiment of the present disclosure is shown as follows. Figure 1 As shown in the above method, the method comprises:

[0063] S101, acquiring image information of each target calibration board in a plurality of calibration boards collected by a first sensing device and point cloud data of each target calibration board in the plurality of calibration boards collected by a second sensing device.

[0064] The terminal device acquires image information of each target calibration board in a plurality of calibration boards placed in a preset manner through a first sensing device. The terminal device acquires point cloud data of each target calibration board in the plurality of calibration boards placed in the same preset manner through a second sensing device. The preset manner can be that the orientations of the plurality of calibration boards are different respectively, and the included angle between any two adjacent calibration boards is within a preset angle range. Exemplarily, the preset angle range can be [30, 60].

[0065] In some embodiments, the first sensor device can be a camera, the second sensor device can be a laser radar, the plurality of calibration boards can be three calibration boards, the three calibration boards can be a group of calibration boards, the arrangement of the calibration boards can be one group or multiple groups, which is not limited herein, and one group of calibration boards is taken as an example in the embodiments. The calibration board can be a black-and-white chessboard including 7x9 squares, the three calibration boards are arranged on an empty land without other interference objects, the three calibration boards are arranged in different directions, and the included angle between any two adjacent calibration boards is [30, 60] degrees. The image information of each target calibration board in the three calibration boards is acquired by the camera. The cloud point data of each target calibration board in the three calibration boards is acquired by adjusting the region of interest (ROI) of the laser radar. The terminal device acquires the image information of each target calibration board collected by the camera and the cloud point data of each target calibration board collected by the laser radar.

[0066] S102, position detection is performed based on the image information of each target calibration board to obtain first position information of each target calibration board in a first coordinate system,

[0067] The first coordinate system is a three-dimensional coordinate system corresponding to the first sensor device.

[0068] In some embodiments, the first position information includes first center point information and first normal vector information, the first center point information includes the center point coordinates of the target calibration board in the first coordinate system, and the first normal vector information includes the normal vector of the target calibration board in the first coordinate system. The terminal device extracts a key point in the image information of each target calibration board, determines the coordinates of the key point in the first coordinate system, and determines the first center point information and the first normal vector information of each target calibration board according to the coordinates of the key point.

[0069] In some embodiments, the first coordinate system can be a three-dimensional coordinate system corresponding to the camera, i.e., a camera coordinate system. The terminal device acquires the image information of each target calibration board collected by the camera, extracts the key point coordinates of each target calibration board by performing position detection on the image information of each target calibration board, and the key point of the calibration board can be a corner point. Therefore, the corner point coordinates of each target calibration board in the camera coordinate system are extracted. The first center point information and the first normal vector information of each target calibration board are calculated according to the corner point coordinates of each target calibration board.

[0070] S103, fitting is performed based on the point cloud data of each target calibration board to obtain second position information of each target calibration board in a second coordinate system,

[0071] The second coordinate system is a three-dimensional coordinate system corresponding to the second sensor device, and the second position information is obtained by position calibration based on the edge information of the calibration board fitted based on the point cloud data and the plane information of the calibration board fitted based on the point cloud data.

[0072] In some embodiments, the second position information includes second center point information and second normal vector information, the second center point information including a center point coordinate of the target calibration board in the second coordinate system, and the second normal vector information including a normal vector of the target calibration board in the second coordinate system. After the terminal device obtains the point cloud data transmitted by the laser radar, the point cloud data is fitted by using a preset algorithm to obtain calibration board plane information, and the point cloud data corresponding to the edges of the target calibration board is fitted to obtain calibration board edge information. The normal vector of the calibration board plane information is calculated to obtain the second normal vector information. According to the connection points between the edges corresponding to the calibration board edge information of each target calibration board, the calibration board is a geometric board with multiple vertices and edges, and the coordinates of the vertices, i.e., the vertex information, are determined. The vertices are connected based on the vertex information to obtain the second center point information of each target calibration board in the second coordinate system.

[0073] In one embodiment, the second coordinate system can be a three-dimensional coordinate system corresponding to the laser radar, i.e., a radar coordinate system. The terminal device obtains the point cloud data of each target calibration board through the laser radar, fits the point cloud data of each target calibration board to obtain the calibration board plane information corresponding to each target calibration board, detects the point cloud data corresponding to the edges of each target calibration board based on the point cloud data, obtains the calibration board edge information, and the calibration board has multiple edges and multiple vertices. According to the connection points between the edges corresponding to the calibration board edge information, the vertex coordinates of the multiple vertices of each target calibration board in the radar coordinate system are determined, and then the diagonal vertices are connected based on the vertex coordinates to determine the center point coordinates of the target calibration board in the radar coordinate system, i.e., the second center point information. For example, the target calibration board can include four vertices and four edges.

[0074] Each calibration board can have four edges and four vertices. According to the connection points between the edges corresponding to the calibration board edge information, the four vertex coordinates of each target calibration board in the radar coordinate system are determined. The four vertices in the radar coordinate system are connected to the inside of the plane corresponding to the target calibration board to obtain the center point coordinates of the target calibration board in the radar coordinate system, i.e., the second center point information.

[0075] S104, coordinate system conversion processing is performed based on the first position information and the second position information of each target calibration board in the multiple calibration boards to obtain target conversion parameters between the first coordinate system and the second coordinate system.

[0076] In some embodiments, the first position information of each target calibration board in the plurality of calibration boards is integrated to obtain a first center point matrix and a first normal vector matrix corresponding to the plurality of calibration boards, the second position information of each target calibration board in the plurality of calibration boards is integrated to obtain a second center point matrix and a second normal vector matrix corresponding to the plurality of calibration boards, and coordinate system conversion processing is performed based on the first center point matrix, the second center point matrix, the third center point matrix and the fourth center point matrix to obtain target conversion parameters between the first coordinate system and the second coordinate system.

[0077] In some embodiments, the first center point information of the three target calibration boards detected by the corner points in the camera coordinate system is integrated into a first center point matrix, the first normal vector information of the three target calibration boards detected by the corner points in the camera coordinate system is integrated into a first normal vector matrix, the second center point information of the three target calibration boards calculated by the point cloud data in the laser radar coordinate system is integrated into a second center point matrix, and the second normal vector information of the three target calibration boards calculated by the point cloud data in the laser radar coordinate system is integrated into a second normal vector matrix. Coordinate system conversion processing is performed based on the first center point matrix, the second center point matrix, the third center point matrix and the fourth center point matrix to obtain target conversion parameters between the camera coordinate system and the radar coordinate system.

[0078] Based on the above technical solution, the calculation steps of the calibration method between the first coordinate system and the second coordinate system are simplified, and the efficiency and automation degree of calibration are improved. The problem of difficult alignment of the information collected by the first sensing device and the information collected by the second sensing device due to coordinate errors is solved, and the robustness and accuracy of calibration are improved.

[0079] In some embodiments, the first position information includes first center point information and first normal vector information, the second position information includes second center point information and second normal vector information, and the target conversion parameters include rotation parameters and translation parameters.

[0080] Please refer to Figure 2 , the target conversion parameters between the first coordinate system and the second coordinate system are obtained based on the first position information and the second position information of each target calibration board in the plurality of calibration boards, including:

[0081] S1041, the first center point information of each target calibration board in the plurality of calibration boards and the first normal vector information of each target calibration board are integrated to obtain a first center point matrix corresponding to the first center point information and a first normal vector matrix corresponding to the first normal vector information;

[0082] S1042, respectively, the second center point information of each target calibration plate in the plurality of calibration plates and the second normal vector information of each target calibration plate is integrated to obtain the second center point matrix corresponding to the second center point information and the second normal vector matrix corresponding to the second normal vector information;

[0083] S1043, based on the first normal vector matrix and the second normal vector matrix, a coordinate system rotation calculation is performed to obtain a rotation parameter;

[0084] S1044, based on the first center point matrix, the second center point matrix and the rotation parameter, a coordinate system translation calculation is performed to obtain a translation parameter.

[0085] In some embodiments, the first center point information of the plurality of calibration plates in the first coordinate system is integrated to obtain a first center point matrix, and the first normal vector information of the plurality of calibration plates in the first coordinate system is integrated to obtain a first normal vector matrix. The second center point information of the plurality of calibration plates in the second coordinate system is integrated to obtain a second center point matrix, and the second normal vector information of the plurality of calibration plates in the second coordinate system is integrated to obtain a second normal vector matrix.

[0086] In some embodiments, the first center point information corresponding to the three target calibration plates detected by the corner point in the camera coordinate system respectively includes the first center point coordinates ocl corresponding to the first target calibration plate, the second center point coordinates oc2 corresponding to the second target calibration plate, and the third center point coordinates oc3 corresponding to the third target calibration plate. The first center point information corresponding to the three target calibration plates in the camera coordinate system is integrated to obtain a first center point matrix O C =[oc1, oc2, oc3]. The first normal vector information corresponding to the three target calibration plates detected by the corner point in the camera coordinate system respectively includes the first normal vector ncl corresponding to the first target calibration plate, the second normal vector nc2 corresponding to the second target calibration plate, and the third normal vector coordinates nc3 corresponding to the third target calibration plate. The first normal vector information corresponding to the three target calibration plates in the camera coordinate system is integrated to obtain a first normal vector matrix N C =[nc1, nc2, nc3].

[0087] The second center point information corresponding to the three target calibration plates detected by the cloud point data in the radar coordinate system respectively includes the fourth center point coordinates oli corresponding to the first target calibration plate, the fifth center point coordinates ol2 corresponding to the second target calibration plate, and the sixth center point coordinates ol3 corresponding to the third target calibration plate. The second center point information corresponding to the three target calibration plates in the radar coordinate system is integrated to obtain a second center point matrix O L= [ol1, ol2, ol3]. The second normal vector information corresponding to the three target calibration boards detected by the cloud point data in the radar coordinate system respectively includes a fourth normal vector nl1 corresponding to the first target calibration board, a fifth normal vector nl2 corresponding to the second target calibration board, and a sixth normal vector nl3 corresponding to the third target calibration board. The second normal vector information corresponding to the three target calibration boards in the radar coordinate system is integrated to obtain a second normal vector matrix N L = [nl1, nl2, nl3].

[0088] The first normal vector matrix and the second normal vector matrix are subjected to coordinate system rotation calculation to obtain rotation parameters between the camera coordinate system and the radar coordinate system. The first center point matrix, the second center point matrix, and the rotation parameters are subjected to coordinate system translation calculation to obtain translation parameters between the camera coordinate system and the radar coordinate system.

[0089] In this way, the center point coordinates and the normal vectors of the plurality of calibration boards are integrated into a matrix corresponding to the center points and a matrix corresponding to the normal vectors, respectively. Only by using the two groups of matrices, the rotation parameters and the translation parameters between the two coordinate systems can be confirmed, and the calibration efficiency and the automation degree are improved.

[0090] Please refer to Figure 3 In some embodiments, the rotation parameters are obtained based on coordinate system rotation calculation of the first normal vector matrix and the second normal vector matrix, and the coordinate system rotation calculation comprises:

[0091] S10431, multiplying the second normal vector matrix and the transpose matrix of the second normal vector matrix to obtain a first matrix;

[0092] S10432, obtaining an inverse matrix of the first matrix;

[0093] S10433, multiplying the second normal vector matrix and the transpose matrix of the first normal vector matrix to obtain a second matrix;

[0094] S10434, obtaining a transpose matrix of the second matrix;

[0095] S10435, multiplying the inverse matrix of the first matrix and the transpose matrix of the second matrix to obtain the rotation parameters.

[0096] In some embodiments, the rotation parameters between the camera coordinate system and the radar coordinate system The calculation formula is as follows:

[0097]

[0098] wherein, N L is the second normal vector matrix, and N C is the first normal vector matrix.

[0099] Thus, the rotation parameters between the two coordinate systems are calculated by using the normal vector of the calibration board plane, and the problem of non-robustness of using feature points to calculate the rotation parameters is overcome.

[0100] Referring to Figure 4 In some embodiments, the coordinate system translation calculation is performed based on the first center point matrix, the second center point matrix, and the rotation parameter to obtain the translation parameter, including:

[0101] S10441, the second center point matrix and the rotation parameter are multiplied to obtain a third matrix;

[0102] S10442, the first center point matrix and the third matrix are subtracted to obtain a fourth matrix;

[0103] S10443, the fourth matrix is averaged to obtain the translation parameter.

[0104] In some embodiments, the translation parameter between the camera coordinate system and the radar coordinate system The calculation formula is as follows:

[0105]

[0106] Wherein, O C is the first center point matrix, O L is the second center point matrix, is the rotation parameter between the camera coordinate system and the radar coordinate system.

[0107] Thus, only the normal vector and the center point of the calibration board in the two coordinate systems are used to calculate the rotation parameter and the translation parameter, two external calibration parameters between the two coordinate systems, reducing the calculation amount and improving the efficiency of automatic calibration.

[0108] Referring to Figure 5 In some embodiments, the point cloud data of each target calibration board is fitted to obtain the second position information of each target calibration board in the second coordinate system, including:

[0109] S1031, the point cloud data of each target calibration board is fitted by using a preset algorithm to obtain the calibration board plane information of each target calibration board;

[0110] S1033, the normal vector analysis is performed based on the calibration board plane information of each target calibration board to obtain the second normal vector information.

[0111] The point cloud data of each target calibration plate is fitted by using a preset algorithm to obtain calibration plate plane information of the target calibration plate, and the calibration plate plane information includes a calibration plate plane equation. A normal vector is solved based on the calibration plate plane equation by using a preset method to obtain second normal vector information of each target calibration plate in the second coordinate system.

[0112] In some embodiments, the preset algorithm includes a RANdom SAmple Consensus (RANSAC) algorithm, which can calculate data model parameters of data from a set of sample data sets containing abnormal data to obtain effective sample data. The point cloud data of each target calibration plate is fitted by using the RANSAC algorithm to obtain a calibration plate plane equation of each target calibration plate in the radar coordinate system. An equation set for solving the normal vector can be established according to the calibration plate plane equation, and the normal vector is solved. There are many preset methods for solving the normal vector by using the calibration plate plane equation, which are not limited herein.

[0113] Based on the above technical solutions, the normal vector of the target calibration plate in the second coordinate system is solved by using the point cloud data of the target calibration plate, which is simple and reliable, and improves the efficiency and robustness of automatic calibration.

[0114] Please refer to Figure 6 In some embodiments, fitting is performed based on the point cloud data of each target calibration plate to obtain second position information of each target calibration plate in the second coordinate system, including:

[0115] S1032, the point cloud data corresponding to the edge of each target calibration plate is fitted to obtain calibration plate edge information of each target calibration plate;

[0116] S1034, edge connection processing is performed based on the calibration plate edge information of each target calibration plate to obtain vertex coordinate information of each target calibration plate;

[0117] S1036, connection processing is performed based on the vertex coordinate information of each target calibration plate to obtain second center point information of each target calibration plate.

[0118] In some embodiments, the terminal device identifies and fits the point cloud data corresponding to the edge of each target calibration plate to obtain calibration plate edge information of each target calibration plate, and the calibration plate edge information includes straight line equations of four edges of the target calibration plate. The straight line equations of the four edges are used to solve the vertex coordinates of the intersection of each two edges of the four edges of the target calibration plate. The four edges correspond to four vertices, and the four vertices of the target calibration plate are connected by diagonal vertices to obtain two diagonal lines. The equations of the two diagonal lines are obtained based on the four vertex coordinates, and the center point coordinates of the target calibration plate in the second coordinate system are obtained according to the equations of the two diagonal lines. The second center point information includes the center point coordinates of the target calibration plate in the second coordinate system.

[0119] Thus, the solving process of the second sensing device for the second center point information of the target calibration board is simplified, the method is reliable, and the accuracy and robustness of the sensing device calibration are improved.

[0120] Referring to Figure 7 In some embodiments, the position detection is performed based on the image information of each target calibration board to obtain first position information of each target calibration board in the first coordinate system, including:

[0121] S1021, image recognition processing is performed based on the image information of each target calibration board to obtain corresponding corner point information of each target calibration board;

[0122] S1022, coordinate calculation processing is performed based on the corresponding corner point information of each target calibration board to obtain corresponding first normal vector information and first center point information of each target calibration board.

[0123] The terminal device performs image recognition on the image information of each target calibration board to determine the black and white grid intersection coordinates, i.e., the corner point coordinates, of each target calibration board. The corner point information includes the corner point coordinates. According to the corner point coordinates of each target calibration board, the plane equation of each target calibration board in the first coordinate system can be calculated, and according to the plane equation of each target calibration board in the first coordinate system, the normal vector and the center point coordinates of the plane corresponding to each target calibration board in the first coordinate system can be solved, i.e., the first normal vector information and the second normal vector information can be solved. There are many methods for solving the normal vector and the center point coordinates using the plane equation, which are not limited here.

[0124] Thus, the normal vector and the center point coordinates of the target calibration board in the first coordinate system are solved using the corner point coordinates of the target calibration board, the method is simple and reliable, and the efficiency and robustness of the automatic calibration are improved.

[0125] Referring to Figure 8 According to a second aspect of the present disclosure, a sensing device calibration device is provided, which includes:

[0126] An initial information acquisition module 10 is configured to acquire image information of each target calibration board in a plurality of calibration boards collected by a first sensing device and point cloud data of each target calibration board in the plurality of calibration boards collected by a second sensing device;

[0127] A first coordinate system calculation module 20 is configured to perform position detection based on the image information of each target calibration board to obtain first position information of each target calibration board in a first coordinate system, and the first coordinate system is a three-dimensional coordinate system corresponding to the first sensing device;

[0128] The second coordinate system calculation module 30 is configured to fit based on the point cloud data of each target calibration plate to obtain second position information of each target calibration plate in a second coordinate system, the second coordinate system being a three-dimensional coordinate system corresponding to the second sensing device, and the second position information being obtained by position calibration based on the edge information of the calibration plate fitted based on the point cloud data and the plane information of the calibration plate fitted based on the point cloud data.

[0129] The target conversion parameter determination module 40 is configured to perform coordinate system conversion processing based on the first position information and the second position information of each target calibration plate in the plurality of calibration plates to obtain target conversion parameters between the first coordinate system and the second coordinate system.

[0130] In some embodiments, the apparatus provided by the embodiments of the present disclosure has functions or includes modules that can be used to perform the methods described in the above method embodiments, and the specific implementation can refer to the description of the above method embodiments. For brevity, it will not be repeated here.

[0131] The embodiments of the present application provide a kind of sensing device calibration equipment, the equipment can be terminal or server, the sensing device calibration equipment includes processor and memory, at least one instruction or at least one program is stored in the memory, the at least one instruction or the at least one program is loaded and executed by the processor to realize the sensing device calibration method as provided in the above method embodiment.

[0132] The memory can be used to store software programs and modules, and the processor can execute various functions and data processing by running the software programs and modules stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store operating systems, application programs required by functions, etc.;The data storage area can store data created according to the use of the device, etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state memory device. Accordingly, the memory can also include a memory controller to provide access for the processor to the memory.

[0133] The method embodiments provided by the embodiments of the present application can be executed in a mobile terminal, a computer terminal, a server or a similar computing device, etc. electronic equipment. Figure 9 is the hardware structure block diagram of the electronic equipment provided by the sensing device calibration method of the embodiments of the present application. As shown in Figure 9As shown, the electronic device 900 can vary significantly due to differences in configuration or performance. It may include one or more central processing units (CPUs) 910 (CPUs 910 may include, but are not limited to, microprocessors such as MCUs or programmable logic devices such as FPGAs), a memory 930 for storing data, and one or more storage media 920 (e.g., one or more mass storage devices) for storing application programs 923 or data 922. The memory 930 and storage media 920 may be temporary or persistent storage. The program stored in the storage media 920 may include one or more modules, each module may include a series of instruction operations on the electronic device. Furthermore, the CPU 910 may be configured to communicate with the storage media 920 and execute the series of instruction operations in the storage media 920 on the electronic device 900. Electronic device 900 may also include one or more power supplies 960, one or more wired or wireless network interfaces 950, one or more input / output interfaces 940, and / or one or more operating systems 921, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0134] The input / output interface 940 can be used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the electronic device 900. In one example, the input / output interface 940 includes a network interface controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the input / output interface 940 may be a radio frequency (RF) module for wireless communication with the Internet.

[0135] Those skilled in the art will understand that Figure 9 The structure shown is for illustrative purposes only and does not limit the structure of the electronic device described above. For example, the electronic device 900 may also include... Figure 9 The more or fewer components shown, or having the same Figure 9 The different configurations shown.

[0136] Embodiments of this application also provide a computer-readable storage medium, which can be disposed in an electronic device to store at least one instruction or at least one program related to implementing a sensor calibration method in the method embodiment. The at least one instruction or the at least one program is loaded and executed by the processor to implement the sensor calibration method provided in the above method embodiment.

[0137] Optionally, in the embodiment, the storage medium can be located in at least one of the plurality of network servers of the computer network. Optionally, in the embodiment, the storage medium can include, but is not limited to, a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and various media capable of storing program codes.

[0138] According to an aspect of the present application, a computer program product or computer program is provided, which includes computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions to enable the computer device to perform the method provided in the various optional implementation manners.

[0139] It can be seen from the embodiments of the document recognition method, device, equipment, terminal, server, storage medium and computer program provided by the present application that, by first acquiring image information of each target calibration plate in the plurality of calibration plates collected by a first sensing device and point cloud data of each target calibration plate in the plurality of calibration plates collected by a second sensing device, data detected by the first sensing device and data monitored by the second sensing device are acquired. The first position information of each target calibration plate in a first coordinate system is obtained by performing position detection on the image information of each target calibration plate, the first coordinate system is a three-dimensional coordinate system corresponding to the first sensing device, and the position information of each target calibration plate in the first coordinate system is determined. The second position information of each target calibration plate in a second coordinate system is obtained by fitting the point cloud data of each target calibration plate, the second coordinate system is a three-dimensional coordinate system corresponding to the second sensing device, the second position information is obtained by position calibration based on the edge information of the calibration plate fitted based on the point cloud data and the plane information of the calibration plate fitted based on the point cloud data, and the robustness and accuracy of the calibration are improved. The target conversion parameters between the first coordinate system and the second coordinate system are obtained by performing coordinate system conversion processing on the first position information and the second position information of each target calibration plate in the plurality of calibration plates, and the extrinsic parameters between the first sensing device and the second sensing device are obtained only by one frame of data, and the efficiency and the degree of automation of the calibration are improved.

[0140] It should be noted that the above-mentioned order of the embodiments of the present application is only for description, and does not represent the advantages and disadvantages of the embodiments. And the above-mentioned specific embodiments of the present application are described. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from the order in the embodiments and still achieve the desired result. In addition, the processes depicted in the figures do not necessarily require the particular order shown or sequential order to achieve the desired results. In certain implementations, multi-task processing and parallel processing are possible or can be advantageous.

[0141] Each of the embodiments in the present application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the difference from other embodiments. Especially, for the device, equipment and storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiments.

[0142] A person of ordinary skill in the art can understand that all or part of the steps of the above-mentioned embodiments can be completed by hardware, or by a program instructing the relevant hardware to complete, and the program can be stored in a computer readable storage medium, which can be a read-only memory, a magnetic disk or an optical disk.

[0143] The above-mentioned is only the preferred embodiment of the present application, and does not limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for calibrating a sensing device, characterized in that, The method comprises: acquiring image information of each target calibration board in a plurality of calibration boards collected by a first sensing device and point cloud data of the each target calibration board in the plurality of calibration boards collected by a second sensing device; performing position detection based on the image information of the each target calibration board to obtain first position information of the each target calibration board in a first coordinate system, the first coordinate system being a three-dimensional coordinate system corresponding to the first sensing device, the first position information comprising first center point information and first normal vector information; performing fitting based on the point cloud data of the each target calibration board to obtain second position information of the each target calibration board in a second coordinate system, the second coordinate system being a three-dimensional coordinate system corresponding to the second sensing device, the second position information being obtained by position calibration based on calibration board edge information fitted based on the point cloud data and calibration board plane information fitted based on the point cloud data, the second position information comprising second center point information and second normal vector information; performing coordinate system conversion processing based on the first position information and the second position information of each target calibration board in the plurality of calibration boards to obtain target conversion parameters between the first coordinate system and the second coordinate system, the target conversion parameters comprising rotation parameters and translation parameters, the rotation parameters being obtained by coordinate system rotation calculation based on first normal vectors corresponding to the plurality of calibration boards and second normal vectors corresponding to the plurality of calibration boards, and the translation parameters being obtained by coordinate system translation calculation based on first center point information corresponding to the plurality of calibration boards, second center point information corresponding to the plurality of calibration boards, and the rotation parameters.

2. The method of claim 1, wherein, The coordinate system conversion processing based on the first position information and the second position information of each target calibration board in the plurality of calibration boards to obtain target conversion parameters between the first coordinate system and the second coordinate system comprises: integrating the first center point information of the each target calibration board and the first normal vector information of the each target calibration board in the plurality of calibration boards respectively to obtain a first center point matrix corresponding to the first center point information and a first normal vector matrix corresponding to the first normal vector information; integrating the second center point information of the each target calibration board and the second normal vector information of the each target calibration board in the plurality of calibration boards respectively to obtain a second center point matrix corresponding to the second center point information and a second normal vector matrix corresponding to the second normal vector information; performing coordinate system rotation calculation based on the first normal vector matrix and the second normal vector matrix to obtain the rotation parameters; performing coordinate system translation calculation based on the first center point matrix, the second center point matrix, and the rotation parameters to obtain the translation parameters.

3. The method of claim 2, wherein, The coordinate system rotation calculation based on the first normal vector matrix and the second normal vector matrix to obtain the rotation parameters comprises: multiplying the second normal vector matrix and a transpose matrix of the second normal vector matrix to obtain a first matrix; obtaining an inverse matrix of the first matrix; The second normal vector matrix is multiplied by a transpose matrix of the first normal vector matrix to obtain a second matrix; A transpose matrix of the second matrix is obtained; The inverse matrix of the first matrix is multiplied by the transpose matrix of the second matrix to obtain the rotation parameter.

4. The method of claim 2, wherein, The translation parameter is obtained by performing coordinate system translation calculation based on the first center point matrix, the second center point matrix and the rotation parameter, including: multiplying the second center point matrix and the rotation parameter to obtain a third matrix; The first center point matrix is subtracted from the third matrix to obtain a fourth matrix; The fourth matrix is averaged to obtain the translation parameter.

5. The method of claim 2, wherein, The second position information of each target calibration plate in the second coordinate system is obtained by fitting the point cloud data of each target calibration plate, including: fitting the point cloud data of each target calibration plate by using a preset algorithm to obtain calibration plate plane information of each target calibration plate; The second normal vector information is obtained based on the calibration plate plane information of each target calibration plate.

6. The method of claim 2, wherein, The second position information of each target calibration plate in the second coordinate system is obtained by fitting the point cloud data of each target calibration plate, including: fitting the point cloud data corresponding to the edges of each target calibration plate to obtain calibration plate edge information of each target calibration plate; The vertex coordinate information of each target calibration plate is obtained by performing edge connection processing based on the calibration plate edge information of each target calibration plate; The second center point information of each target calibration plate is obtained by performing line connection processing based on the vertex coordinate information of each target calibration plate.

7. The method of claim 2, wherein, The first position information of each target calibration plate in the first coordinate system is obtained by performing position detection based on the image information of each target calibration plate, including: performing image recognition processing based on the image information of each target calibration plate to obtain corner point information corresponding to each target calibration plate; The first normal vector information and the first center point information corresponding to each target calibration plate are obtained by performing coordinate calculation processing based on the corner point information corresponding to each target calibration plate.

8. A sensor device calibration apparatus, characterized by The device comprises: An initial information acquisition module is configured to acquire image information of each target calibration plate in a plurality of calibration plates collected by a first sensing device and point cloud data of the each target calibration plate in the plurality of calibration plates collected by a second sensing device; A first coordinate system calculation module is configured to perform position detection based on the image information of each target calibration plate to obtain first position information of the each target calibration plate in a first coordinate system, the first coordinate system being a three-dimensional coordinate system corresponding to the first sensing device, and the first position information including first center point information and first normal vector information. a second coordinate system calculation module, configured to perform fitting based on the point cloud data of each target calibration plate to obtain second position information of the target calibration plate in a second coordinate system, the second coordinate system being a three-dimensional coordinate system corresponding to the second sensing device, the second position information being obtained by position calibration based on fitted calibration plate edge information and fitted calibration plate plane information based on the point cloud data, and the second position information including second center point information and second normal vector information; a target conversion parameter determination module, configured to perform coordinate system conversion processing based on the first position information and the second position information of each target calibration plate in the plurality of calibration plates to obtain target conversion parameters between the first coordinate system and the second coordinate system, the target conversion parameters including rotation parameters and translation parameters, the rotation parameters being obtained by coordinate system rotation calculation based on the first normal vectors corresponding to the plurality of calibration plates and the second normal vectors corresponding to the plurality of calibration plates, and the translation parameters being obtained by coordinate system translation calculation based on the first center point information corresponding to the plurality of calibration plates, the second center point information corresponding to the plurality of calibration plates, and the rotation parameters.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by the processor to implement the sensing device calibration method according to any one of claims 1-7.

10. An electronic device, comprising: The device includes at least one processor and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the at least one processor implements the sensing device calibration method according to any one of claims 1-7 by executing the instructions stored in the memory.

Images