Sensor calibration method, computer equipment and storage medium

By acquiring the data of sensors and radar devices for dedistortion processing and matching, the problems of mechanical tooling and complex operations during sensor calibration are solved, and an efficient and low-cost calibration and calibration process is achieved.

CN118711170BActive Publication Date: 2024-11-29HANGZHOU HUACHENG SOFTWARE TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202411200792.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-29
Publication Date
2024-11-29
Estimated Expiration
2044-08-29

AI Technical Summary

Technical Problem

Existing sensor calibration processes require additional mechanical workouts and complex operations, resulting in high cost and inefficiency.

Method used

By obtaining the sensing data of the target device and the point cloud data collected by the radar device, dedistortion processing is performed, and matching it with the preset grid map, adjusting the point cloud data until the matching conditions are met, the target error term is determined, and the sensor automatic calibration is achieved .

Benefits of technology

Fully automatic and low-cost sensor calibration and calibration are achieved, avoiding additional tooling and complex operations, and improving calibration efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118711170B_ABST
    Figure CN118711170B_ABST
Patent Text Reader

Abstract

The present application discloses a sensor calibration method, a computer device and a storage medium. The method includes: obtaining sensor data collected by a target sensor of a target device and point cloud data collected by a radar device; using the sensor data, performing dedistortion processing on the point cloud data to obtain a dedistorted point cloud; matching the dedistorted point cloud with a preset grid map, and adjusting the dedistorted point cloud according to the matching result until the adjusted dedistorted point cloud and the preset grid map meet the preset matching condition; determining a target error term based on the adjusted dedistorted point cloud; and calibrating the target sensor based on the target error term. The above scheme can improve the efficiency of sensor calibration.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of sensor calibration, and in particular to a sensor calibration method, computer equipment and storage medium. Background Art

[0002] As devices become more intelligent, more and more sensors are used in smart devices. Due to the working conditions of sensors and other reasons, the accuracy of general sensors is poor, and usually their errors need to be calibrated and calibrated.

[0003] At present, the calibration of sensors generally requires additional mechanical tools and complex operating procedures to complete, which has a high calibration cost and low calibration efficiency. Summary of the invention

[0004] The main technical problem solved by the present application is to provide a sensor calibration method, computer equipment and storage medium, which can improve the efficiency of sensor calibration.

[0005] In order to solve the above problems, the first aspect of the present application provides a sensor calibration method, which includes: obtaining sensor data collected by a target sensor of a target device and point cloud data collected by a radar device; using the sensor data, dedistorting the point cloud data to obtain a dedistorted point cloud; matching the dedistorted point cloud with a preset grid map, and adjusting the dedistorted point cloud according to the matching result until the adjusted dedistorted point cloud and the preset grid map meet preset matching conditions; determining a target error term based on the adjusted dedistorted point cloud; and calibrating the target sensor based on the target error term.

[0006] In order to solve the above problem, the second aspect of the present application provides a computer device, which includes a memory and a processor coupled to each other, the memory stores program data, and the processor is used to execute the program data to implement any step of the above sensor calibration method.

[0007] In order to solve the above-mentioned problem, the third aspect of the present application provides a computer-readable storage medium, which stores program data that can be executed by a processor, and the program data is used to implement any step of the above-mentioned sensor calibration method.

[0008] The above scheme obtains the sensor data collected by the target sensor of the target device and the point cloud data collected by the radar device, and uses the sensor data to dedistort the point cloud data to obtain a dedistorted point cloud. Then, the dedistorted point cloud is matched with a preset grid map, and the dedistorted point cloud is adjusted according to the matching result until the adjusted dedistorted point cloud and the preset grid map meet the preset matching conditions. Based on the adjusted dedistorted point cloud, the target error term is determined; based on the target error term, the target sensor is calibrated, and the target error term is obtained by using the inverse process of dedistortion of the point cloud data. The sensor calibration and calibration process can be completed fully automatically and at low cost without the need for additional external tooling equipment and complex operating procedures, which can improve the calibration efficiency of the sensor.

[0009] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in this application, the following briefly introduces the drawings required in the description of the embodiments. Obviously, the drawings described below are only some embodiments of this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0011] Figure 1 It is a flow chart of the first embodiment of the sensor calibration method of the present application;

[0012] Figure 2 This is an exemplary schematic diagram of an embodiment of motion distortion of point cloud data of the present application;

[0013] Figure 3 This is an example schematic diagram of an embodiment of the present application before and after point cloud data dedistortion;

[0014] Figure 4 It is a flow chart of the second embodiment of the sensor calibration method of the present application;

[0015] Figure 5 It is a structural schematic diagram of an embodiment of a calibration device for a sensor of the present application;

[0016] Figure 6 It is a structural schematic diagram of an embodiment of a computer device of the present application;

[0017] Figure 7 It is a structural diagram of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0019] The terms "first" and "second" in this application are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Thus, the features defined as "first" and "second" can explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "multiple" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes other steps or units inherent to these processes, methods, products or devices.

[0020] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0021] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the objects associated before and after are in an "or" relationship. In addition, "many" in this article means two or more than two. In addition, the term "at least one" in this article means any combination of at least two of any one or more of a plurality of, for example, including at least one of A, B, and C, can mean including any one or more elements selected from the set consisting of A, B, and C.

[0022] The present application provides the following embodiments, and each embodiment is described in detail below.

[0023] It is understandable that the sensor calibration method in the present application can be executed by a computer device, and the computer device can be any computer device capable of executing the method embodiment of the present application, wherein the computer device can be an intelligent mobile device, a mobile robot, etc., and the present application does not limit this. In some possible implementations, the method can be implemented by a processor calling program data stored in a memory.

[0024] See also Figure 1 , Figure 1 1 is a flow chart of the first embodiment of the sensor calibration method of the present application. The method may include the following steps:

[0025] S11: Acquire sensor data collected by a target sensor of a target device and point cloud data collected by a radar device.

[0026] Target devices include intelligent mobile devices, mobile robots, etc., for example, sweeping robots, AGVs (Automated Guided Vehicles), etc. Target devices can include any movable devices equipped with target sensors and radar devices. Target sensors and radar devices can be used to implement functions such as positioning and navigation of target devices. Exemplarily, the following description takes the target device as a mobile robot, and this application does not limit the target device. Since the target device is equipped with a target sensor and a radar device, the target device can obtain the sensor data collected by the target sensor during the movement / working process, and obtain the point cloud data collected by the radar device.

[0027] For radar devices, the types of radar devices may include continuous wave radar, pulse radar or phased array radar, etc., and the radar device may be used to collect point cloud data. For example, it may be a laser radar, a millimeter wave radar, etc., and this application takes a laser radar as an example for explanation. This application does not limit the radar device.

[0028] The target sensor of the present application may be an IMU (Inertial Measurement Unit), a mileage sensor (such as a wheel speed odometer), etc. The target sensor can be used to collect sensor data. Among them, the inertial measurement unit includes an accelerometer and a gyroscope, which are mainly used to measure linear acceleration and angular velocity, respectively. The wheel speed odometer tracks the rotation speed of each wheel based on a photoelectric encoder installed on the target device. It is understandable that the target sensor of the present application may also include other sensors, such as other sensors for navigation, positioning, etc. Exemplarily, the following description takes the target sensor as an IMU as an example, and the present application does not limit the target sensor.

[0029] In some embodiments, before step S11, a preset grid map may be acquired, wherein the preset grid map is obtained using point cloud data collected in the first motion state.

[0030] In some implementations, obtaining a preset grid map may include the following steps:

[0031] Control the target device to collect several frames of point cloud data in a first motion state. Exemplarily, the mobile robot can be controlled to move in a first motion state within a first area, so that the installed radar device can collect several frames of point cloud data. Among them, the first motion state includes a motion speed lower than or equal to a first preset speed, and / or, in the first motion state, the maximum motion distortion error caused by the collected point cloud data is less than a preset error, and the preset error includes a preset ratio of the ranging accuracy of the radar device. The preset ratio is, for example, 0.5, 0.4, etc., and the present application does not limit the preset ratio. In some application scenarios, when the motion speed is not greater than the first preset speed, the maximum motion distortion error caused by the point cloud data collected by the mobile robot is less than the preset error, and the preset error includes a preset ratio of the ranging accuracy of the radar device. Exemplarily, the first motion state can be a low-speed disordered motion, and the low speed can refer to a motion speed lower than or equal to the first preset speed. In the first motion state, during the process of the radar device scanning a circle of point cloud data, the maximum motion distortion error caused by the motion speed is less than the preset ratio of the ranging accuracy of the radar device. The present application does not limit the first motion state.

[0032] During the movement, the point cloud data of the first frame is used as the reference point cloud, that is, as the benchmark, and the point cloud data of other frames are matched with the reference point cloud frame by frame, such as matching the point cloud data of other frames with the point cloud data of the first frame in turn. The matching method may include a brute force point cloud matching method, SLAM technology (Simultaneous Localization and Mapping, simultaneous positioning and map construction), etc. This application does not limit the matching method. After matching, the reference pose of the point cloud data of each frame can be obtained, that is, the pose in the coordinate system of the reference point cloud can be obtained.

[0033] Then, the reference poses of several frames of point cloud data are converted to the grid map to obtain a preset grid map, that is, a grid map of the surrounding environment (such as the surrounding environment of the first area). In the above process, since the mobile robot moves in the first motion state and maintains a low motion speed, it can be considered that the collected point cloud data has no motion distortion, and the generated preset grid map can be considered to have no error introduced by motion distortion, and is a relatively accurate grid map, which can be used as a grid map for subsequent movement reference.

[0034] After the preset grid map is constructed, the above step S11 may be performed.

[0035] During the above step S11, the target device can be controlled to move in a second motion state, such as moving in a second motion state within a second area, wherein the second area can be the same as the first area or include the first area or be different from the first area. The second motion state includes a motion speed greater than or equal to a second preset speed. The second preset speed is greater than or equal to the first preset speed. The second motion state can be a disordered motion at a second preset speed, and the second preset speed can refer to the maximum possible working speed of the mobile robot. The second motion state can be fast or orderly, etc., and can be determined according to the working state of a specific application scenario, etc., and the present application does not limit the second motion state.

[0036] The target device acquires the sensing data collected by the target sensor and the point cloud data collected by the radar device in the second motion state. Point cloud data and sensing data of several frames or a circle may be collected. The point cloud data in the following embodiments refers to the point cloud data collected in the second motion state, and this application does not limit this.

[0037] S12: Dedistort the point cloud data using the sensor data to obtain a dedistorted point cloud.

[0038] The sensor data collected in the second motion state may be used to perform dedistortion processing on the point cloud data collected in the second motion state to obtain a dedistorted point cloud. The present application does not limit the dedistortion processing method.

[0039] Due to the working principle of the radar device, the point cloud data measured by the radar device during the movement of the mobile robot has motion distortion errors. Specifically, in the systems of many target devices, multiple sensor devices can be installed, such as IMU, wheel speed odometer, etc., and used together with the radar device to achieve the function of positioning and mapping through information fusion. Since most radar devices use mechanical rotating laser radars, which generally have only one laser transmitter and receiver, they scan the surrounding environment point by point by rotating to obtain a circle of point cloud data about the surrounding environment. The emission point of all point cloud data is the center of the laser radar coordinate system by default, so if the laser radar moves during the data collection process, the emission points of a circle of point cloud data will not overlap. If all the emission points of the point cloud are considered to overlap during the use of the point cloud data, a large error will occur. This point cloud data error caused by the movement of the laser radar is called motion distortion.

[0040] See also Figure 2, L1 and L2 are two positions in the laser radar's scanning process. The laser radar's point cloud P is the data measured by the laser radar at the L2 position. If it is used directly without considering motion distortion, the point cloud P will be regarded as the data measured at the L1 position, and the calculated point cloud data will be the point cloud P'. In fact, the point cloud P is the data measured at the L2 position after moving relative to the L1 position. This point cloud data error caused by the laser radar movement is called motion distortion. In order to reduce the impact of motion distortion on the accuracy of point cloud data, motion distortion can generally be calibrated. This process is generally called distortion removal. In some application scenarios, distortion removal is to solve the position information of point cloud P relative to L1.

[0041] Exemplarily, IMU is mainly used to measure the angular velocity and linear acceleration of a mobile robot during its motion, and calculates the motion angle and motion displacement through primary and secondary integration, respectively. Due to the large noise and initial error of IMU (such as low-cost consumer-grade IMU), relevant error term parameters need to be calibrated and calibrated before use. IMU is generally calibrated and calibrated through high-precision two-axis or three-axis rotating fixtures, and the calibration cost is high and the efficiency is low. In this regard, the present application can de-distort the point cloud data through sensor data to obtain a de-distorted point cloud, and then achieve automatic, low-cost, and accurate calibration and calibration of sensors such as IMU by adopting the inverse de-distortion process of the point cloud data.

[0042] The dedistortion process of the point cloud data can obtain the relative posture relationship between each point cloud data and the reference coordinate system (such as the coordinate system of the first point cloud data). The more accurate the relative posture relationship is, the better the dedistortion effect is.

[0043] S13: matching the dedistorted point cloud with a preset grid map, and adjusting the dedistorted point cloud according to the matching result, until the adjusted dedistorted point cloud and the preset grid map meet a preset matching condition.

[0044] The dedistorted point cloud can be matched with the preset grid map to obtain a matching result, which can be a matching score. The larger the matching score, the higher the corresponding matching degree. The matching score can be a value of 0-1, and this application does not limit the matching result.

[0045] The dedistorted point cloud can be adjusted according to the matching results, and the dedistorted point cloud can be continuously optimized and adjusted until the adjusted dedistorted point cloud and the preset grid map meet the preset matching conditions. The preset matching conditions can be that the matching score is greater than the preset score or meets the optimization conditions, etc. This application does not limit the preset matching conditions.

[0046] For example, see Figure 3 , Figure 31 is an example schematic diagram of an embodiment of the present application before and after point cloud data dedistortion. Figure 3 Different colors can be used to represent the point cloud data before and after dedistortion. For example, red points are used to represent the point cloud data before dedistortion, and green points are used to represent the point cloud data after dedistortion. If the mobile robot moves at a fast speed, the error of the point cloud data due to motion distortion will be large, and the shape of the point cloud output by the radar device will be greatly different from the actual environment. Directly matching the point cloud data collected by fast motion with the preset grid map generated by slow motion will not obtain accurate matching results. If the motion distortion error of the point cloud data can be eliminated or reduced, the point cloud data may have a more accurate matching result with the preset grid map. That is, by dedistorting the point cloud data, the dedistorted point cloud and the preset grid map can obtain a more accurate matching result.

[0047] Then, by adjusting the dedistorted point cloud according to the matching result, the matching result between the adjusted dedistorted point cloud and the preset grid map can be made more accurate.

[0048] S14: Determine a target error term based on the adjusted dedistorted point cloud.

[0049] Since the motion distortion of the dedistorted point cloud is eliminated or reduced, and the adjusted dedistorted point cloud and the preset grid map meet the preset matching conditions, the target error term is determined based on the adjusted dedistorted point cloud. The better the matching result between the adjusted dedistorted point cloud and the preset grid map, the smaller the error of the point cloud data is, the better the dedistortion effect is, and thus, the higher the accuracy of the calculated target error term is.

[0050] In some implementations, the adjusted dedistorted point cloud may be compared with the sensor data to determine the target error term. Alternatively, since the dedistorted point cloud is obtained by dedistortion processing using the sensor data, and the adjusted dedistorted point cloud is obtained by matching with a preset grid map, etc., the target error term may be determined directly based on the adjusted dedistorted point cloud.

[0051] S15: Calibrate the target sensor based on the target error term.

[0052] According to the obtained target error term, the target sensor is calibrated and calibrated to improve / correct the acquisition accuracy of the target sensor. Since a more accurate target error term can be obtained, the calibration and calibration are more accurate. This application does not limit the calibration and calibration process.

[0053] The above scheme obtains the sensor data collected by the target sensor of the target device and the point cloud data collected by the radar device, and uses the sensor data to dedistort the point cloud data to obtain a dedistorted point cloud. Then, the dedistorted point cloud is matched with a preset grid map, and the dedistorted point cloud is adjusted according to the matching result until the adjusted dedistorted point cloud and the preset grid map meet the preset matching conditions. Based on the adjusted dedistorted point cloud, the target error term is determined; based on the target error term, the target sensor is calibrated, and the target error term is obtained by using the inverse process of dedistortion of the point cloud data. The sensor calibration and calibration process can be completed fully automatically and at low cost without the need for additional external tooling equipment and complex operating procedures, which can improve the calibration efficiency of the sensor.

[0054] Regarding the steps of the above embodiment, steps S12 to S14 of the above embodiment can be further expanded, and the target error term can be obtained by adopting a preset optimization model. For details, please refer to the following embodiment.

[0055] See also Figure 4 , Figure 4 1 is a flow chart of a second embodiment of a sensor calibration method of the present application. The method may include the following steps:

[0056] S21: using each sensor data to determine a motion transformation relationship; wherein each sensor data is represented by sensor data with an initial error term.

[0057] In some implementations, the above step S12 may include steps S21 to S22 of this embodiment.

[0058] The point cloud data includes a plurality of point cloud data, and correspondingly, the sensor data includes a plurality of sensor data. The plurality of point cloud data can correspond to the plurality of sensor data one by one, such as by using a timestamp to correspond the plurality of point cloud data to the plurality of sensor data one by one. That is, the point cloud data and the sensor data collected by the mobile robot at the same position can correspond.

[0059] Then, each sensing data is represented by sensing data with an initial error term, and the motion transformation relationship is determined using each sensing data.

[0060] In some embodiments, the motion transformation relationship includes a rotation parameter and a translation parameter. The rotation parameter may represent a rotation matrix converted from other coordinate systems to the reference coordinate system, and the translation parameter may represent a translation vector converted from other coordinate systems to the reference coordinate system.

[0061] Each sensor data is represented by sensor data with an initial error term. The initial error term and the target error term include at least one of the following: an axis deviation sub-error term, a scale sub-error term, a drift sub-error term, a white noise sub-error term, etc. The axis deviation sub-error term can be used to represent the transformation matrix of the axis deviation, the scale sub-error term can be used to represent the scale factor, the drift sub-error term can be used to represent zero bias and random walk, and the white noise sub-error term can be used to represent white noise. It can be understood that other sub-error terms can also be included, and the present application does not limit the initial error term and the target error term.

[0062] In some embodiments, the sensor data can be represented by the product of the sensor data and the axis deviation sub-error term, and the product of the sensor data and the sum of errors such as the scale sub-error term, the drift sub-error term, and the white noise sub-error term. This application does not impose any restrictions on this.

[0063] For example, taking IMU as an example, the accelerometer and gyroscope included in the IMU can be used to collect linear acceleration and angular velocity respectively, and the linear acceleration and angular velocity can be represented by the linear acceleration and angular velocity of the initial error term. Among them, the linear acceleration and angular velocity can be represented by the following formulas respectively:

[0064] Formula 1

[0065] Formula 2

[0066] Among them, a represents linear acceleration, w represents angular velocity, L represents linear accelerometer, g represents gyroscope, B represents orthogonal reference coordinate system, S represents non-orthogonal reference coordinate system, T represents axis deviation sub-error term (that is, the transformation matrix of axis deviation), K represents scale sub-error term (that is, scale factor), b represents drift sub-error term (that is, zero bias and random walk), and n represents white noise sub-error term.

[0067] Then, each sensor data is integrated to obtain the relative angle change and relative translation change. The integration processing may be a primary integration processing, a secondary integration processing, etc. For different sensor data, different methods may be used to determine the relative angle change and relative translation change (i.e., displacement change), and this application does not limit this.

[0068] For example, the linear acceleration Perform secondary integration to obtain the relative translation change The secondary integration process can include the initial linear acceleration and the linear acceleration The sum of is integrated to obtain the linear velocity, and then the linear velocity is integrated to obtain the relative translation change. Among them, the secondary integral processing can be expressed by the following formula:

[0069] Formula 3

[0070] Formula 4

[0071] Where v represents the linear velocity, Represents the relative translation change, represents the initial linear acceleration, and t represents the time.

[0072] For example, the angular velocity Perform an integral process to obtain the relative angle change , where one integral process can be expressed by the following formula:

[0073] Formula 5

[0074] Thus, the relative angle variation is used to determine the rotation parameter; and the relative translation variation is used to determine the translation parameter. That is, the relative angle variation and the relative translation variation can be converted to obtain the rotation parameter and the translation parameter. For example, the relative angle variation can be converted to the relative translation variation. Using the first function To convert, the first function It can be used to represent the process of converting Euler angles to rotation matrices to obtain rotation parameters . The relative translation change can be Using the second function To convert, the second function It can be used to represent the process of converting displacement to translation vector and obtain the translation parameter t. The process of determining the rotation parameter using the relative angle change and the process of determining the translation parameter using the relative translation change can be represented by the following formulas:

[0075] Formula 6

[0076] t Formula 7

[0077] Continue reading Figure 2 Taking L1 and L2 coordinate systems as an example, the L1 coordinate system can be used as the reference coordinate system. By processing the sensor data collected by L1 and L2 in the above manner, the rotation parameter R can be obtained. 12 and the translation parameter t 12 The rotation parameter R 12 Represents the rotation parameter and translation parameter t from the L2 coordinate system to the L1 coordinate system 12 Represents the translation parameter from the L2 coordinate system to the L1 coordinate system.

[0078] It should be noted that, when the target sensor includes other sensors, the sensor data collected by other sensors can be processed to obtain rotation parameters and translation parameters, and this application does not impose any restrictions on this.

[0079] In the above method, the relative posture transformation between the two coordinate systems can be obtained by using the sensor data, and the motion transformation relationship converted to the reference coordinate system can be obtained, that is, the relative posture transformation between the other coordinate systems and the reference coordinate system.

[0080] S22: Determine the point cloud pose of each point cloud data in the reference coordinate system by using the motion transformation relationship as the dedistorted point cloud.

[0081] After determining the motion transformation relationship, each point cloud data can be transformed by the motion transformation relationship to convert it to the reference coordinate system to obtain the point cloud pose of each point cloud data in the reference coordinate system. The point cloud pose in the reference coordinate system can be used as the dedistorted point cloud. Therefore, in the subsequent matching process, each point cloud pose is used to correspond to each point cloud data for matching.

[0082] In some embodiments, one of the point cloud data can be used as the target point cloud data, such as the first point cloud data as the target point cloud data, and the coordinate system of the target point cloud data as the reference coordinate system. The remaining point cloud data are used as other point cloud data. The timestamps of the target point cloud data and the sensor data corresponding to the reference coordinate system are corresponding. According to the above-mentioned motion transformation relationship, each other point cloud data can be converted to the reference coordinate system. Specifically, the motion transformation relationship can be used to transform the posture of other point cloud data to obtain the point cloud posture of other point cloud data in the reference coordinate system. The posture of each point cloud is expressed as: the product of other point cloud data and the rotation parameter, and the sum of the translation parameters. Therefore, all point cloud data of one circle / all frames can be converted to the reference coordinate system (such as the coordinate system of the first point cloud data) in the above manner.

[0083] Exemplarily, the pose of each point cloud data can be transformed by using the motion transformation relationship through the following formula to obtain the point cloud pose of each point cloud data in the reference coordinate system. The formula is as follows:

[0084] Formula 8

[0085] in, represents the point cloud pose in coordinate system i, represents the rotation parameter from coordinate system j to coordinate system i, represents the translation parameter from coordinate system j to coordinate system i, Represents point cloud data (like other point cloud data).

[0086] For example, see Figure 2Taking the L1 and L2 coordinate systems as examples, the L1 coordinate system can be used as the reference coordinate system, and the point cloud data can be converted from the L2 coordinate system to the L1 coordinate system using Formula 8 to obtain the point cloud pose in the L1 coordinate system.

[0087] Since the above process converts each point cloud data into a reference coordinate system, it can be considered that the emission centers of all point cloud data coincide with each other, that is, the point cloud has no motion distortion error and can be used as a dedistorted point cloud.

[0088] In some implementations, the above steps S12 to S13 may include the following steps:

[0089] S23: using a preset optimization model to match the dedistorted point cloud with a preset grid map to obtain a matching result, optimizing the dedistorted point cloud according to the matching result, and outputting a target error term.

[0090] The dedistorted point cloud is represented by point cloud data containing initial error terms, and the optimization processing of the dedistorted point cloud according to the matching results includes: optimizing and adjusting the initial error terms according to the matching results until the dedistorted point cloud after optimization and adjustment and the preset grid map meet the preset matching conditions. Since the initial error terms include at least one of the following: axis deviation sub-error term, scale sub-error term, drift sub-error term, and white noise sub-error term, correspondingly, the target error terms include at least one of the following: axis deviation sub-error term T, scale sub-error term K, drift sub-error term b, and white noise sub-error term n.

[0091] Through the above process, the problem of obtaining the target error term can be converted into an optimization problem, so only a certain amount of data needs to be collected to obtain the target error term. Matching the dedistorted point cloud data with the preset grid map will get an optimal matching score. The smaller the point cloud distortion error, the closer the matching score is to 1. Conversely, the higher the matching score, the smaller the point cloud error, the better the dedistortion effect, the higher the IMU solution accuracy, and the more accurate the IMU target error term calibration and calibration.

[0092] In some implementations, a preset optimization model may be used to match the dedistorted point cloud with a preset grid map to obtain a matching result, and the dedistorted point cloud may be optimized according to the matching result to output a target error term.

[0093] The matching result may represent the matching scores of each dedistorted point cloud and the preset grid map. Alternatively, the matching result may represent the matching difference of each dedistorted point cloud, where the matching difference is the difference between the matching score of each dedistorted point cloud and the preset grid map and a preset threshold. Alternatively, the matching result may represent the sum of the differences of the matching differences of each dedistorted point cloud.

[0094] Among them, the optimization processing of the dedistorted point cloud according to the matching result includes: optimization processing. Such as minimization processing, maximization processing, etc. For example, the matching result represents the matching score, which can be a maximization processing; for example, the matching result represents the matching difference or the sum of the differences, which can be a minimization processing, etc. The present application does not limit the optimization processing. Correspondingly, the preset matching conditions may include optimization conditions, such as minimization conditions, maximization conditions, etc. Exemplarily, the optimization conditions include reaching the maximum number of iterations, or the sum of the differences is less than the preset total difference, or the matching difference is less than the preset difference, or the matching score is greater than the preset score, etc. The present application does not limit the preset matching conditions.

[0095] Exemplarily, optimizing the dedistorted point cloud according to the matching result may include minimizing the matching result. Specifically, the matching scores corresponding to the poses of each point cloud (i.e., the dedistorted point cloud) and the preset grid map are obtained respectively, and then the matching difference between the matching score and the preset threshold is obtained, where, for example, the matching score is a value between 0 and 1, and the preset threshold can be 1. Then, the sum of the difference of the matching difference corresponding to each dedistorted point cloud is obtained, and the sum of the difference is minimized, so as to output the target error term.

[0096] The above-mentioned optimization and adjustment of the initial error term according to the matching result can be expressed as minimization processing based on the sum of differences, and the initial error term is optimized and adjusted in an optimized iterative manner until the dedistorted point cloud after optimization and adjustment and the preset grid map meet the preset matching conditions, that is, the value of the sum of matching differences meets the minimization condition, thereby outputting the target error term.

[0097] Among them, the preset optimization model can be expressed by the following formula:

[0098] Formula 9

[0099] in, represents the target error term, represents the matching function between the dedistorted point cloud and the preset grid map. k is a positive integer, which represents the total number of point cloud poses, that is, the total number of dedistorted point clouds. m represents the number of point cloud poses, that is, the number of dedistorted point clouds. In some application scenarios, it can be understood that the formula 8 can be replaced by Substituting into formula 9, we get Substitute the f3 matching function in Formula 9 to match with the preset grid map.

[0100] The preset optimization model contains the unknown error terms such as axis deviation sub-error term T, scale sub-error term K, drift sub-error term b, and white noise sub-error term n waiting for calibration and calibration. A large number of error terms can be constructed through the preset optimization model, and the preset optimization model, i.e. By minimizing the processing, the matching scores of all point cloud poses and the preset grid map can reach the optimal value, and the target error term can be output.

[0101] Through the above method, a tightly coupled preset optimization model is used to put all error terms together for iterative optimization and solution, which can well solve the coupling properties between the error terms and make the calibration accuracy of the error terms reach an optimal balanced state, so that each error term can obtain the best calibration and calibration value.

[0102] In addition, by using the inverse process of point cloud data dedistortion to obtain the target error term, the sensor calibration and calibration process can be completed fully automatically and at low cost, without the need for additional external tooling equipment and complex operating procedures, thereby improving the calibration efficiency of the sensor.

[0103] It is understandable that in the above method of the specific implementation mode, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0104] With respect to the above embodiments, the present application provides a sensor calibration device, which is used to implement the sensor calibration method of any of the above embodiments.

[0105] See also Figure 5 , Figure 5 The sensor calibration device 30 of the present invention comprises: an acquisition module 31 , a de-distortion module 32 , an error term module 33 and a calibration module 34 .

[0106] The acquisition module 31 is used to obtain the sensing data collected by the target sensor of the target device and the point cloud data collected by the radar device.

[0107] The dedistortion module 32 is used to perform dedistortion processing on the point cloud data using the sensor data to obtain a dedistorted point cloud.

[0108] The error term module 33 is used to match the dedistorted point cloud with a preset grid map, and adjust the dedistorted point cloud according to the matching result until the adjusted dedistorted point cloud and the preset grid map meet the preset matching condition; based on the adjusted dedistorted point cloud, determine the target error term.

[0109] The calibration module 34 is used to calibrate the target sensor based on the target error term.

[0110] It should be noted that the sensor calibration device provided in the above embodiment and the sensor calibration method provided in the above embodiment belong to the same concept, wherein the specific manner in which each module and unit performs the operation has been described in detail in the method embodiment, and this application will not repeat it here. In practical applications, the sensor calibration device provided in the above embodiment can distribute the above functions to different functional modules as needed, that is, divide the internal structure of the device into different functional modules to complete all or part of the functions described above, and this application does not limit this.

[0111] For the above embodiment, the present application provides a computer device, see Figure 6 , Figure 6 4 is a schematic diagram of a computer device of the present application. The computer device 40 includes a memory 41 and a processor 42, wherein the memory 41 and the processor 42 are coupled to each other, the memory 41 stores program data, and the processor 42 is used to execute the program data to implement the steps of any embodiment of the sensor calibration method described above.

[0112] In this embodiment, the processor 42 may also be referred to as a CPU (Central Processing Unit). The processor 42 may be an integrated circuit chip having the ability to process signals. The processor 42 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor, or the processor 42 may be any conventional processor, etc.

[0113] The method of the above embodiment can be implemented in the form of a computer program, so the present application proposes a computer readable storage medium, see Figure 7 , Figure 7 The computer-readable storage medium 50 stores program data 51 that can be run by a processor, and the program data 51 can be executed by the processor to implement the steps of any embodiment of the sensor calibration method described above.

[0114] The computer-readable storage medium 50 of this embodiment may be a medium that can store the program data 51, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or may be a server storing the program data 51. The server may send the stored program data 51 to other devices for execution, or may execute the stored program data 51 by itself.

[0115] In some embodiments, the functions or modules included in the device provided in the above embodiments of the present application can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, the present application will not go into details here.

[0116] The above description of the various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other. For the sake of brevity, this application will not go into details here.

[0117] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0118] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0119] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0120] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium, which is a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform all or part of the steps of each implementation method of the present application.

[0121] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a computer-readable storage medium and executed by a computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. In this way, the present application is not limited to any specific combination of hardware and software.

[0122] The above descriptions are merely embodiments of the present application and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A sensor calibration method, It is characterized in that include: Controlling the target device to collect a plurality of frames of point cloud data in a first motion state; wherein the first motion state includes a motion speed lower than or equal to a first preset speed, and / or, in the first motion state, a maximum motion distortion error caused by the collected point cloud data is less than a preset error, and the preset error includes a preset ratio of the ranging accuracy of the radar device; Using the first frame of point cloud data as the reference point cloud, matching each frame of point cloud data with the reference point cloud frame by frame to obtain a reference pose of each frame of point cloud data; Converting the reference poses of the plurality of frames of point cloud data into a grid map to obtain a preset grid map; Acquire sensor data collected by a target sensor of a target device and point cloud data collected by a radar device; Using the sensing data, performing dedistortion processing on the point cloud data to obtain a dedistorted point cloud; Matching the dedistorted point cloud with the preset grid map, and adjusting the dedistorted point cloud according to the matching result until the adjusted dedistorted point cloud and the preset grid map meet a preset matching condition; Determining a target error term based on the adjusted dedistorted point cloud; The target sensor is calibrated based on the target error term.

2. The method according to claim 1, It is characterized in that The dedistorted point cloud is matched with the preset grid map, and the dedistorted point cloud is adjusted according to the matching result until the adjusted dedistorted point cloud and the preset grid map meet a preset matching condition; And, based on the adjusted dedistorted point cloud, the step of determining a target error term comprises: Using a preset optimization model to match the dedistorted point cloud with the preset grid map to obtain a matching result, optimizing the dedistorted point cloud according to the matching result, and outputting the target error term; The dedistorted point cloud is represented by point cloud data including an initial error term, and the optimizing processing of the dedistorted point cloud according to the matching result includes: optimizing and adjusting the initial error term according to the matching result until the optimized and adjusted dedistorted point cloud and the preset grid map meet a preset matching condition.

3. The method according to claim 2, It is characterized in that The initial error term and the target error term include at least one of the following: an axis deviation sub-error term, a scale sub-error term, a drift sub-error term, and a white noise sub-error term.

4. The method according to claim 1, It is characterized in that The point cloud data includes a plurality of point cloud data; the sensor data includes a plurality of sensor data; the plurality of point cloud data corresponds to the plurality of sensor data one by one; and the sensor data is used to perform dedistortion processing on the point cloud data to obtain a dedistorted point cloud, including: Using each sensor data, a motion transformation relationship is determined; wherein each sensor data is represented by sensor data with an initial error term; The motion transformation relationship is used to determine the point cloud pose of each point cloud data in the reference coordinate system as the dedistorted point cloud.

5. The method according to claim 4, It is characterized in that The step of using the motion transformation relationship to determine the point cloud pose of each point cloud data in the reference coordinate system includes: One of the point cloud data is used as the target point cloud data, and the remaining point cloud data is used as other point cloud data, and the coordinate system of the target point cloud data is used as the reference coordinate system; Performing a pose transformation on the other point cloud data using the motion transformation relationship to obtain a point cloud pose of the other point cloud data in the reference coordinate system; And / or, the motion transformation relationship includes a rotation parameter and a translation parameter; the point cloud pose is represented by: the product of other point cloud data and the rotation parameter, and the sum of the translation parameter; And / or, the determining of the motion transformation relationship by using the sensor data includes: Each sensor data is represented by a sensor data with an initial error term, and each sensor data is integrated to obtain a relative angle change and a relative translation change; Determining a rotation parameter using the relative angle variation; and The relative translation change is used to determine the translation parameter.

6. The method according to claim 1, It is characterized in that The step of acquiring the sensing data collected by the target sensor of the target device and the point cloud data collected by the radar device includes: Controlling the target device in a second motion state to obtain sensing data collected by a target sensor of the target device and point cloud data collected by a radar device; The second motion state includes a motion speed greater than or equal to a second preset speed.

7. A computer device, It is characterized in that The invention comprises a memory and a processor coupled to each other, wherein the memory stores program data, and the processor is used to execute the program data to implement the steps of the method according to any one of claims 1 to 6.

8. A computer-readable storage medium, It is characterized in that Program data that can be executed by a processor are stored, and the program data is used to implement the steps of the method according to any one of claims 1 to 6.

Citation Information

Patent Citations

  • Multi-sensor fusion high-precision pose estimation algorithm under satellite weak observation condition

    CN112781582A

  • Point cloud data processing method and device, electronic equipment and storage medium

    CN112907747A