Sensor calibration method, device, equipment and automatic driving vehicle

Abstract

The present disclosure provides a sensor calibration method, an electronic device and an autonomous vehicle, and relates to the technical field of autonomous driving and intelligent transportation, in particular to the technical field of sensor calibration and big data processing. The scheme comprises: determining a first moving track and a first inter-frame relative pose of an inertial measurement unit based on first data collected by the inertial measurement unit, determining a second moving track and a second inter-frame relative pose of a sensor based on second data collected by the sensor; determining initial calibration parameters of the sensor relative to the inertial measurement unit, and determining initial inter-frame relative pose of the sensor based on the first inter-frame relative pose and the initial calibration parameters; determining intermediate calibration parameters of the sensor relative to the inertial measurement unit based on the second inter-frame relative pose and the initial inter-frame relative pose; and determining target calibration parameters of the sensor relative to the inertial measurement unit based on the first moving track, the second moving track and the intermediate calibration parameters. The above method can reduce cost and increase efficiency.

Description

Technical Field

[0001] This disclosure relates to the fields of autonomous driving and intelligent transportation technologies, specifically to the fields of sensor calibration and big data processing. Background Technology

[0002] During the operation of autonomous vehicles, multiple sensors on the vehicle need to work together to perceive and locate the vehicle. To ensure the accuracy of the perception and location results, sensor calibration is required in advance to determine the positional relationships between the sensors. However, in related technologies, the calibration process for multiple sensors requires different scenarios and tools, which makes the calibration process overly cumbersome and inefficient. Summary of the Invention

[0003] This disclosure provides a sensor calibration method, an electronic device, and an autonomous vehicle.

[0004] According to a first aspect of this disclosure, a sensor calibration method is provided, the method comprising:

[0005] The first moving trajectory and the first inter-frame relative pose of the inertial measurement unit are determined based on the first data collected by the inertial measurement unit, and the second moving trajectory and the second inter-frame relative pose of the sensor are determined based on the second data collected by the sensor.

[0006] Determine the initial calibration parameters of the sensor relative to the inertial measurement unit, and determine the initial inter-frame relative pose of the sensor based on the first inter-frame relative pose and the initial calibration parameters;

[0007] Based on the second inter-frame relative pose and the initial inter-frame relative pose, the intermediate calibration parameters of the sensor relative to the inertial measurement unit are determined.

[0008] Based on the first movement trajectory, the second movement trajectory, and intermediate calibration parameters, the target calibration parameters of the sensor relative to the inertial measurement unit are determined.

[0009] In some embodiments of this disclosure, the sensor includes a radar; determining the second movement trajectory of the sensor and the relative pose between the second frames based on second data acquired by the sensor includes:

[0010] Acquire multiple consecutive frames of radar data collected by the radar;

[0011] Radar mileage information is determined by using multiple consecutive frames of radar data, and the second movement trajectory of the radar is determined based on the radar mileage information.

[0012] After substituting multiple consecutive frames of radar data into a preset point registration algorithm to optimize the preset residuals, the relative pose between the second frame of the radar is obtained.

[0013] In some embodiments of this disclosure, when the radar is a lidar, the residual is the distance between a point and a surface; when the radar is a millimeter-wave radar, the residual is the distance between points.

[0014] In some embodiments of this disclosure, the sensor includes a camera; determining a second movement trajectory of the sensor and a second inter-frame relative pose based on second data acquired by the sensor includes:

[0015] Acquire multiple consecutive frames of image data captured by the camera;

[0016] Feature points in multiple consecutive frames of image data are determined, and a second motion trajectory to the camera and relative pose between the second frames are determined based on the features.

[0017] In some embodiments of this disclosure, determining feature points in multiple consecutive frames of image data, and determining a second motion trajectory to the camera and a relative pose between the second frames based on the features, includes:

[0018] Determine feature points in multiple consecutive frames of image data;

[0019] Based on preset parallax information, multiple keyframe image data are determined from multiple consecutive image data.

[0020] The feature points of adjacent keyframe image data are homogenized, and then the target keyframe image data is selected from multiple keyframe image data.

[0021] Based on the feature points in the target keyframe image data and the feature points in the last keyframe image data, the second motion trajectory to the camera and the relative pose between the second frames are determined.

[0022] In some embodiments of this disclosure, determining the second motion trajectory to the camera and the relative pose between the second frame based on feature points in the target keyframe image data and feature points in the last keyframe image data includes:

[0023] Triangulation is performed based on feature points in the target keyframe image data and feature points in the last keyframe image data;

[0024] The second movement trajectory of the camera and the relative pose between the second frame are determined based on the triangulated feature points.

[0025] In some embodiments of this disclosure, intermediate calibration parameters of the sensor relative to the inertial measurement unit are determined based on the second inter-frame relative pose and the initial inter-frame relative pose, including any one of the following:

[0026] When the sensor is a lidar, the angle calibration parameters and displacement calibration parameters in the initial calibration parameters are adjusted based on the second inter-frame relative pose and the initial inter-frame relative pose to obtain the intermediate calibration parameters.

[0027] When the sensor is a millimeter-wave radar, the angle calibration parameters in the initial calibration parameters are adjusted based on the second inter-frame relative pose and the initial inter-frame relative pose to obtain the intermediate calibration parameters.

[0028] In some embodiments of this disclosure, intermediate calibration parameters of the sensor relative to the inertial measurement unit are determined based on the second inter-frame relative pose and the initial inter-frame relative pose, including:

[0029] When the sensor is a camera, the angle calibration parameters in the initial calibration parameters are adjusted based on the second inter-frame relative pose and the initial inter-frame relative pose to obtain the intermediate calibration parameters.

[0030] In some embodiments of this disclosure, the target calibration parameters of the sensor relative to the inertial measurement unit are determined based on a first movement trajectory, a second movement trajectory, and intermediate calibration parameters, including:

[0031] An alignment process is performed on the first and second moving trajectories. During the alignment process, intermediate calibration parameters are adjusted to obtain the target calibration parameters of the sensor relative to the inertial measurement unit.

[0032] According to a second aspect of this disclosure, a sensor calibration device is provided, which includes an initial data acquisition module, an initial pose determination module, an intermediate parameter determination module, and a target parameter determination module.

[0033] The initial data acquisition module is used to determine the first movement trajectory and the first inter-frame relative pose of the inertial measurement unit based on the first data collected by the inertial measurement unit, and to determine the second movement trajectory and the second inter-frame relative pose of the sensor based on the second data collected by the sensor.

[0034] The initial pose determination module is used to determine the initial calibration parameters of the sensor relative to the inertial measurement unit, and to determine the initial inter-frame relative pose of the sensor based on the first inter-frame relative pose and the initial calibration parameters.

[0035] The intermediate parameter determination module is used to determine the intermediate calibration parameters of the sensor relative to the inertial measurement unit based on the second inter-frame relative pose and the initial inter-frame relative pose.

[0036] The target parameter determination module is used to determine the target calibration parameters of the sensor relative to the inertial measurement unit based on the first movement trajectory, the second movement trajectory, and intermediate calibration parameters.

[0037] In some embodiments of this disclosure, the sensor includes a radar; the initial data acquisition module, when determining the second movement trajectory of the sensor and the relative pose between the second and second frames based on the second data acquired by the sensor, is specifically used for:

[0038] Acquire multiple consecutive frames of radar data collected by the radar;

[0039] Radar mileage information is determined by using multiple consecutive frames of radar data, and the second movement trajectory of the radar is determined based on the radar mileage information.

[0040] After substituting multiple consecutive frames of radar data into a preset point registration algorithm to optimize the preset residuals, the relative pose between the second frame of the radar is obtained.

[0041] In some embodiments of this disclosure, when the radar is a lidar, the residual is the distance between a point and a surface; when the radar is a millimeter-wave radar, the residual is the distance between points.

[0042] In some embodiments of this disclosure, the sensor includes a camera; the initial data acquisition module, when determining the second movement trajectory of the sensor and the relative pose between the second frames based on the second data acquired by the sensor, is specifically used for:

[0043] Acquire multiple consecutive frames of image data captured by the camera;

[0044] Feature points in multiple consecutive frames of image data are determined, and a second motion trajectory to the camera and relative pose between the second frames are determined based on the features.

[0045] In some embodiments of this disclosure, the initial data acquisition module, when determining feature points in multiple consecutive frames of image data, and determining the second motion trajectory to the camera and the relative pose between the second frames based on the features, is specifically used for:

[0046] Determine feature points in multiple consecutive frames of image data;

[0047] Based on preset parallax information, multiple keyframe image data are determined from multiple consecutive image data.

[0048] The feature points of adjacent keyframe image data are homogenized, and then the target keyframe image data is selected from multiple keyframe image data.

[0049] Based on the feature points in the target keyframe image data and the feature points in the last keyframe image data, the second motion trajectory to the camera and the relative pose between the second frames are determined.

[0050] In some embodiments of this disclosure, the initial data acquisition module, when determining the second motion trajectory to the camera and the relative pose between the second frame based on feature points in the target keyframe image data and feature points in the last keyframe image data, is specifically used for:

[0051] Triangulation is performed based on feature points in the target keyframe image data and feature points in the last keyframe image data;

[0052] The second movement trajectory of the camera and the relative pose between the second frame are determined based on the triangulated feature points.

[0053] In some embodiments of this disclosure, when the intermediate parameter determination module determines the intermediate calibration parameters of the sensor relative to the inertial measurement unit based on the second inter-frame relative pose and the initial inter-frame relative pose, it is specifically used to perform any one of the following:

[0054] When the sensor is a lidar, the angle calibration parameters and displacement calibration parameters in the initial calibration parameters are adjusted based on the second inter-frame relative pose and the initial inter-frame relative pose to obtain the intermediate calibration parameters.

[0055] When the sensor is a millimeter-wave radar, the angle calibration parameters in the initial calibration parameters are adjusted based on the second inter-frame relative pose and the initial inter-frame relative pose to obtain the intermediate calibration parameters.

[0056] In some embodiments of this disclosure, the intermediate parameter determination module, when determining intermediate calibration parameters of the sensor relative to the inertial measurement unit based on the second inter-frame relative pose and the initial inter-frame relative pose, is specifically used for:

[0057] When the sensor is a camera, the angle calibration parameters in the initial calibration parameters are adjusted based on the second inter-frame relative pose and the initial inter-frame relative pose to obtain the intermediate calibration parameters.

[0058] In some embodiments of this disclosure, the target parameter determination module, when determining the target calibration parameters of the sensor relative to the inertial measurement unit based on the first movement trajectory, the second movement trajectory, and intermediate calibration parameters, is specifically used for:

[0059] An alignment process is performed on the first and second moving trajectories. During the alignment process, intermediate calibration parameters are adjusted to obtain the target calibration parameters of the sensor relative to the inertial measurement unit.

[0060] According to a third aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method as described in the first aspect.

[0061] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions for causing a computer to perform the method described in the first aspect.

[0062] According to a fifth aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the method according to the first aspect.

[0063] According to a sixth aspect of this disclosure, an autonomous vehicle is provided, which includes the electronic equipment provided in a third aspect of this disclosure.

[0064] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description.

[0065] The beneficial effects of the technical solution provided in this disclosure are:

[0066] The sensor calibration method provided in this disclosure can calibrate sensors by analyzing and comparing the data collected by the inertial measurement unit and each sensor in the same scenario. The calibration scenario required for multiple sensors is the same, and the calibration process does not require complex devices such as calibration stations and corner reflectors, which can significantly save site and labor costs and also help improve calibration efficiency. Attached Figure Description

[0067] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0068] Figure 1 A schematic diagram of a vehicle to which the sensor calibration method provided in this disclosure can be applied is shown;

[0069] Figure 2 A schematic flowchart of a sensor calibration method provided in this disclosure is shown;

[0070] Figure 3 This illustration shows a flowchart of a process for obtaining a motion trajectory and inter-frame relative pose, as provided in this disclosure.

[0071] Figure 4 This illustration shows another flowchart for obtaining movement trajectory and inter-frame relative pose provided by this disclosure;

[0072] Figure 5 A schematic diagram of triangulation of feature points in target keyframe image data is shown;

[0073] Figure 6 A schematic diagram of a sensor calibration device provided in this disclosure is shown;

[0074] Figure 7 A schematic block diagram of an example electronic device that can be used to implement this disclosure is shown. Detailed Implementation

[0075] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0076] It should be understood that in the embodiments of this disclosure, the character " / " generally indicates that the preceding and following objects are in an "or" relationship. The terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated.

[0077] During the operation of autonomous vehicles, multiple sensors on the vehicle need to work together to perceive and locate the vehicle. To ensure the accuracy of the perception and localization results, sensor calibration is required in advance to determine the positional relationships between the sensors. However, in related technologies, the calibration process for multiple sensors requires different scenarios and tools, making the calibration process overly cumbersome and inefficient. For example, calibrating LiDAR requires the vehicle to be in motion outdoors, while calibrating a camera requires the vehicle to be stationary. Calibrating millimeter-wave radar requires a calibration station and the installation of corner reflectors and other devices. The different requirements of each sensor for the scenario and tools make the calibration process overly cumbersome and inefficient.

[0078] The execution subject of this method can be a terminal device, a computer, a server, or other devices with data processing capabilities. No limitation is placed on the execution subject of this method. In some embodiments, the sensor calibration method provided in this disclosure is executed by a terminal device (such as an onboard computer) on a vehicle.

[0079] Optionally, the terminal device can be a mobile phone, tablet computer, wearable device, in-vehicle device, augmented reality (AR) / virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), etc. The specific type of terminal device is not limited in the embodiments of this disclosure.

[0080] In some embodiments, the server can be a single server, or it can be a server cluster consisting of multiple servers. In some embodiments, the server cluster can also be a distributed cluster. This disclosure does not limit the specific implementation of the server.

[0081] The sensor calibration method provided in this disclosure is illustrated below.

[0082] Figure 1 A schematic diagram of a vehicle to which the sensor calibration method provided in this disclosure can be applied is shown, such as... Figure 1 As shown, the vehicle is equipped with an Inertial Measurement Unit (IMU) and at least one sensor. An IMU typically refers to a combined unit consisting of three accelerometers and three gyroscopes, mounted on mutually perpendicular measurement axes. Sensors may include cameras and radar, with radar further including lidar and millimeter-wave radar. It should be noted that... Figure 1 The positions of the inertial measurement unit and various sensors shown are schematic; their specific installation positions on the vehicle can be determined according to actual design requirements.

[0083] It is understood that the installation positions and attitudes of the inertial measurement unit (IMU) and various sensors on the vehicle are determined during the design phase. Therefore, the calibration parameters of the sensors relative to the IMU are also determined. For ease of understanding and description, this embodiment defines the calibration parameters of each sensor relative to the IMU determined during the design phase as initial calibration parameters. However, during actual installation, various errors (such as product size errors, vehicle size errors, and installation errors) may cause the actual installation positions and attitudes of the IMU and various sensors on the vehicle to differ from those determined during the design phase. This results in a difference between the actual calibration parameters of the sensors relative to the IMU and the aforementioned first calibration parameters. For ease of understanding and description, this embodiment defines the actual calibration parameters of the sensors relative to the IMU as target calibration parameters. Here, the target calibration parameters are the parameters that need to be determined by the sensor calibration method provided in this embodiment.

[0084] Figure 2 A schematic flowchart of a sensor calibration method provided in this disclosure is shown, as follows: Figure 2 As shown, the method mainly includes the following steps:

[0085] S210: Determine the first movement trajectory and the first inter-frame relative pose of the inertial measurement unit based on the first data collected by the inertial measurement unit, and determine the second movement trajectory and the second inter-frame relative pose of the sensor based on the second data collected by the sensor.

[0086] Here, the inertial measurement unit and sensors are set up in the same vehicle (e.g., Figure 1 On the vehicle shown, an inertial measurement unit (IMU) and sensors can collect data during vehicle movement. For ease of understanding and description, the data collected by the IMU is defined as the first data, and the data collected by the sensors is defined as the second data. Here, to collect more effective first and second data, the vehicle can be driven in a feature-rich environment, such as an area with many buildings, road signs, and trees. The IMU and sensors can then collect the first and second data respectively in this environment.

[0087] Based on the first data, a first movement trajectory and a first inter-frame relative pose of the inertial measurement unit (IMU) are determined. The first movement trajectory characterizes the trajectory of the IMU as it moves with the vehicle, and the first inter-frame relative pose characterizes the pose change of the IMU between two frames of data. Based on the second data, a second movement trajectory and a second inter-frame relative pose of the sensor are determined. The second movement trajectory characterizes the trajectory of the sensor as it moves with the vehicle, and the second inter-frame relative pose characterizes the pose change of the sensor between two frames of data.

[0088] Optionally, embodiments of this disclosure can perform B-spline optimization processing on the first data to obtain a continuous-time trajectory, thereby obtaining the first movement trajectory of the inertial measurement unit. Of course, embodiments of this disclosure can also use other pre-integration methods to process the first data to obtain the first movement trajectory. The specific process of determining the second movement trajectory of the sensor and the relative pose between the second frame will be further described in subsequent sections.

[0089] S220: Determine the initial calibration parameters of the sensor relative to the inertial measurement unit, and determine the initial inter-frame relative pose of the sensor based on the first inter-frame relative pose and the initial calibration parameters.

[0090] Here, the initial calibration parameters are the calibration parameters of the sensor relative to the inertial measurement unit determined during the design phase; that is, the initial calibration parameters are known data. The initial calibration parameters indicate the conversion relationship between the sensor data and the inertial measurement unit data. Based on the conversion relationship indicated by the initial calibration parameters, the initial inter-frame relative pose of the sensor is obtained by converting the first inter-frame relative pose.

[0091] S230: Based on the second inter-frame relative pose and the initial inter-frame relative pose, determine the intermediate calibration parameters of the sensor relative to the inertial measurement unit.

[0092] As mentioned earlier, sensors can include cameras and radar, with radar further including lidar and millimeter-wave radar. Calibration parameters can include angle calibration parameters and displacement calibration parameters. Angle calibration parameters represent the angular transformation between the sensor and the inertial measurement unit (IMU), while displacement calibration parameters represent the positional transformation. Because the second data acquired by millimeter-wave radar is relatively sparse, six-DOF extrinsic parameter optimization is difficult and robustness is generally poor. Therefore, in S230, only the angle calibration parameters of the millimeter-wave radar can be adjusted. The second data acquired by lidar is relatively dense, allowing for six-DOF optimization.

[0093] Specifically, in S230, when the sensor is a lidar, the angle calibration parameters and displacement calibration parameters in the initial calibration parameters can be adjusted based on the second inter-frame relative pose and the initial inter-frame relative pose to obtain intermediate calibration parameters; when the sensor is a millimeter-wave radar, the angle calibration parameters in the initial calibration parameters are adjusted based on the second inter-frame relative pose and the initial inter-frame relative pose to obtain intermediate calibration parameters; when the sensor is a camera, the angle calibration parameters in the initial calibration parameters are adjusted based on the second inter-frame relative pose and the initial inter-frame relative pose to obtain intermediate calibration parameters.

[0094] Understandably, ideally, the relative pose between the second and initial frames should be identical. However, due to various errors during actual installation, differences arise between the relative poses. Therefore, the initial calibration parameters of the sensor relative to the inertial measurement unit (IMU) can be adjusted based on these differences to obtain intermediate calibration parameters.

[0095] In S230, the first transformation relationship between the initial inter-frame relative pose and the second inter-frame relative pose can be calculated. This first transformation relationship is used to convert the initial inter-frame relative pose into the second inter-frame relative pose. Then, based on this first transformation relationship, the initial calibration parameters are transformed to obtain the intermediate calibration parameters.

[0096] Specifically, multiple initial inter-frame relative poses and multiple second inter-frame relative poses can be acquired, with a one-to-one correspondence between the initial and second inter-frame relative poses. Each initial inter-frame relative pose and each second inter-frame relative pose are treated as a data set. By fitting the transformation relationship between the initial and second inter-frame relative poses from multiple data sets, the final first transformation relationship between the initial and second inter-frame relative poses is obtained. Here, calibration parameters can include angle calibration parameters and displacement calibration parameters. The angle calibration parameters represent the calibration relationship between the sensor and the inertial measurement unit (IMU) in the angular dimension, and the displacement calibration parameters represent the calibration relationship between the sensor and the IMU in the position dimension. When adjusting the initial calibration parameters, the angle calibration parameters in the initial calibration parameters can be transformed based on the aforementioned first transformation relationship to obtain a new angle calibration parameter, thereby completing the conversion of the initial calibration parameters into intermediate calibration parameters. More specifically, the first transformation relationship includes an angle adjustment function that adjusts the angle parameters. By adjusting the angle parameters in the initial calibration parameters using this angle adjustment function, a new angle calibration parameter can be obtained. Here, after replacing the original angle calibration parameter in the initial calibration parameters with the aforementioned new angle calibration parameter, the intermediate calibration parameter can be obtained.

[0097] S240: Based on the first movement trajectory, the second movement trajectory, and intermediate calibration parameters, determine the target calibration parameters of the sensor relative to the inertial measurement unit.

[0098] In S240, after obtaining the intermediate calibration parameters of the sensor relative to the inertial measurement unit, the intermediate calibration parameters can be adjusted based on the difference between the first moving trajectory of the inertial measurement unit and the second moving trajectory of the sensor, thereby obtaining the target calibration parameters of the sensor relative to the inertial measurement unit.

[0099] Optionally, in some embodiments, an alignment process can be performed on the first and second moving trajectories in S240, during which intermediate calibration parameters are adjusted to obtain target calibration parameters of the sensor relative to the inertial measurement unit.

[0100] In S240, a second conversion relationship between the second movement trajectory and the first movement trajectory can be calculated. This second conversion relationship is used to convert the second movement trajectory into the first movement trajectory. Then, this second conversion relationship is used as a calibration parameter in the intermediate calibration parameters, thereby converting the intermediate calibration parameters into target calibration parameters.

[0101] Specifically, the first movement trajectory contains first trajectory points at multiple times, and the second movement trajectory contains second trajectory points at multiple times, with each first trajectory point corresponding to a second trajectory point at the same time. It can be understood that the aforementioned second transformation relationship is used to convert each second trajectory point into a first trajectory point at the same time. Here, the first and second trajectory points at the same time can be treated as a set of data. By fitting the transformation relationship between the first and second trajectory points in multiple sets of data, a second transformation relationship between the second and first movement trajectories is obtained. This second transformation relationship is then used as a calibration parameter in the intermediate calibration parameters, thereby converting the intermediate calibration parameters into target calibration parameters. More specifically, as mentioned above, calibration parameters can include angle calibration parameters and displacement calibration parameters. Here, the second transformation relationship can be used as a new displacement calibration parameter, replacing the displacement calibration parameters in the intermediate calibration parameters, thereby converting the intermediate calibration parameters into target calibration parameters.

[0102] The sensor calibration method provided in this disclosure can calibrate sensors by analyzing and comparing the data collected by the inertial measurement unit and each sensor in the same scenario. The calibration scenario required for multiple sensors is the same, and the calibration process does not require complex devices such as calibration stations and corner reflectors, which can significantly save site and labor costs and also help improve calibration efficiency.

[0103] This embodiment of the invention adjusts the sensor calibration parameters in two stages. In the first stage, the initial inter-frame relative pose of the sensor, determined by the first inter-frame relative pose of the inertial measurement unit and the initial calibration parameters, is compared with the second inter-frame relative pose determined based on second data acquired by the sensor itself. This comparison adjusts the initial calibration parameters to obtain intermediate calibration parameters. In the second stage, the first movement trajectory of the inertial measurement unit is compared with the second movement trajectory of the sensor. This comparison adjusts the intermediate initial calibration parameters to obtain the final target calibration parameters. These two stages utilize data from different dimensions of the inertial measurement unit to ensure the accuracy of the final target calibration parameters of the sensor relative to the inertial measurement unit.

[0104] Optionally, in some embodiments, the sensor includes radar. When the sensor is radar, it can be employed... Figure 3 The process shown determines the sensor's second movement trajectory and the relative pose between the second and second frames. Figure 3 This illustration shows a flowchart of a process for obtaining a motion trajectory and inter-frame relative pose, as provided in this disclosure. Figure 3 As shown, the process mainly includes the following steps:

[0105] S310: Acquires multiple consecutive frames of radar data collected by the radar.

[0106] S320: Determines radar mileage information through multiple consecutive frames of radar data, and determines the radar's second movement trajectory based on the radar mileage information.

[0107] S330: Substitute multiple consecutive frames of radar data into a preset point registration algorithm to optimize the preset residuals, and then obtain the relative pose of the radar between the second frame.

[0108] In the S330, the point registration algorithm that can be used is the Iterative Closest Point (ICP) algorithm. When the radar is a lidar, the residual used in the algorithm is the distance between a point and an area; when the radar is a millimeter-wave radar, the residual used in the algorithm is the distance between points. Appropriate parameters are selected based on the characteristics of each radar type as the residual in its algorithm optimization process to ensure the accuracy of the final calculated inter-frame relative pose.

[0109] The embodiments disclosed herein can determine the radar's movement trajectory and inter-frame relative pose based on radar data collected by the radar itself, eliminating the need for various complex tools required in the traditional calibration process, which helps to improve calibration efficiency and reduce calibration costs.

[0110] Optionally, in some embodiments, the sensor includes a camera. When the sensor is a camera, it can be employed... Figure 4 The process shown determines the sensor's second movement trajectory and the relative pose between the second and second frames. Figure 4 This illustration shows another flowchart of obtaining the motion trajectory and inter-frame relative pose provided by this disclosure, such as... Figure 4 As shown, the process mainly includes the following steps:

[0111] S410: Acquires multiple consecutive frames of image data captured by the camera.

[0112] S420: Determine feature points in multi-frame continuous image data, and determine the second motion trajectory to the camera and the relative pose between the second frames based on the features.

[0113] The embodiments disclosed herein can determine the camera's movement trajectory and inter-frame relative pose based on the image data acquired by the camera itself, eliminating the need for various complex tools required in the traditional calibration process, which helps to improve calibration efficiency and reduce calibration costs.

[0114] Optionally, in some embodiments, S420 may specifically include: determining feature points in multiple consecutive image data frames; determining multiple keyframe image data frames from the multiple consecutive image data frames based on preset parallax information; homogenizing the feature points of adjacent keyframe image data frames, and then selecting target keyframe image data frames from the multiple keyframe image data frames; determining the second motion trajectory of the camera and the relative pose between the second and third frames based on the feature points in the target keyframe image data frames and the feature points in the last keyframe image data frame. Here, keyframe image data frames that meet the pose change requirements can be selected as target keyframe image data frames using a sliding window. This embodiment of the present disclosure can determine target keyframe image data frames from multiple consecutive image data frames, and then determine the second motion trajectory of the camera and the relative pose between the second and third frames frames based on the feature points in the target keyframe image data frames. This process accurately filters out the most effective data frames from the multiple consecutive image data frames, and the second motion trajectory and relative pose between the second and third frames frames determined based on the most effective data frames are more accurate.

[0115] In some embodiments of this disclosure, determining the second motion trajectory to the camera and the relative pose between the second and third frames based on feature points in the target keyframe image data and the last keyframe image data includes: triangulation based on feature points in the target keyframe image data and the last keyframe image data; and determining the second motion trajectory to the camera and the relative pose between the second and third frames based on the triangulated feature points. Here, the second motion trajectory to the camera and the relative pose between the second and third frames can be obtained based on the triangulated feature points using the PNP algorithm and fullBA optimization.

[0116] Specifically, both the target keyframe image data and the last keyframe image data contain three feature points. Based on the three feature points in the target keyframe image data, the PNP algorithm is used to calculate the camera pose at the moment the target keyframe image data is acquired; based on the three feature points in the last keyframe image data, the PNP algorithm is used to calculate the camera pose at the moment the last keyframe image data is acquired; based on the camera poses at the above two moments, the relative pose between the second frames is determined.

[0117] Taking the calculation of the camera's pose at the moment of acquiring keyframe image data of the target as an example, Figure 5 This diagram illustrates the triangulation of feature points in the target keyframe image data. Figure 5 In the diagram, O represents the camera position, and points a, b, and c are three feature points in the target keyframe image data. Points a, b, and c are points in the camera coordinate system, while points A, B, and C are points in the world coordinate system corresponding to points a, b, and c, respectively. Figure 5In the diagram, triangles oab and oAB are similar, triangles oac and oAC are similar, and triangles obc and oBC are similar. Based on this similarity relationship, oA, oB, and oC can be solved. Then, the camera pose at the moment of acquiring the target keyframe image data is obtained by iterating the nearest point. The same method is used to obtain the camera pose at the moment of acquiring the target keyframe image data. Based on the camera pose at the above two moments, the relative pose between the second frame is determined. Of course, the camera pose obtained by the above process can be used as a preliminary pose estimate. After obtaining the preliminary pose estimate, it can be used as the initial value. Using full bundle adjustment (BA) with the estimate as the initial value, a least squares optimization problem is constructed to adjust the pose estimate to obtain the final camera pose.

[0118] Understandably, in the process of calculating the relative pose between the second frames, the relative position information of the camera relative to the same point in the real world at different times is calculated, and the second movement trajectory of the camera can be estimated based on the relative position information at these different times.

[0119] Based on the same principle as the sensor calibration method described above, this disclosure provides a sensor calibration device. Figure 6 A schematic diagram of the first sensor calibration device provided in this disclosure is shown. Figure 6 As shown, the sensor calibration device 600 includes an initial data acquisition module 610, an initial pose determination module 620, an intermediate parameter determination module 630, and a target parameter determination module 640.

[0120] The initial data acquisition module 610 is used to determine the first movement trajectory and the first inter-frame relative pose of the inertial measurement unit based on the first data acquired by the inertial measurement unit, and to determine the second movement trajectory and the second inter-frame relative pose of the sensor based on the second data acquired by the sensor.

[0121] The initial pose determination module 620 is used to determine the initial calibration parameters of the sensor relative to the inertial measurement unit, and to determine the initial inter-frame relative pose of the sensor based on the first inter-frame relative pose and the initial calibration parameters.

[0122] The intermediate parameter determination module 630 is used to determine the intermediate calibration parameters of the sensor relative to the inertial measurement unit based on the second inter-frame relative pose and the initial inter-frame relative pose.

[0123] The target parameter determination module 640 is used to determine the target calibration parameters of the sensor relative to the inertial measurement unit based on the first movement trajectory, the second movement trajectory, and intermediate calibration parameters.

[0124] The sensor calibration device provided in this disclosure can calibrate sensors by analyzing and comparing the data collected by the inertial measurement unit and each sensor in the same scenario. The calibration scenario required for multiple sensors is the same, and the calibration process does not require complex devices such as calibration stations and corner reflectors, which can significantly save site and labor costs and also help improve calibration efficiency.

[0125] In some embodiments of this disclosure, the sensor includes a radar; the initial data acquisition module 610, when determining the second movement trajectory of the sensor and the relative pose between the second frames based on the second data acquired by the sensor, is specifically used for:

[0126] Acquire multiple consecutive frames of radar data collected by the radar;

[0127] Radar mileage information is determined by using multiple consecutive frames of radar data, and the second movement trajectory of the radar is determined based on the radar mileage information.

[0128] After substituting multiple consecutive frames of radar data into a preset point registration algorithm to optimize the preset residuals, the relative pose between the second frame of the radar is obtained.

[0129] In some embodiments of this disclosure, when the radar is a lidar, the residual is the distance between a point and a surface; when the radar is a millimeter-wave radar, the residual is the distance between points.

[0130] In some embodiments of this disclosure, the sensor includes a camera; the initial data acquisition module 610, when determining the second movement trajectory of the sensor and the relative pose between the second frames based on the second data acquired by the sensor, is specifically used for:

[0131] Acquire multiple consecutive frames of image data captured by the camera;

[0132] Feature points in multiple consecutive frames of image data are determined, and a second motion trajectory to the camera and relative pose between the second frames are determined based on the features.

[0133] In some embodiments of this disclosure, the initial data acquisition module 610, when determining feature points in multiple consecutive frames of image data and determining the second motion trajectory to the camera and the relative pose between the second frames based on the features, is specifically used for:

[0134] Determine feature points in multiple consecutive frames of image data;

[0135] Based on preset parallax information, multiple keyframe image data are determined from multiple consecutive image data.

[0136] The feature points of adjacent keyframe image data are homogenized, and then the target keyframe image data is selected from multiple keyframe image data.

[0137] Based on the feature points in the target keyframe image data and the feature points in the last keyframe image data, the second motion trajectory to the camera and the relative pose between the second frames are determined.

[0138] In some embodiments of this disclosure, the initial data acquisition module 610, when determining the second movement trajectory to the camera and the relative pose between the second and third frames based on feature points in the target keyframe image data and feature points in the last keyframe image data, is specifically used for:

[0139] Triangulation is performed based on feature points in the target keyframe image data and feature points in the last keyframe image data;

[0140] The second movement trajectory of the camera and the relative pose between the second frame are determined based on the triangulated feature points.

[0141] In some embodiments of this disclosure, when the intermediate parameter determination module 630 determines the intermediate calibration parameters of the sensor relative to the inertial measurement unit based on the second inter-frame relative pose and the initial inter-frame relative pose, it is specifically used to perform any one of the following:

[0142] When the sensor is a lidar, the angle calibration parameters and displacement calibration parameters in the initial calibration parameters are adjusted based on the second inter-frame relative pose and the initial inter-frame relative pose to obtain the intermediate calibration parameters.

[0143] When the sensor is a millimeter-wave radar, the angle calibration parameters in the initial calibration parameters are adjusted based on the second inter-frame relative pose and the initial inter-frame relative pose to obtain the intermediate calibration parameters.

[0144] In some embodiments of this disclosure, the intermediate parameter determination module 630, when determining intermediate calibration parameters of the sensor relative to the inertial measurement unit based on the second inter-frame relative pose and the initial inter-frame relative pose, is specifically used for:

[0145] When the sensor is a camera, the angle calibration parameters in the initial calibration parameters are adjusted based on the second inter-frame relative pose and the initial inter-frame relative pose to obtain the intermediate calibration parameters.

[0146] In some embodiments of this disclosure, the target parameter determination module 640, when determining the target calibration parameters of the sensor relative to the inertial measurement unit based on the first movement trajectory, the second movement trajectory, and intermediate calibration parameters, is specifically used for:

[0147] An alignment process is performed on the first and second moving trajectories. During the alignment process, intermediate calibration parameters are adjusted to obtain the target calibration parameters of the sensor relative to the inertial measurement unit.

[0148] It is understood that the modules of the sensor calibration device in this embodiment have the function of implementing the corresponding steps of the sensor calibration method described above. This function can be implemented in hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the functions described above. These modules can be software and / or hardware; each module can be implemented individually or multiple modules can be integrated. For a detailed description of the functions of each module of the sensor calibration device, please refer to the corresponding description of the sensor calibration method described above; further details will not be repeated here.

[0149] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0150] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, a computer program product, and an autonomous vehicle.

[0151] In an exemplary embodiment, the electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method described in the above embodiments. The electronic device may be the computer or server described above.

[0152] In an exemplary embodiment, the readable storage medium may be a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the method described in the above embodiments.

[0153] In an exemplary embodiment, the computer program product includes a computer program that, when executed by a processor, implements the method described in the above embodiments.

[0154] In an exemplary embodiment, the autonomous vehicle includes the aforementioned electronic device, which may specifically be an onboard computer.

[0155] Figure 7A schematic block diagram of an example electronic device 700 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0156] like Figure 7 As shown, the electronic device 700 includes a computing unit 701, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. The RAM 703 may also store various programs and data required for the operation of the device 700. The computing unit 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0157] Multiple components in electronic device 700 are connected to I / O interface 705, including: input unit 706, such as keyboard, mouse, etc.; output unit 707, such as various types of displays, speakers, etc.; storage unit 708, such as disk, optical disk, etc.; and communication unit 709, such as network card, modem, wireless transceiver, etc. Communication unit 709 allows electronic device 700 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0158] The computing unit 701 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above, such as sensor calibration methods. For example, in some embodiments, the sensor calibration method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 700 via ROM 702 and / or communication unit 709. When the computer program is loaded into RAM 703 and executed by the computing unit 701, one or more steps of the sensor calibration method described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform the sensor calibration method by any other suitable means (e.g., by means of firmware).

[0159] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0160] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0161] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0162] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0163] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0164] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0165] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0166] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A sensor calibration method, the method comprising: The first movement trajectory and the first inter-frame relative pose of the inertial measurement unit are determined based on the first data collected by the inertial measurement unit, and the second movement trajectory and the second inter-frame relative pose of the sensor are determined based on the second data collected by the sensor. Determine the initial calibration parameters of the sensor relative to the inertial measurement unit, and determine the initial inter-frame relative pose of the sensor based on the first inter-frame relative pose and the initial calibration parameters; Based on the second inter-frame relative pose and the initial inter-frame relative pose, intermediate calibration parameters of the sensor relative to the inertial measurement unit are determined; Based on the first movement trajectory, the second movement trajectory, and the intermediate calibration parameters, the target calibration parameters of the sensor relative to the inertial measurement unit are determined.

2. The method according to claim 1, wherein, The sensor includes radar; The determination of the second movement trajectory and the relative pose between the second and second frames based on the second data collected by the sensor includes: Acquire multiple consecutive frames of radar data collected by the radar; Radar mileage information is determined by the multiple consecutive frames of radar data, and the second movement trajectory of the radar is determined based on the radar mileage information. After substituting the multi-frame continuous radar data into a preset point registration algorithm to optimize the preset residuals, the relative pose between the second frame of the radar is obtained.

3. The method according to claim 2, wherein, When the radar is a lidar, the residual is the distance between a point and a surface; When the radar is a millimeter-wave radar, the residual is the distance between points.

4. The method according to claim 1, wherein, The sensor includes a camera; The determination of the second movement trajectory and the relative pose between the second and second frames based on the second data collected by the sensor includes: Acquire multiple consecutive frames of image data captured by the camera; Feature points in the multi-frame continuous image data are determined, and a second movement trajectory to the camera and relative pose between the second frame are determined based on the features.

5. The method according to claim 4, wherein, The step of determining feature points in the multi-frame continuous image data, and determining the second motion trajectory to the camera and the relative pose between the second frames based on the features, includes: Determine the feature points in the multi-frame continuous image data; Multiple keyframe image data are determined from the multiple consecutive image data based on preset parallax information; The feature points of adjacent keyframe image data are homogenized, and then the target keyframe image data is selected from the plurality of keyframe image data. Based on the feature points in the target keyframe image data and the feature points in the last keyframe image data, the second movement trajectory to the camera and the relative pose between the second frames are determined.

6. The method according to claim 5, wherein, The step of determining the second motion trajectory to the camera and the relative pose between the second frame based on feature points in the target keyframe image data and feature points in the last keyframe image data includes: The triangulation is performed based on the feature points in the target keyframe image data and the feature points in the last keyframe image data; The second movement trajectory to the camera and the relative pose between the second frame are determined based on the triangulated feature points.

7. The method according to any one of claims 1-6, wherein, The determination of intermediate calibration parameters of the sensor relative to the inertial measurement unit based on the second inter-frame relative pose and the initial inter-frame relative pose includes any one of the following: When the sensor is a lidar, the angle calibration parameters and displacement calibration parameters in the initial calibration parameters are adjusted based on the second inter-frame relative pose and the initial inter-frame relative pose to obtain the intermediate calibration parameters. When the sensor is a millimeter-wave radar, the angle calibration parameters in the initial calibration parameters are adjusted based on the second inter-frame relative pose and the initial inter-frame relative pose to obtain the intermediate calibration parameters.

8. The method according to any one of claims 1-6, wherein, The step of determining the intermediate calibration parameters of the sensor relative to the inertial measurement unit based on the second inter-frame relative pose and the initial inter-frame relative pose includes: When the sensor is a camera, the angle calibration parameters in the initial calibration parameters are adjusted based on the second inter-frame relative pose and the initial inter-frame relative pose to obtain the intermediate calibration parameters.

9. The method according to any one of claims 1-6, wherein, The step of determining the target calibration parameters of the sensor relative to the inertial measurement unit based on the first movement trajectory, the second movement trajectory, and the intermediate calibration parameters includes: An alignment process is performed on the first and second movement trajectories, during which the intermediate calibration parameters are adjusted to obtain the target calibration parameters of the sensor relative to the inertial measurement unit.

10. A sensor calibration device, the device comprising: The initial data acquisition module is used to determine the first movement trajectory and the first inter-frame relative pose of the inertial measurement unit based on the first data collected by the inertial measurement unit, and to determine the second movement trajectory and the second inter-frame relative pose of the sensor based on the second data collected by the sensor. An initial pose determination module is used to determine the initial calibration parameters of the sensor relative to the inertial measurement unit, and to determine the initial inter-frame relative pose of the sensor based on the first inter-frame relative pose and the initial calibration parameters. The intermediate parameter determination module is used to determine the intermediate calibration parameters of the sensor relative to the inertial measurement unit based on the second inter-frame relative pose and the initial inter-frame relative pose. The target parameter determination module is used to determine the target calibration parameters of the sensor relative to the inertial measurement unit based on the first movement trajectory, the second movement trajectory, and the intermediate calibration parameters.

11. The apparatus according to claim 10, wherein, The sensor includes radar; The initial data acquisition module, when used to determine the second movement trajectory of the sensor and the relative pose between the second frame based on the second data acquired by the sensor, is specifically used for: Acquire multiple consecutive frames of radar data collected by the radar; Radar mileage information is determined by the multiple consecutive frames of radar data, and the second movement trajectory of the radar is determined based on the radar mileage information. After substituting the multi-frame continuous radar data into a preset point registration algorithm to optimize the preset residuals, the relative pose between the second frame of the radar is obtained.

12. The apparatus according to claim 11, wherein, When the radar is a lidar, the residual is the distance between a point and a surface; When the radar is a millimeter-wave radar, the residual is the distance between points.

13. The apparatus according to claim 10, wherein, The sensor includes a camera; The initial data acquisition module, when used to determine the second movement trajectory of the sensor and the relative pose between the second frame based on the second data acquired by the sensor, is specifically used for: Acquire multiple consecutive frames of image data captured by the camera; Feature points in the multi-frame continuous image data are determined, and a second movement trajectory to the camera and relative pose between the second frame are determined based on the features.

14. The apparatus according to claim 13, wherein, When the initial data acquisition module determines feature points in the multi-frame continuous image data and determines the second movement trajectory to the camera and the relative pose between the second frames based on the features, it is specifically used for: Determine the feature points in the multi-frame continuous image data; Multiple keyframe image data are determined from the multiple consecutive image data based on preset parallax information; The feature points of adjacent keyframe image data are homogenized, and then the target keyframe image data is selected from the plurality of keyframe image data. Based on the feature points in the target keyframe image data and the feature points in the last keyframe image data, the second movement trajectory to the camera and the relative pose between the second frames are determined.

15. The apparatus according to claim 14, wherein, When the initial data acquisition module determines the second movement trajectory of the camera and the relative pose between the second frame based on feature points in the target keyframe image data and feature points in the last keyframe image data, it is specifically used for: The triangulation is performed based on the feature points in the target keyframe image data and the feature points in the last keyframe image data; The second movement trajectory to the camera and the relative pose between the second frame are determined based on the triangulated feature points.

16. The apparatus according to any one of claims 10-15, wherein, When the intermediate parameter determination module determines the intermediate calibration parameters of the sensor relative to the inertial measurement unit based on the second inter-frame relative pose and the initial inter-frame relative pose, it is specifically used to perform any one of the following: When the sensor is a lidar, the angle calibration parameters and displacement calibration parameters in the initial calibration parameters are adjusted based on the second inter-frame relative pose and the initial inter-frame relative pose to obtain the intermediate calibration parameters. When the sensor is a millimeter-wave radar, the angle calibration parameters in the initial calibration parameters are adjusted based on the second inter-frame relative pose and the initial inter-frame relative pose to obtain the intermediate calibration parameters.

17. An electronic device comprising: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-9.

18. A non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the method according to any one of claims 1-9.

19. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-9.

20. An autonomous vehicle comprising the electronic device of claim 17.

Images