Vehicle sensor calibration method and device and electronic equipment

Abstract

The invention provides a vehicle sensor calibration method and device and electronic equipment. The method comprises the steps that first data and second data are acquired; the external parameters of the to-be-calibrated sensor are calibrated according to the first data and the second data, the data of the inertial sensor and the data of the to-be-calibrated sensor are collected at the same time under the preset condition, the vehicle-mounted sensor can be calibrated, the preset condition can be set according to needs, and compared with the prior art, the complex scene is not a necessary option, and the calibration efficiency is improved. Therefore, the appropriate preset condition can be matched according to the calibration accuracy requirement, in addition, a large number of complex scene supports are not needed on the premise that the accuracy requirement is met, the requirements for the scene and the driving mode in the data acquisition process are low, the calibration process is easy to operate, the calibration accuracy is not affected, and the problem that the current calibration process is complex is solved.

Description

Technical Field

[0001] The present application relates to the field of sensors, and in particular to a sensor calibration method, device and electronic equipment for a vehicle. Background Art

[0002] The online calibration of the external parameters of vehicle-mounted sensors after leaving the factory is usually more complicated. It generally requires the vehicle to drive straight and fast on an outdoor road. At the same time, there may be additional requirements such as it must be a highway, clear and complete lane lines must be visible, driving must be parallel to the lane lines, and there must be as few vehicles around as possible. These requirements are generally difficult to meet in urban traffic. In order to record enough data to complete the calibration, the customer may be required to drive the vehicle to a specific area far away and drive for a long time before returning, so there are many supplements. Summary of the invention

[0003] In view of the problem that the calibration of external parameters of vehicle-mounted sensors in the prior art requires a strict calibration environment to accurately calibrate the external parameters of the vehicle-mounted sensors, the present application provides a vehicle sensor calibration method, device and electronic equipment.

[0004] A first aspect of the present application provides a sensor calibration method for a vehicle, comprising:

[0005] Acquire first data and second data, wherein the first data is collected by an inertial sensor, the second data is collected by a sensor to be calibrated, and the first data and the second data are collected under preset conditions;

[0006] The external parameters of the sensor to be calibrated are calibrated according to the first data and the second data.

[0007] In an optional implementation, the preset condition includes the driver driving the vehicle to turn an angle in an area where the flatness is lower than a set threshold and driving in a straight line for at least one section of the road.

[0008] In an optional implementation manner, before calibrating the external parameters of the sensor to be calibrated according to the first data and the second data, the sensor calibration method further includes:

[0009] Determine the section of road with the highest track straightness;

[0010] The second data corresponding to a section of the track with the highest straightness is used to calibrate the external parameters of the sensor to be calibrated.

[0011] In an optional implementation manner, determining a section of road with the highest trajectory straightness includes:

[0012] With the angular velocity corresponding to each road section as the Y-axis and the track duration as the X-axis, a coordinate graph of the angular velocity versus track duration is drawn;

[0013] The area of ​​the closed region formed by the angular velocity curve on the X-axis is calculated, and the section with the smallest area is determined as the section with the highest trajectory straightness.

[0014] In an optional implementation, the sensor to be calibrated is a laser radar, the second data is point cloud data, and the step of determining a section of the road with the highest trajectory straightness includes:

[0015] Determine the rotation angle corresponding to each road section according to the point cloud data of the laser radar at the starting point and the end point of each road section;

[0016] The section with the smallest rotation angle is regarded as the section with the highest trajectory straightness.

[0017] In an optional implementation, the sensor to be calibrated is a camera sensor, the second data is image data, and the step of determining a section of the road with the highest trajectory straightness includes:

[0018] Determine the camera posture at the initial position and the end position based on the image data continuously captured during the driving process of each road section;

[0019] Determine the rotation angle corresponding to each road section according to the camera posture at the initial position and the end position of each road section;

[0020] The section with the smallest rotation angle is regarded as the section with the highest trajectory straightness.

[0021] In an optional implementation, the sensor to be calibrated includes a camera sensor and a laser radar, and the step of determining a section of the road with the highest trajectory straightness includes:

[0022] With the angular velocity corresponding to each road section as the Y-axis and the track duration as the X-axis, a coordinate graph of the angular velocity versus track duration is drawn;

[0023] Calculating the area of ​​a closed region formed between the x-axis and the real trajectory corresponding to each road segment;

[0024] Determine a first rotation angle corresponding to each road section according to the point cloud data of the laser radar at the starting point and the end point of each road section;

[0025] Determine the camera posture at the initial position and the end position based on the image data continuously captured during the driving process of each road section;

[0026] Determine a second rotation angle corresponding to each road section according to the camera posture at the initial position and the end position of each road section;

[0027] The section of the track with the highest straightness is determined according to the area, the first rotation angle, and the second rotation angle in combination with preset weights corresponding to the area, the first rotation angle, and the second rotation angle.

[0028] In an optional implementation manner, calibrating the external parameters of the sensor to be calibrated according to the first data and the second data includes:

[0029] Determine a section of the track with the highest straightness according to the first data; obtain second data corresponding to the section of the track with the highest straightness; and calibrate external parameters of the sensor to be calibrated according to the second data.

[0030] It can be understood that using the second data to determine the external parameters of the sensor to be calibrated includes performing relative posture calculations on the start and end points of each trajectory, and then converting the calculation results into external parameter calibration results through the principles of linear algebra.

[0031] A second aspect of the present application provides a sensor calibration device for a vehicle, comprising:

[0032] An acquisition module, which acquires first data and second data when a driver drives the vehicle to turn an angle and travels straight for at least one section of a road in an area where the flatness is lower than a set threshold, wherein the first data is acquired by an inertial sensor and the second data is acquired by a sensor to be calibrated;

[0033] The calibration module calibrates the external parameters of the sensor to be calibrated according to the first data and the second data.

[0034] A third aspect of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the sensor calibration method for the vehicle as described above when executing the program.

[0035] A fourth aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the sensor calibration method for the vehicle as described above.

[0036] It can be seen from the above technical scheme that the present application provides a vehicle sensor calibration method, device and electronic equipment. The present application simultaneously collects the data of the inertial sensor and the sensor to be calibrated under preset conditions, so as to calibrate the vehicle-mounted sensor. The preset conditions can be set as needed. Compared with the prior art, complex scenes are not a must. Therefore, appropriate preset conditions can be matched according to the accuracy requirements of the calibration. In addition, on the premise that the accuracy requirements are met, a large number of complex scenes are not required to support it. The data collection process has low requirements on the scene and driving method. The calibration process is simple to operate and does not affect the calibration accuracy, thereby solving the problem of the complexity of the current calibration process.

[0037] In a further preferred embodiment, the present application sets the preset condition as the driver driving the vehicle to turn an angle and drive straight for at least one section of the road in an area where the flatness is lower than a set threshold. Based on the inventor's discovery that flatness is one of the factors affecting the calibration results, after overcoming the influence of flatness, since the most important influencing factor has been overcome, it is no longer necessary to correct the calibration error through a large amount of data formed by complex scenes, and it is sufficient to directly configure the straight-line driving of a short distance and the rotation of an angle in the present application. The scene complexity of the present application is much lower than the complex scene of the prior art, and the calibration accuracy is not reduced, thereby providing a simpler and more efficient vehicle sensor calibration solution. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0039] Figure 1 It is a flow chart of a sensor calibration method for a vehicle in an embodiment of the present application.

[0040] Figure 2 In the embodiment of this application Figure 1 Specific flow chart of step S1 in FIG.

[0041] Figure 3 It is one of the specific flow charts of step S01 in the embodiment of the present application.

[0042] Figure 4 In the embodiment of this application Figure 3 Schematic diagram of the scene.

[0043] Figure 5 This is the second specific flow chart of step S01 in the embodiment of the present application.

[0044] Figure 6 In the embodiment of this application Figure 5 Schematic diagram of the scene.

[0045] Figure 7 This is the third specific flow chart of step S01 in the embodiment of the present application.

[0046] Figure 8 In the embodiment of this application Figure 7 Schematic diagram of the scene.

[0047] Fig. 9 This is the fourth specific flow chart of step S01 in the embodiment of the present application.

[0048] Fig.10 It is a structural schematic diagram of a sensor calibration device for a vehicle in an embodiment of the present application.

[0049] Fig.11 It is a schematic diagram of the structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0051] It should be noted that the vehicle sensor calibration method, system, electronic device and computer-readable storage medium disclosed in the present application can be used in the field of vehicle sensor calibration technology, and can also be used in any field except the field of vehicle sensor calibration technology. The application field of the vehicle sensor calibration method, system, electronic device and computer-readable storage medium disclosed in the present application is not limited.

[0052] Extrinsic parameters refer to the attitude and position information of vehicle-mounted sensors relative to the vehicle coordinate system. For vehicle-mounted sensors such as cameras, radars, and lidars, their accurate external parameters are the basis for perception and positioning tasks. Accurate factory calibration is generally performed before the vehicle leaves the factory, but due to external factors such as bumps and collisions after leaving the factory, the external parameters of the sensor change, which requires online calibration to obtain accurate external parameters. Online calibration refers to the derivation of the external parameters of the sensor through specific operations and data collection during the actual vehicle driving process. However, in order to ensure the accuracy of the calibration, some specific conditions usually need to be met. These conditions include: Straight and fast driving on outdoor roads: During the calibration process, the vehicle needs to drive at a higher speed on the outdoor road to ensure that the sensor obtains enough data. Fast driving can provide more scene changes and perspective information, which helps to improve the accuracy of calibration. Highways: Vehicles on highways drive relatively stably, lane lines are clear, and there are relatively few vehicles, which can provide a more ideal calibration environment. Clear and complete lane lines: Lane lines are an important reference for calibration. Clear and complete lane lines help to accurately determine the position and posture of the sensor relative to the vehicle coordinate system. Driving parallel to the lane lines: In order to obtain accurate external parameters, the vehicle needs to drive parallel to the lane lines, which can better reflect the relative relationship between the sensor and the vehicle coordinate system. There are as few surrounding vehicles as possible: Too many surrounding vehicles will interfere with the calibration process, so try to choose an area with fewer vehicles for calibration to reduce the impact of interference. Due to the complexity of urban traffic and the diversity of road conditions, it is often difficult to meet the above requirements in urban traffic. Therefore, in order to record enough data to complete the calibration, it may be necessary to drive the vehicle to a specific area far away from the city and drive there for a long time to obtain enough calibration data. The reason for this is that there are many interference factors in the urban traffic environment, such as traffic lights, pedestrians, other vehicles, etc., which may interfere with the data collection and calibration process of the sensor. In addition, special sections of urban roads such as curves, intersections, parking lots, etc. may also have an adverse effect on calibration.

[0053] Therefore, in order to ensure the accuracy and reliability of calibration, it is necessary to select specific areas far away from the city, such as suburbs, highways, etc., so as to perform calibration in a more ideal environment. In these areas, vehicles can travel at higher speeds, the lane lines are relatively clear, and there are relatively few vehicles around, which is conducive to obtaining accurate calibration data. During calibration, it is necessary to drive for a long time to ensure that enough data is obtained to complete the calibration and obtain accurate external parameters, thereby improving the perception and positioning capabilities of the on-board sensors, and thus improving the vehicle's autonomous driving and safety performance.

[0054] The implementation methods of the present application are described in detail below with reference to the accompanying drawings.

[0055] Figure 1 An example of a sensor calibration method for a vehicle in an embodiment of the present application is shown. Figure 1 As shown, including:

[0056] S1: Obtain first data and second data, wherein the first data is collected by an inertial sensor, and the second data is collected by a sensor to be calibrated, and the first data and the second data are collected under preset conditions; wherein the calibration sensor may be a lidar or a depth camera, and the present application does not impose a sole restriction on the type of the calibration sensor.

[0057] S2: Calibrate the external parameters of the sensor to be calibrated according to the first data and the second data.

[0058] Determine a section of the track with the highest straightness according to the first data; obtain second data corresponding to the section of the track with the highest straightness; and calibrate external parameters of the sensor to be calibrated according to the second data.

[0059] It can be understood that using the second data to determine the external parameters of the sensor to be calibrated includes performing relative posture calculations on the start and end points of each trajectory, and then converting the calculation results into external parameter calibration results through the principles of linear algebra.

[0060] The preset condition includes the driver driving the vehicle to turn an angle and driving straight for at least one section of the road in an area where the flatness is lower than a set threshold. That is, the second data includes data collected by driving straight and data collected by turning an angle.

[0061] From the above description, it can be seen that the vehicle sensor calibration method provided in the embodiment of the present application simultaneously collects the data of the inertial sensor and the sensor to be calibrated under preset conditions, so as to calibrate the vehicle-mounted sensor. The preset conditions can be set as needed. Compared with the prior art, complex scenes are not a must. Therefore, appropriate preset conditions can be matched according to the accuracy requirements of the calibration. In addition, on the premise that the accuracy requirements are met, there is no need for a large number of complex scene support. The data collection process has low requirements on the scene and driving method. The calibration process is simple to operate and does not affect the calibration accuracy, thereby solving the problem of the current complex calibration process.

[0062] The preset conditions of this application can be set as needed. The complex scene in the prior art is a must, while in this application, the scene can be set according to the calibration accuracy requirements, so a simpler scene can be set while meeting the accuracy requirements.

[0063] In addition, the current technical solutions for the calibration of vehicle-mounted sensors are relatively complex. Based on this, the inventors of the present application have discovered through extensive research that the calibration process of the external parameters of vehicle-mounted sensors is more affected by the flatness of the ground during driving than is conventionally known, and ordinary technicians in this field have not discovered this feature. As a result, the current calibration technology still has the problem of processing too much data and a complex calculation process on the one hand, and on the other hand, the original real data often cannot converge when correcting errors. That is, when the error of calibrating the external parameters is small in a certain scenario, even if a similar scenario is simply replaced, errors may recur, thereby further increasing the workload of the verification link.

[0064] This application is based on the unique discovery of the inventor that the impact of flatness is far greater than conventional knowledge, and provides a solution for calibrating vehicle-mounted sensors on a flat ground with a flatness below a set threshold. Since the solution is carried out on a ground with high flatness, the impact of the ground flatness is reduced to a minimum, or even to zero, so that the calibration of the vehicle-mounted sensors is not affected by flatness. Therefore, there is no need to set up a large number of driving behaviors and scene arrangements, and there is no need for a large amount of driving data to support the elimination of calibration errors.

[0065] Specifically, in the preferred embodiment provided by the present application, the preset condition includes the driver driving the vehicle to turn an angle and driving straight for at least one section of the road in an area where the flatness is lower than the set threshold. In this further preferred embodiment, the present application sets the preset condition as the driver driving the vehicle to turn an angle and driving straight for at least one section of the road in an area where the flatness is lower than the set threshold. Based on the inventor's discovery that flatness is the most important reason affecting the calibration result, after overcoming the influence of flatness, since the most important influencing factor has been overcome, it is no longer necessary to correct the calibration error through the huge amount of data formed by complex scenes. It is sufficient to directly configure the straight-line driving of a short distance and the rotation of an angle in the present application. The scene complexity of the present application is much lower than the complex scene of the prior art, and the calibration accuracy is not reduced, thereby providing a simpler and more efficient vehicle sensor calibration scheme. Specifically, in the embodiment of the present application, the flatness is lower than the set threshold, where the set threshold can be set according to the actual situation, for example, the flatness is less than 5cm, that is, in the entire area, the horizontal plane of the highest level ground and the horizontal plane of the lowest level ground differ by no more than 5cm.

[0066] Of course, the flatness of the present application affects the error size of the final calibration, that is, within an acceptable error range, a wider flatness can be set, and in any scenario, the flatness setting threshold can be set as needed, as long as it is within the range that meets the error requirements, and the present application does not impose any restrictions on this.

[0067] In the implementation manner of the present application, turning an angle and driving in a straight line are both simple single operations performed in an area where the flatness is lower than a set threshold. Specifically, for example, turning an angle can be performed by an operation similar to "reversing into a garage", that is, turning the steering wheel to a fixed angle and then starting it for a period of time. Similarly, driving in a straight line for at least one section of road, that is, returning the steering wheel to the correct position and driving a short distance at least once in the area, is, the area of ​​the present application is not a complex scene such as a highway.

[0068] It should be noted that the first data is the data collected by the inertial sensor, that is, it includes the angular velocity during each straight-line driving and each rotation process. The second data is the data collected by the sensor to be calibrated. Taking the lidar as an example, it includes the point cloud data during each straight-line driving and each rotation process. By analyzing the point cloud data, the rotation angle and the driving distance can be obtained. Taking the camera as an example, it includes the image during each straight-line driving and each rotation process. The rotation angle and the driving distance can be obtained through image recognition (which can be based on the simultaneous positioning and mapping algorithm (slam)), etc. This application will not go into details.

[0069] Specifically, an inertial sensor is a sensor used to measure the acceleration, angular velocity, and attitude change of an object. It can be composed of an accelerometer and a gyroscope. The accelerometer is used to measure the acceleration of an object, that is, the linear acceleration of an object in space. It can measure the acceleration of an object in the X, Y, and Z axes, thereby determining the object's state of motion. The gyroscope is used to measure the angular velocity of an object, that is, the speed at which an object rotates around the X, Y, and Z axes. It can measure the angular velocity of an object on each axis, thereby determining the object's rotation state. By combining the measurement data of the accelerometer and the gyroscope, the inertial sensor can provide information on the object's attitude change, including the object's angle, direction, and rotation speed.

[0070] In the implementation manner of the present application, the inertial sensor can be used to measure the rotation angle and angular velocity change of the vehicle. By recording the initial angle and the angle difference after rotation, the actual rotation angle can be calculated.

[0071] The sensor to be calibrated may be a lidar sensor or a camera sensor, which is not limited in this application.

[0072] Exemplarily, the sensor to be calibrated is a lidar sensor, which can be used to measure the distance and position information of an object. Specifically, the distance between the object and the sensor is determined by emitting a laser beam and measuring the reflection time of the laser beam. By measuring the reflection time, the sensor can calculate the distance between the object and the sensor.

[0073] Exemplarily, the sensor to be calibrated is a camera sensor, and the calibration of the camera's external parameters can determine the camera's position and posture. Specifically, the surrounding environment images before and after the driver starts the vehicle can be captured, and then the same objects appearing in the images can be selected as targets. The rotation angle or straight-line distance can be calculated through geometric relationships (such as the lateral length of the object in the image), or the rotation angle or straight-line distance can be calculated using an image recognition algorithm.

[0074] In an optional embodiment, the laser radar sensor and the camera sensor may be provided with a calibration plate for calibration. In the laser radar sensor and the camera sensor, the calibration plate may be set at a fixed position and used to assist positioning.

[0075] In the implementation manner of the present application, at least one road section can specifically be one road section or multiple road sections. It should be understood that the present application is based on the concept of flatness being lower than a set threshold, and can be calibrated using only one road section. In fact, the present application only requires one "straight" road section. Multiple road sections can be implemented in two ways. One is to use the results of homogenization of multiple road sections to eliminate the impact caused by "actual non-straightness". The second is that the present application can calculate the straightness of each road section and select the driving trajectory with the highest straightness.

[0076] For the first implementation method, the present application can perform trajectory fitting and averaging on the trajectories of multiple road sections to obtain a comprehensive average trajectory, and then use the comprehensive trajectory to calibrate the lidar sensor or camera sensor. For the second implementation method, the present application can first calculate the straightness of the trajectory of each of the multiple road sections, and then select the trajectory with the highest straightness as the straight line calibration trajectory of the sensor to be calibrated. Specifically, before calibrating the external parameters of the sensor to be calibrated according to the first data and the second data, such as Figure 2 As shown, the sensor calibration method also includes:

[0077] S01: determine the section with the highest trajectory straightness;

[0078] S02: Calibrate the external parameters of the sensor to be calibrated according to the second data corresponding to the section of the track with the highest straightness.

[0079] It can be understood that the second data includes data collected by the vehicle straight-line driving sensor and data collected by the rotation angle sensor.

[0080] It should be noted that the present application provides methods for determining the straightness of different trajectories of different sensors. The sensors involved in the present application include lidar sensors, camera sensors and inertial sensors. The specific methods for determining the straightness of the above three sensors are explained one by one below.

[0081] For the inertial sensor in the embodiment of the present application, the straightness can be determined in the following manner: Figure 3 and Figure 4 As shown, the determination of a section of road with the highest trajectory straightness includes:

[0082] S011a: With the angular velocity corresponding to each road section as the Y-axis and the track duration as the X-axis, a coordinate graph of the angular velocity versus track duration is drawn;

[0083] S012a: Calculate the area of ​​the closed region formed by the angular velocity curve on the X-axis, and determine the section with the smallest area as the section with the highest trajectory straightness.

[0084] Specifically, Figure 4 As shown, by calculating Figure 4 The shadow area S of the middle curve is used to determine the cumulative rotation angle of the vehicle body in the entire track. The calculation method in the program is numerical integration. If the curve is distributed in both the positive and negative areas of the y-axis, the total shadow area of ​​the positive and negative areas is calculated respectively, and then the absolute value of the difference between the two is calculated as S. The track segment with the smallest S is the straightest. It can be seen that the above implementation provides a method for measuring the straightness of an inertial sensor, which cleverly uses the change in angular velocity to measure the straightness.

[0085] In addition, for the laser radar sensor, the straightness can be measured by point cloud data. Specifically, the sensor to be calibrated is a laser radar, the second data is point cloud data, and the section with the highest straightness of the trajectory is determined, such as Figure 5 As shown, including:

[0086] S011b: Determine the rotation angle corresponding to each road section according to the point cloud data of the laser radar at the starting point and the end point of each road section;

[0087] S011b: The section with the smallest rotation angle is regarded as the section with the highest trajectory straightness.

[0088] For example, by calculating Figure 6The rotation transformation relationship R between the laser point clouds in the initial and final static states is characterized as (rotation axis, rotation angle), and the trajectory with the smallest absolute value of the rotation angle is the trajectory with the best straightness. Specifically, the point cloud data in the static state at the starting point position is compared with the point cloud data in the static state at the end point position, and the point clouds of the two point cloud images are subjected to rotation transformation to obtain the rotation angle relationship between the two, and the trajectory with the smallest absolute value of the rotation angle is selected as the true trajectory with the highest trajectory straightness. It can be seen that the present application determines the straightness through rotation transformation for the laser radar sensor.

[0089] Furthermore, when the sensor is a camera sensor, the rotation angle can be determined by an image. Specifically, the sensor to be calibrated is a camera sensor, the second data is image data, and the section of the road with the highest straightness of the trajectory is determined, such as Figure 7 As shown, including:

[0090] S011c: Determine the camera posture at the initial position and the end position according to the image data continuously captured during the driving process of each road section;

[0091] S012c: Determine the rotation angle corresponding to each road section according to the camera posture at the initial position and the end position of each road section;

[0092] S013c: The section with the smallest rotation angle is regarded as the section with the highest trajectory straightness.

[0093] For example, the visual SLAM (Simultaneous Localization and Mapping) method can be used to calculate the image quality of a series of continuous images captured during the trajectory. Figure 8 The rotation transformation relationship R between the camera postures in the initial and final static states is represented as (rotation axis, rotation angle). The trajectory with the smallest absolute value of the rotation angle is the trajectory with the best straightness.

[0094] Of course, for a vehicle, camera sensors, lidar sensors and inertial sensors are often present. Therefore, based on this, the present application further provides a method for determining straightness by comprehensively utilizing the above-mentioned multiple sensors. Specifically, the method determines a section of the road with the highest straightness of the trajectory, such as Fig. 9 As shown, including:

[0095] S011d: With the angular velocity corresponding to each road section as the Y-axis and the track duration as the X-axis, a coordinate graph of the angular velocity versus track duration is drawn;

[0096] S012d: Calculate the area of ​​the closed region formed between the x-axis and the real trajectory corresponding to each road segment;

[0097] S013d: Determine a first rotation angle corresponding to each road section according to the point cloud data of the laser radar at the starting point and the end point of each road section;

[0098] S014d: Determine the camera posture at the initial position and the end position according to the image data continuously captured during the driving process of each road section;

[0099] S015d: Determine a second rotation angle corresponding to each road section according to the camera posture at the initial position and the end position of each road section;

[0100] S016d: Determine the section of the track with the highest straightness according to the area, the first rotation angle, and the second rotation angle in combination with preset weights corresponding to the area, the first rotation angle, and the second rotation angle.

[0101] It should be noted that the track straightness is inversely proportional to the area, the first rotation angle and the second rotation angle, so the track straightness can be set to k1 / area+k2 / first rotation angle+k3 / second rotation angle, where k1+k2+k3=1. In this embodiment, the three sensors are integrated, so the track straightness calculation is more accurate.

[0102] After selecting a straight section with the highest trajectory straightness, the external parameters of the sensor to be calibrated are calibrated based on the straight section and sensor data before and after a driver turns an angle.

[0103] It should be understood that for LiDAR and camera, extrinsic parameters refer to the parameters of their posture and position relative to the vehicle coordinate system or the world coordinate system. The extrinsic parameters include translation vectors and rotation matrices, which are used to transform the LiDAR point cloud data or the 3D feature points of the camera from the LiDAR coordinate system and the camera coordinate system to the vehicle coordinate system or the world coordinate system, that is, to transform the coordinate system of the sensor to be calibrated to a unified coordinate system, so that the data of the points in the same coordinate system can correspond.

[0104] Taking the lidar sensor as an example, the specific calibration process includes the ICP (Iterative Closest Point) point cloud registration algorithm, which is used to calculate the rigid body transformation between two point clouds. By performing ICP calculation between the point clouds in the starting and ending states, the optimal rigid body transformation between them can be obtained.

[0105] In the implementation mode of this application, the basic steps of the ICP algorithm are as follows:

[0106] Step 1: Initialize an initial transformation matrix, such as the identity matrix.

[0107] Step 2: In each iteration, the point cloud in the initial state is transformed according to the current transformation matrix to obtain the transformed point cloud.

[0108] Step 3: Match the transformed point cloud with the point cloud in the terminal state to find the nearest neighbor point correspondence.

[0109] Step 4: Calculate the optimal rigid body transformation matrix based on the nearest neighbor point correspondence found.

[0110] Step 5: Current transformation matrix, multiply the optimal rigid body transformation matrix with the current transformation matrix.

[0111] Repeat steps 2-5 until the stopping condition is met (for example, the number of iterations reaches a set value or the error is less than a threshold).

[0112] Exemplarily, in the implementation of the present application, the ICP calculation of the point cloud of the two segments of data in the starting and ending states can be performed according to the following steps: preprocess the point cloud of the starting state and the point cloud of the ending state, such as removing outliers, filtering, etc., initialize the initial transformation matrix, such as the unit matrix, and in each iteration, transform the point cloud of the starting state according to the current transformation matrix to obtain the transformed point cloud, match the transformed point cloud with the point cloud of the ending state, find the nearest neighbor point correspondence, calculate the optimal rigid body transformation matrix based on the nearest neighbor point correspondence found, update the current transformation matrix, and multiply the optimal rigid body transformation matrix with the current transformation matrix. Repeat the above steps until the stop condition is met (for example, the number of iterations reaches the set value or the error is less than the threshold), and the final transformation matrix is ​​the optimal rigid body transformation from the point cloud of the starting state to the point cloud of the ending state.

[0113] The method for calculating the final rotation extrinsic parameter matrix (3x3) (i.e., the rotation extrinsic parameter matrix of the radar) from the rigid body transformation results (4x4) of the start and end point clouds obtained by the two ICPs is:

[0114] The direction (3x1) of the translation component is extracted from the ICP result of the straight trajectory, and the direction vector is the first column of the rotation extrinsic parameter matrix; the rotation axis (3x1) of the rotation component is extracted from the ICP result of the turning trajectory, and the direction vector is the third column of the rotation extrinsic parameter matrix; the second column of the rotation extrinsic parameter matrix is ​​obtained by the cross product of the vectors of the first and third columns.

[0115] It should be noted that in the implementation of the present application, the ICP algorithm is sensitive to the selection of the initial transformation matrix. Selecting a suitable initial transformation matrix can speed up the convergence of the algorithm. In addition, the ICP algorithm has high requirements for the initial alignment between point clouds. If there is a large initial deviation between two point clouds, it may be necessary to use other methods for coarse registration, and then use the ICP algorithm for fine registration.

[0116] In addition, for the camera, the specific calibration of external parameters can be used to extract and match features of two segments of data (a straight section and a rotation angle) throughout the entire process, and perform scale-free visual SLAM (simultaneous localization and mapping) calculations.

[0117] Specifically, for the scale-free visual SLAM calculation of the camera, the following steps can be followed:

[0118] 1. Feature extraction: For each frame of the image, use a feature extraction algorithm (such as SIFT, ORB, etc.) to extract key points and their corresponding descriptors.

[0119] 2. Feature matching: Match the feature descriptor of the current frame with the feature description of the previous frame. Feature matching algorithms (such as FLANN, RANSAC, etc.) can be used to match feature points.

[0120] 3. Triangulation: Based on the matching relationship of feature points, the triangulation algorithm is used to calculate the camera's pose and the 3D points in the scene.

[0121] 4. Initialize the map: In the initial frame, add the calculated 3D points to the map.

[0122] 5. Pose estimation: Based on the known 3D points and their projections in the current frame, the PnP algorithm is used to estimate the camera's pose.

[0123] It should be noted that the above steps are only the basic framework of the SLAM algorithm, and many details need to be considered in the specific implementation, such as feature selection, matching screening, initial value of posture estimation, etc. In addition, for the scale-free SLAM algorithm, it can be solved by motion estimation, gravity constraint and other methods, which will not be described in detail in this application.

[0124] It can be seen that the implementation method of the present application can calibrate the vehicle-mounted sensor by rotating the vehicle on flat ground and moving a short distance, so that the data collection process has low requirements on the scene and driving method. The calibration process is simple to operate and does not affect the calibration accuracy, solving the problem of the current complex calibration process.

[0125] Furthermore, in other optional embodiments, the present application also provides a sensor calibration device for a vehicle, such as Fig.10 As shown, including:

[0126] Acquisition module 1, acquiring first data and second data when the driver drives the vehicle to turn an angle and travels straight for at least one section of the road in an area where the flatness is lower than a set threshold, wherein the first data is acquired by an inertial sensor and the second data is acquired by a sensor to be calibrated;

[0127] The calibration module 2 calibrates the external parameters of the sensor to be calibrated according to the first data and the second data.

[0128] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / modules are based on the same concept as the method embodiment of the present application. Their specific functions and technical effects can be found in the method embodiment part and will not be repeated here.

[0129] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0130] From a hardware perspective, in order to provide an embodiment of an electronic device for implementing all or part of the contents of the sensor calibration method, the electronic device specifically includes the following contents:

[0131] Processor, memory, communication interface and bus; wherein the processor, memory and communication interface communicate with each other through the bus; the communication interface is used to realize information transmission between related devices such as servers, devices, distributed message middleware cluster devices, various databases and user terminals; the electronic device can be a desktop computer, a tablet computer and a mobile terminal, etc., but the present embodiment is not limited thereto. In the present embodiment, the electronic device can be implemented with reference to the embodiment of the sensor calibration method and the embodiment of the sensor calibration device in the embodiment, and the contents thereof are incorporated herein and the repeated parts are not repeated.

[0132] Fig.11 FIG. 9 is a schematic block diagram of a system structure of an electronic device 9600 according to an embodiment of the present invention. Fig.11 As shown, the electronic device 9600 may include a central processor 9100 and a memory 9140; the memory 9140 is coupled to the central processor 9100. It is worth noting that Fig.11is exemplary; other types of structures may also be used to supplement or replace this structure to implement telecommunication functions or other functions.

[0133] In one embodiment, the sensor calibration function can be integrated into the central processing unit 9100.

[0134] In another embodiment, the sensor calibration device may be configured separately from the central processing unit 9100. For example, the sensor calibration device may be configured as a chip connected to the central processing unit 9100, and the sensor calibration function may be implemented under the control of the central processing unit.

[0135] like Fig.11 As shown, the electronic device 9600 may also include: a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It is worth noting that the electronic device 9600 does not necessarily have to include Fig.11 In addition, the electronic device 9600 may also include Fig.11 For components not shown, reference may be made to the prior art.

[0136] like Fig.11 As shown, the central processing unit 9100 is sometimes also referred to as a controller or an operation control, and may include a microprocessor or other processor device and / or logic device. The central processing unit 9100 receives input and controls the operation of various components of the electronic device 9600.

[0137] The memory 9140 may be, for example, one or more of a cache, a flash memory, a hard drive, a removable medium, a volatile memory, a non-volatile memory or other suitable devices. The above-mentioned information related to the failure may be stored, and a program for executing the relevant information may also be stored. The CPU 9100 may execute the program stored in the memory 9140 to implement information storage or processing, etc.

[0138] The input unit 9120 provides input to the central processing unit 9100. The input unit 9120 is, for example, a key or a touch input device. The power supply 9170 is used to provide power to the electronic device 9600. The display 9160 is used to display display objects such as images and texts. The display may be, for example, an LCD display, but is not limited thereto.

[0139] The memory 9140 may be a solid-state memory, such as a read-only memory (ROM), a random access memory (RAM), a SIM card, etc. It may also be a memory that saves information even when the power is off, can be selectively erased, and is provided with more data, examples of which are sometimes referred to as EPROMs, etc. The memory 9140 may also be some other type of device. The memory 9140 includes a buffer memory 9141 (sometimes referred to as a buffer). The memory 9140 may include an application / function storage unit 9142, which is used to store application programs and function programs or processes for executing the operation of the electronic device 9600 through the central processor 9100.

[0140] The memory 9140 may also include a data storage unit 9143 for storing data, such as contacts, digital data, pictures, sounds, and / or any other data used by the electronic device. The driver storage unit 9144 of the memory 9140 may include various drivers for communication functions of the electronic device and / or for executing other functions of the electronic device (such as messaging applications, address book applications, etc.).

[0141] The communication module 9110 is a transmitter / receiver 9110 that sends and receives signals via an antenna 9111. The communication module (transmitter / receiver) 9110 is coupled to the central processor 9100 to provide input signals and receive output signals, which may be the same as the case of a conventional mobile communication terminal.

[0142] Based on different communication technologies, multiple communication modules 9110 may be provided in the same electronic device, such as a cellular network module, a Bluetooth module and / or a wireless LAN module, etc. The communication module (transmitter / receiver) 9110 is also coupled to a speaker 9131 and a microphone 9132 via an audio processor 9130 to provide an audio output via the speaker 9131 and receive an audio input from the microphone 9132, thereby realizing a common telecommunication function. The audio processor 9130 may include any suitable buffer, decoder, amplifier, etc. In addition, the audio processor 9130 is also coupled to the central processor 9100, so that recording can be performed on the local machine through the microphone 9132, and the sound stored on the local machine can be played through the speaker 9131.

[0143] An embodiment of the present invention also provides a computer-readable storage medium capable of implementing all the steps of the sensor calibration method in the above embodiment, wherein the execution subject may be a server. A computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, all the steps of the sensor calibration method in the above embodiment are implemented.

[0144] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0145] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (apparatus), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0146] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0147] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0148] The present invention uses specific embodiments to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A vehicle sensor calibration method, characterized in that: include: Acquire first data and second data, wherein the first data is collected by an inertial sensor, the second data is collected by a sensor to be calibrated, and the first data and the second data are collected under preset conditions; The external parameters of the sensor to be calibrated are calibrated according to the first data and the second data.

2. The vehicle sensor calibration method according to claim 1, characterized in that: The preset conditions include the driver driving the vehicle to turn an angle in an area where the flatness is lower than a set threshold and driving straight for at least one section of the road.

3. The vehicle sensor calibration method according to claim 1, characterized in that: Before calibrating the external parameters of the sensor to be calibrated according to the first data and the second data, the sensor calibration method further includes: Determine the section of road with the highest track straightness; The external parameters for calibrating the sensor to be calibrated are obtained according to the first data corresponding to the section of the track with the highest straightness.

4. The vehicle sensor calibration method according to claim 3, characterized in that: The first data includes an angular velocity, and the determining of a section of the road with the highest trajectory straightness includes: With the angular velocity corresponding to each road section as the Y-axis and the track duration as the X-axis, a coordinate graph of the angular velocity versus track duration is drawn; The area of ​​the closed region formed by the angular velocity curve on the X-axis is calculated, and the section with the smallest area is determined as the section with the highest trajectory straightness.

5. The sensor calibration method for a vehicle according to claim 3, characterized in that: The sensor to be calibrated is a laser radar, the second data is point cloud data, and the determining of a section of the road with the highest trajectory straightness includes: Determine the rotation angle corresponding to each road section according to the point cloud data of the laser radar at the starting point and the end point of each road section; The section with the smallest rotation angle is regarded as the section with the highest trajectory straightness.

6. The vehicle sensor calibration method according to claim 3, characterized in that: The sensor to be calibrated is a camera sensor, the second data is image data, and the determining of a section of a road with the highest trajectory straightness includes: Determine the camera posture at the initial position and the end position based on the image data continuously captured during the driving process of each road section; Determine the rotation angle corresponding to each road section according to the camera posture at the initial position and the end position of each road section; The section with the smallest rotation angle is regarded as the section with the highest trajectory straightness.

7. The vehicle sensor calibration method according to claim 3, characterized in that: The sensors to be calibrated include camera sensors and laser radars, and the section of the road with the highest trajectory straightness is determined by: With the angular velocity corresponding to each road section as the Y-axis and the track duration as the X-axis, a coordinate graph of the angular velocity versus track duration is drawn; Calculating the area of ​​a closed region formed between the X-axis and the real trajectory corresponding to each road segment; Determine a first rotation angle corresponding to each road section according to the point cloud data of the laser radar at the starting point and the end point of each road section; Determine the camera posture at the initial position and the end position based on the image data continuously captured during the driving process of each road section; Determine a second rotation angle corresponding to each road section according to the camera posture at the initial position and the end position of each road section; The section of the track with the highest straightness is determined according to the area, the first rotation angle, and the second rotation angle in combination with preset weights corresponding to the area, the first rotation angle, and the second rotation angle.

8. The vehicle sensor calibration method according to claim 1, characterized in that: The step of calibrating the external parameters of the sensor to be calibrated according to the first data and the second data includes: Determine a section of the track with the highest straightness according to the first data; Acquire second data corresponding to a section of the track with the highest straightness; The external parameters of the sensor to be calibrated are calibrated according to the second data.

9. A sensor calibration device for a vehicle, characterized in that: include: An acquisition module, acquiring first data and second data, wherein the first data is acquired by an inertial sensor, the second data is acquired by a sensor to be calibrated, and the first data and the second data are acquired under preset conditions; The calibration module calibrates the external parameters of the sensor to be calibrated according to the first data and the second data.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the sensor calibration method for the vehicle according to any one of claims 1 to 8 is implemented.

Images