A data calibration method and system for a vehicle and the vehicle

Abstract

The application provides a data calibration method and system of a vehicle and the vehicle, and relates to the technical field of automatic driving of the vehicle.In the application, initial point cloud data of the vehicle is acquired by using a first sensor of the vehicle when the vehicle is running, then voxelization processing is performed on the initial point cloud data, and intermediate features are acquired through a sparse convolution network, then a residual neural network is constructed, the intermediate features are extracted based on the residual neural network to obtain final features, and finally the final features are fused to obtain processed target point cloud data.The technical scheme is used to prevent the Centerpoint network from degrading, the residual neural network (ResNet) idea is used, the neuron information of the first several layers of the neural network is retained, some layers of the neural network are artificially caused to skip the connection of the next layer of neurons, the two neurons of the layers are connected, the strong connection between each layer is weakened, and the accuracy of target recognition is improved.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and in particular to a data calibration method, data calibration system, and vehicle for vehicles. Background Technology

[0002] With the development of artificial intelligence and 5G technology, more and more companies and universities are focusing on the research of autonomous vehicles. An autonomous vehicle is an intelligent vehicle system that achieves driverless operation through an onboard computer system; it is also known as a driverless car, computer-driven car, or wheeled mobile robot. It consists of four parts: a perception system, a decision-making system, an execution system, and a communication system. Among these, the perception system is the core of autonomous driving intelligence, and sensor calibration is the foundation for maintaining the normal operation of the perception system.

[0003] Current academic and industrial research on the calibration of radar (Lidar), depth camera, and ADAS Positioning Unit (ADPU) in autonomous vehicles largely focuses on using specialized calibration boards. This involves extracting feature points from the board and then calibrating the parameters between various sensors in conjunction with specific vehicle movements. This method has high requirements for the testing environment, and if calibration parameter issues arise during subsequent user use, recalibration at the factory is necessary, resulting in long processing times, high costs, and a poor user experience. Other methods take a different approach, using hand-eye calibration and its improvements to solve for the parameters between sensors. However, these methods require significant vehicle rotation along each coordinate axis, which is impractical for autonomous vehicles that can only move in a planar plane. Furthermore, these methods are susceptible to interference from complex environments, leading to incorrect calibration results. There are also improved two-step methods for calibrating radar and GPS, but these methods do not consider the influence of the environment and the sensors themselves. They directly use point cloud matching algorithms to calculate the radar displacement, making them susceptible to dynamic and complex environments, resulting in significant deviations. GPS pose transformation is also easily affected by signals, which can introduce incorrect transformation matrices. These deviations ultimately lead to large errors in the results, which may exceed 10cm. It is necessary to repeatedly calibrate in a simple static environment until a reasonable result is obtained. Summary of the Invention

[0004] The first objective of this invention is to provide a vehicle data calibration method to solve the technical problem of inaccurate vehicle data calibration in the prior art.

[0005] The second aspect of the present invention is to provide a vehicle data calibration system.

[0006] The third aspect of the present invention is to provide a vehicle having a data calibration system.

[0007] According to a first aspect of the present invention, the present invention provides a vehicle data calibration method, comprising the following steps:

[0008] The initial point cloud data of the vehicle is acquired using the vehicle's first sensor while the vehicle is in motion.

[0009] The initial point cloud data is voxelized, and intermediate features are obtained through a sparse convolutional network.

[0010] Construct a residual neural network;

[0011] The intermediate features are extracted based on the residual neural network to obtain the final features;

[0012] The final features are fused to obtain the processed target point cloud data.

[0013] Optionally, the step of fusing the final features to obtain the processed target point cloud data further includes the following steps:

[0014] The target point cloud data at the current moment and the target point cloud data at the next moment are matched to obtain the transformation relationship of the first sensor in a pre-established coordinate system.

[0015] Optionally, the following steps are also included:

[0016] The vehicle's position information is obtained using the vehicle's second sensor while the vehicle is in motion;

[0017] The transformation relationship of the second sensor in the coordinate system is obtained based on the position information at the current moment and the position information at the next moment.

[0018] Optionally, the step of obtaining the transformation relationship of the second sensor in the coordinate system based on the position information at the current moment and the position information at the next moment further includes the following steps:

[0019] Abnormal data in the transformation relationships of the first sensor and the second sensor are deleted based on the transformation relationships of the first sensor and the second sensor.

[0020] Optionally, the first sensor includes radar and a depth camera, and the second sensor includes an autonomous driving positioning unit.

[0021] Optionally, the step of deleting abnormal data in the transformation relationships of the first sensor and the second sensor based on the transformation relationships of the first sensor specifically includes the following steps:

[0022] The rotation matrices in the transformation relationship of the radar, the transformation relationship of the depth camera, and the transformation relationship of the autonomous driving positioning unit are respectively converted into Euler angles;

[0023] The first variance value is calculated based on the Euler angles of the radar, the Euler angles of the depth camera, and the Euler angles of the autonomous driving positioning unit;

[0024] When the first variance value is greater than the first preset value, delete the rotation matrix in the transformation relationship of the radar, the rotation matrix in the transformation relationship of the depth camera, and the rotation matrix in the transformation relationship of the autonomous driving positioning unit.

[0025] Optionally, the step of obtaining the transformation relationship of the second sensor in the coordinate system based on the position information at the current moment and the position information at the next moment further includes the following steps:

[0026] The displacement distance is calculated based on the translation matrix in the transformation relationship of the radar, the rotation matrix in the transformation relationship of the depth camera, and the translation matrix in the transformation relationship of the autonomous driving positioning unit, respectively.

[0027] The second variance value is calculated based on the displacement distance of the radar, the displacement distance of the depth camera, and the displacement distance of the autonomous driving positioning unit;

[0028] When the second variance value is greater than the second preset value, delete the translation matrix in the transformation relationship of the radar, the translation matrix in the transformation relationship of the depth camera, and the translation matrix in the transformation relationship of the autonomous driving positioning unit.

[0029] Optionally, the step of matching the target point cloud data at the current moment and the target point cloud data at the next moment to obtain the transformation relationship of the first sensor in a pre-established coordinate system further includes the following steps before the step:

[0030] Obstacles that are semantically dynamic obstacles and ground point cloud data are removed from the target point cloud data using a semantic segmentation model or the RANSAC algorithm.

[0031] According to a second aspect of the present invention, the present invention also provides a vehicle data calibration system, comprising:

[0032] The control module includes a memory and a processor. The memory stores a calculation program, which, when executed by the processor, is used to implement the data calibration method described above.

[0033] According to a third aspect of the present invention, the present invention also provides a vehicle including the data calibration system described above.

[0034] In this invention, the vehicle's first sensor acquires initial point cloud data while the vehicle is in motion. This initial point cloud data is then voxelized, and intermediate features are obtained through a sparse convolutional network. A residual neural network is then constructed, and the intermediate features are extracted based on the residual neural network to obtain the final features. Finally, the final features are fused to obtain the processed target point cloud data. To prevent Centerpoint network degradation, the above technical solution uses the concept of a residual neural network (ResNet), preserving information from the first few layers of neurons. It artificially allows certain neurons in the neural network to skip connections to the next layer, connecting neurons in alternate layers and weakening the strong connections between layers, thereby improving the accuracy of target recognition.

[0035] The above and other objects, advantages and features of the present invention will become more apparent to those skilled in the art from the following detailed description of specific embodiments of the invention in conjunction with the accompanying drawings. Attached Figure Description

[0036] The following sections will describe some specific embodiments of the invention in detail by way of example and not limitation, with reference to the accompanying drawings. The same reference numerals in the drawings denote the same or similar parts or portions. Those skilled in the art should understand that these drawings are not necessarily drawn to scale. In the drawings:

[0037] Figure 1 This is a schematic flowchart of a vehicle data calibration method according to an embodiment of the present invention;

[0038] Figure 2 This is a schematic flowchart of a vehicle data calibration method according to another embodiment of the present invention;

[0039] Figure 3 This is a schematic connection block diagram of a vehicle data calibration system according to an embodiment of the present invention. Detailed Implementation

[0040] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0041] Figure 1 This is a schematic flowchart of a vehicle data calibration method according to an embodiment of the present invention. Figure 1 As shown, in a specific embodiment, the vehicle data calibration method includes the following steps:

[0042] Step S100: Acquire initial point cloud data of the vehicle using the vehicle's first sensor while the vehicle is in motion;

[0043] Step S200: Voxelize the initial point cloud data and obtain intermediate features through a sparse convolutional network;

[0044] Step S300: Construct a residual neural network;

[0045] Step S400: Extract intermediate features based on residual neural network to obtain final features;

[0046] Step S500: The final features are fused to obtain the processed target point cloud data.

[0047] Here, the final features are extracted using a residual neural network, namely ResNet, as the backbone. Then, the final features are concatenated and fused through the neck, and finally passed to the centerhead to complete the target recognition.

[0048] To prevent Centerpoint network degradation, this embodiment uses the concept of ResNet, which preserves the information of neurons in the first few layers of the neural network. It artificially allows neurons in certain layers of the neural network to skip the connection of neurons in the next layer, so that the two neurons in the next layer are connected, weakening the strong connection between each layer, thereby improving the accuracy of target recognition.

[0049] This embodiment uses an offline calculation method because it is time-consuming, requires high computing power, and demands high accuracy. In other embodiments, if the Centerpoint network is not improved, an online calculation method is used. Although this method is time-efficient, it requires less computing power and has lower accuracy.

[0050] In this embodiment, step S500 is followed by the following steps:

[0051] Step S600: Match the target point cloud data at the current time with the target point cloud data at the next time to obtain the transformation relationship of the first sensor in the pre-established coordinate system. Here, NDT (Normal Distribution Transform) and ICP (Iterative Closest Point) are used to match the target point cloud data.

[0052] In this embodiment, the first sensor includes a radar and a depth camera, which are mounted on the vehicle. When the autonomous vehicle performs an S-shaped motion, the radar and depth camera respectively collect data throughout the vehicle's movement. For the radar, the point cloud recorded at time i is called Pointcloud.i The radar point cloud is recorded at time i+1 as Pointcloud. i+1 This embodiment improves the Centerpoint network by adopting the ResNet concept, preserving information from the first few layers of neurons and thus improving target recognition accuracy. This embodiment utilizes the improved Centerpoint network to target Pointcloud. i With Pointcloud i+1 To perform target recognition in point clouds, dynamic obstacles such as people, vehicles, and animals are selected and separated from the point cloud. After this process, what remains in the point cloud is the static environment.

[0053] In this embodiment, the following steps are included before step S600:

[0054] Obstacles that are semantically dynamic obstacles and ground point cloud data are removed from the target point cloud data using a semantic segmentation model or the RANSAC algorithm. Here, the semantic segmentation model is CENet.

[0055] After the above steps, the point cloud matching algorithms NDT and ICP are used to match the point cloud where only the static environment remains. i With Pointcloud i+1 The transformation relationship of the radar in the world coordinate system from time i to i+1 is denoted as: At this point, the radar's transformation relationship only includes the influence of some sensor-inherent noise, and no longer includes the influence of complex environmental factors. In principle, the depth camera processes point cloud data in the same way as the radar, obtaining the transformation relationship of the depth camera in the world coordinate system from time i to i+1, denoted as...

[0056] Figure 2 This is a schematic flowchart of a vehicle data calibration method according to another embodiment of the present invention. Figure 2 As shown, in this embodiment, in step S100, when the vehicle is in motion, not only is the initial point cloud data of the vehicle obtained using the vehicle's first sensor, but the vehicle's position information is also obtained using the vehicle's second sensor. The data calibration method further includes the following steps:

[0057] Step S700: Based on the current position information and the next position information, the transformation relationship of the second sensor in the coordinate system is obtained. Here, there is no sequential relationship between steps S700 and S600. In this embodiment, the first sensor includes radar and a depth camera, and the second sensor includes an autonomous driving positioning unit, i.e., ADPU.

[0058] Following steps S600 and S700, the following steps are also included:

[0059] Step S710: Convert the rotation matrix in the transformation relationship of the radar, the rotation matrix in the transformation relationship of the depth camera, and the rotation matrix in the transformation relationship of the autonomous driving positioning unit into Euler angles respectively.

[0060] Step S720: Calculate the first variance value based on the Euler angles of the radar, the Euler angles of the depth camera, and the Euler angles of the autonomous driving positioning unit.

[0061] Step S730: When the first variance value is greater than the first preset value, delete the rotation matrix in the transformation relationship of the corresponding radar, the rotation matrix in the transformation relationship of the depth camera, and the rotation matrix in the transformation relationship of the autonomous driving positioning unit; here, the first preset value is set according to specific design requirements.

[0062] Step S740: Calculate the displacement distance based on the translation matrix in the radar transformation relationship, the rotation matrix in the depth camera transformation relationship, and the translation matrix in the autonomous driving positioning unit transformation relationship, respectively.

[0063] Step S750: Calculate the second variance value based on the displacement distance of the radar, the displacement distance of the depth camera, and the displacement distance of the autonomous driving positioning unit.

[0064] In step S760, when the second variance value is greater than the second preset value, the translation matrix in the transformation relationship of the corresponding radar, the translation matrix in the transformation relationship of the depth camera, and the translation matrix in the transformation relationship of the autonomous driving positioning unit are deleted. It should be noted that steps S740 and S710 are not sequential. The second preset value is set according to specific design requirements.

[0065] In existing technologies, radar and depth cameras may exhibit data anomalies, and ADPU may experience positioning lag or significant deviations when the signal is poor. These anomalies negatively impact the calculation of subsequent sensor parameters. Therefore, this embodiment uses mutual constraints among the three sensors to remove abnormal data. As the vehicle moves from time i to time i+1, we can assume that the vehicle's velocity direction at time i is v. i The angle between v and the y-axis is α, v i With v i+1 The included angle is β, and at time i, the sensor S i The positive direction of v i The included angle is γ. Since the sensor is rigidly mounted, at time i+1, the sensor S... i+1 The positive direction of v i+1 The included angle is still γ. Therefore, S i+1 Positive direction and S iThe included angle in the positive direction is (α+β+γ)-(α+γ)=β, which shows that no matter how the object moves, all sensors rotate at the same angle within the same time period.

[0066] This embodiment cleanses point cloud data from two aspects: rotation angle and translation. This embodiment uses known radar transformation relationships in the world coordinate system. in It is a rotation matrix. It is a translation matrix. Similarly, for a depth camera, the rotation matrix is... The translation matrix is For ADPU, the rotation matrix is: The translation matrix is Then and Convert to Euler angles and calculate the first variance. If the calculated first variance is greater than a first preset value, it indicates a problem with these three values, and the three rotation matrices in this data set need to be deleted, which represent the radar, depth camera, and ADPU data at this moment. Simultaneously, it is also necessary to... and The displacement distances are calculated separately, and then the second variance is calculated for each of the three displacement distances. If the calculated second variance is greater than a second preset value, it indicates that there is a problem with these three values, and the three translation matrices in this data set need to be deleted. This removes abnormal data caused by the sensor itself. This embodiment calibrates the ADPU together with the radar and depth camera, utilizing the synchronization of multiple sensors and fusing data from multiple sensors to improve the accuracy of single data calibration.

[0067] In this embodiment, after removing abnormal rotation and translation matrices, the origin of the vehicle coordinate system used is the projection point of the ADPU on the ground. Therefore, it is necessary to calculate the distances from the radar and depth camera to the ground, as well as the height difference between the radar and depth camera. The ground point cloud is separated using the semantic segmentation model CENet, and then the ground is fitted using the RANSAC algorithm. The distance z from the radar and depth camera to the ground can then be calculated using the point-to-surface distance calculation formula, thus allowing the calculation of the height difference between the radar and depth camera. This embodiment uses the ground as a reference and, in conjunction with intelligent algorithms, calculates the height difference between each sensor, solving the problem of Z-axis offset not being calibrated during planar motion.

[0068] In this embodiment, after calculating the distance Z from the radar and depth camera to the ground, the three sensors are calibrated pairwise according to the improved two-step hand-eye calibration method. The radar, depth camera, and ADPU can calibrate each other, as long as the displacement and rotation matrices are available. Once the three sensors are calibrated pairwise, it is equivalent to all three sensors being calibrated. For example, knowing the calibration relationship between the radar and depth camera, and the calibration relationship between the depth camera and ADPU, allows us to know the calibration relationship between the radar and ADPU. This embodiment takes the calibration of the radar and depth camera as an example. In this embodiment, the transformation matrix of the depth camera from time i to time i+1 is known. The transformation matrix of the radar from time i to time i+1 To find the transformation matrix T of the radar to the depth camera L,C ,set up Based on the hand-eye calibration formula, we can obtain the following formulas (1) and (2):

[0069]

[0070]

[0071] In formula (2) above, I3 represents the third-order identity matrix.

[0072] To eliminate interference from real-world noise, the noise is substituted into formulas (1) and (2) above. As can be seen from the principle of hand-eye calibration, noise will not appear at T. L,C In this process, only and In these two processes, we record... The noise generated by translation is The noise generated by translation is remember The noise generated by rotation is The noise generated by rotation is Then formula (1) can be transformed into formula (4), and formula (2) can be transformed into formula (5):

[0073]

[0074]

[0075] This embodiment requires the calculation of six parameters: the translation matrix [x, y, z] and the Euler angles [α, β, γ] corresponding to the rotation matrix. The first step of the two-step method optimizes β and γ, i.e., the roll angle and the pitch angle, through the angle relationship, i.e., formula (4). The second step optimizes the results x, y, z and α, i.e., the yaw angle, through the angle and displacement relationship, i.e., formula (5). Finally, the calculated z is added to obtain all the parameters.

[0076] This embodiment uses a noisy, improved two-step kinematic model to calculate calibration parameters, eliminating the influence of noise during motion and improving the accuracy of single-data calibration.

[0077] This embodiment improves the Centerpoint network for target recognition, giving it more feature scales and incorporating the ideas of ResNet. Combined with CENet, it efficiently identifies and removes dynamic targets and ground in the environment, degrading complex dynamic environments into simple static environments and eliminating environmental interference. Utilizing the synchronization of multiple sensors, it fuses data from multiple sensors, removing the influence of abnormal sensor data and improving the accuracy of single-data calibration. Furthermore, it employs a kinematic-based annotation method, eliminating the need for calibration boards or specific sites and eliminating interference from complex environments. Autonomous vehicles can complete the calibration of radar, depth cameras, and ADPU simply by driving normally. It is applicable to calibration in any complex scenario where driving is possible at any time, reducing calibration costs and increasing calibration flexibility.

[0078] Figure 3 This is a schematic flowchart of a vehicle data calibration system according to an embodiment of the present invention. Figure 3 As shown, in a specific embodiment, the vehicle data calibration system 100 includes a control module 10. The control module 10 includes a memory 11 and a processor 12. The memory 11 stores a calculation program, which, when executed by the processor, is used to implement the aforementioned data calibration method. The processor 12 can be a central processing unit (CPU), a digital processing unit, etc. The processor 12 sends and receives data via a communication interface. The memory 11 is used to store the program executed by the processor 12. The memory 11 can be any medium capable of carrying or storing desired program code in the form of instructions or data structures, and accessible by a computer; it can also be a combination of multiple memories 11. The aforementioned calculation program can be downloaded from a computer-readable storage medium to a corresponding computing / processing device or downloaded to a computer or external storage device via a network (e.g., the Internet, a local area network, a wide area network, and / or a wireless network). Here, the control module 10 can be a vehicle body controller.

[0079] For the purposes of this embodiment, a "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. More specific examples of computer-readable media (a non-exhaustive list) include: an electrical connection (electronic device) having one or more wires, a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, a computer-readable medium can even be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory 11.

[0080] This embodiment also provides a vehicle, which includes the data calibration system 100 described above. Details of the data calibration system 100 are omitted here. In this embodiment, the data calibration system 100 further includes a radar, a depth camera, and an autonomous driving positioning unit. The radar, depth camera, and autonomous driving positioning unit are all mounted on the vehicle. The radar and depth camera are used to acquire initial point cloud data for vehicle-to-everything (V2X) communication while the vehicle is running, and the autonomous driving positioning unit is used to acquire the vehicle's location information.

[0081] Therefore, those skilled in the art should recognize that although numerous exemplary embodiments of the present invention have been shown and described in detail herein, many other variations or modifications conforming to the principles of the present invention can be directly determined or derived from the disclosure of the present invention without departing from the spirit and scope of the invention. Thus, the scope of the present invention should be understood and construed as covering all such other variations or modifications.

Claims

1. A method for calibrating vehicle data, characterized in that, Includes the following steps: The initial point cloud data of the vehicle is acquired using the vehicle's first sensor while the vehicle is in motion. The initial point cloud data is voxelized, and intermediate features are obtained through a sparse convolutional network. Construct a residual neural network; The intermediate features are extracted based on the residual neural network to obtain the final features; The final features are fused to obtain the processed target point cloud data; Obstacles that are semantically dynamic obstacles and ground point cloud data are removed from the target point cloud data using a semantic segmentation model or RANSAC algorithm; The target point cloud data at the current moment and the target point cloud data at the next moment are matched to obtain the transformation relationship of the first sensor in a pre-established coordinate system.

2. The data calibration method according to claim 1, characterized in that, It also includes the following steps: The vehicle's position information is obtained using the vehicle's second sensor while the vehicle is in motion; The transformation relationship of the second sensor in the coordinate system is obtained based on the position information at the current moment and the position information at the next moment.

3. The data calibration method according to claim 2, characterized in that, The step of obtaining the transformation relationship of the second sensor in the coordinate system based on the position information at the current moment and the position information at the next moment, further includes the following steps: Abnormal data in the transformation relationships of the first sensor and the second sensor are deleted based on the transformation relationships of the first sensor and the second sensor.

4. The data calibration method according to claim 3, characterized in that, The first sensor includes radar and a depth camera, and the second sensor includes an autonomous driving positioning unit.

5. The data calibration method according to claim 4, characterized in that, The step of deleting abnormal data in the transformation relationships of the first sensor and the second sensor based on the transformation relationships of the second sensor specifically includes the following steps: The rotation matrices in the transformation relationship of the radar, the transformation relationship of the depth camera, and the transformation relationship of the autonomous driving positioning unit are respectively converted into Euler angles; The first variance value is calculated based on the Euler angles of the radar, the Euler angles of the depth camera, and the Euler angles of the autonomous driving positioning unit; When the first variance value is greater than the first preset value, delete the rotation matrix in the transformation relationship of the radar, the rotation matrix in the transformation relationship of the depth camera, and the rotation matrix in the transformation relationship of the autonomous driving positioning unit.

6. The data calibration method according to claim 5, characterized in that, The step of obtaining the transformation relationship of the second sensor in the coordinate system based on the position information at the current moment and the position information at the next moment, further includes the following steps: The displacement distance is calculated based on the translation matrix in the transformation relationship of the radar, the rotation matrix in the transformation relationship of the depth camera, and the translation matrix in the transformation relationship of the autonomous driving positioning unit, respectively. The second variance value is calculated based on the displacement distance of the radar, the displacement distance of the depth camera, and the displacement distance of the autonomous driving positioning unit; When the second variance value is greater than the second preset value, delete the translation matrix in the transformation relationship of the radar, the translation matrix in the transformation relationship of the depth camera, and the translation matrix in the transformation relationship of the autonomous driving positioning unit.

7. A vehicle data calibration system, characterized in that, include: A control module, comprising a memory and a processor, wherein the memory stores a calculation program, which, when executed by the processor, is used to implement the data calibration method according to any one of claims 1-6.

8. A vehicle, characterized in that, Includes the data calibration system as described in claim 7.

Images