A data calibration method and device, electronic equipment and storage medium

Abstract

The application provides a data calibration method and device, electronic equipment and storage medium. The first coordinate data of the point cloud data collected in a preset sampling time unit in a radar coordinate system and the second coordinate data of the image data in a camera coordinate system are determined, the pose data group collected by the positioning device in the preset sampling time unit is subjected to interpolation processing, the reference coordinate conversion data is obtained, the first coordinate data is subjected to coordinate conversion processing based on the candidate calibration data and the reference coordinate conversion data, the third coordinate data of the point cloud data in the camera coordinate system is obtained, the candidate calibration data is adjusted according to the second coordinate data and the third coordinate data, and the adjustment is ended until the end condition is met, and the target calibration data is obtained. The application embodiment can reduce the problem that the subsequent point cloud data and the image data cannot be coincided due to the time matching error, and can improve the accuracy of the calibration data.

Description

Technical Field

[0001] This invention relates to the field of intelligent driving technology, and in particular to a data calibration method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the maturation of camera imaging technology and the improvement of the perception capabilities of various sensors, the number of cameras and sensors installed in autonomous vehicles has been increasing in recent years. Autonomous vehicles perceive environmental information around the vehicle through cameras and sensors and fuse the perceived environmental information. However, due to errors in the acquisition time between cameras, between cameras and sensors, and between sensors, the environmental information perceived by the cameras and sensors cannot be matched. Summary of the Invention

[0003] To address the problem of mismatch between environmental information sensed by existing cameras and sensors, this application provides a data calibration method, apparatus, device, and medium:

[0004] According to a first aspect of this application, a data annotation method is provided, comprising:

[0005] Acquire point cloud data collected by radar and image data collected by vehicle camera within a preset sampling time unit, and determine the first coordinate data of point cloud data in radar coordinate system and the second coordinate data of image data in camera coordinate system;

[0006] The pose data set collected by the positioning device within a preset sampling time unit is acquired, and the pose data set is interpolated to obtain reference coordinate transformation data; the reference coordinate transformation data represents the coordinate transformation relationship between the vehicle coordinate system and the world coordinate system;

[0007] Based on candidate calibration data and reference coordinate transformation data, coordinate transformation processing is performed on the first coordinate data to obtain the third coordinate data of the point cloud data in the camera coordinate system; the candidate calibration data represents the coordinate transformation relationship between the radar coordinate system and the vehicle coordinate system, as well as the coordinate transformation relationship between the vehicle coordinate system and the camera coordinate system;

[0008] The candidate calibration data are adjusted based on the second and third coordinate data until the termination condition is met, thus obtaining the target calibration data.

[0009] According to a second aspect of this application, a data calibration apparatus is provided, comprising:

[0010] The first acquisition module is used to acquire point cloud data collected by the radar and image data collected by the vehicle camera within a preset sampling time unit, and to determine the first coordinate data of the point cloud data in the radar coordinate system and the second coordinate data of the image data in the camera coordinate system.

[0011] The second acquisition module is used to acquire the pose data set collected by the positioning device within a preset sampling time unit, and to perform interpolation processing on the pose data set to obtain reference coordinate transformation data; the reference coordinate transformation data represents the coordinate transformation relationship between the vehicle coordinate system and the world coordinate system.

[0012] The coordinate transformation module is used to perform coordinate transformation processing on the first coordinate data based on the candidate calibration data and the reference coordinate transformation data to obtain the third coordinate data of the point cloud data in the camera coordinate system; the candidate calibration data represents the coordinate transformation relationship between the radar coordinate system and the vehicle coordinate system, as well as the coordinate transformation relationship between the vehicle coordinate system and the camera coordinate system;

[0013] The adjustment module is used to adjust the candidate calibration data based on the second and third coordinate data until the termination condition is met, thereby obtaining the target calibration data.

[0014] On the other hand, the first acquisition module is used to acquire point cloud data collected by the radar at the first sampling time and image data collected by the vehicle-mounted camera at the second sampling time.

[0015] The first sampling time and the second sampling time are both within a preset sampling time unit, and the time difference between the first sampling time and the second sampling time is within a preset time difference interval.

[0016] On the other hand, the second acquisition module is used to acquire the first pose data set collected by the positioning device within the first preset sampling time unit corresponding to the first sampling time and the second pose data set collected within the second preset sampling time unit corresponding to the second sampling time; the first preset sampling time unit and the second preset sampling time unit are within the preset sampling time unit;

[0017] Interpolation processing is performed on the first pose data group and the second pose data group respectively to determine the first reference coordinate transformation data corresponding to the first pose data group and the second reference coordinate transformation data corresponding to the second pose data group, thus obtaining the reference coordinate transformation data.

[0018] On the other hand, the second acquisition module is used to acquire the first pose data group collected by the positioning device within the first preset sampling time unit corresponding to the first sampling time; the first preset sampling time unit is within the preset sampling time unit;

[0019] Interpolate the first pose data set to obtain the first reference coordinate transformation data corresponding to the first pose data set;

[0020] The first reference coordinate transformation data is inverted to obtain the second reference coordinate transformation data corresponding to the first pose data group;

[0021] The first and second reference coordinate transformation data are integrated to obtain the reference coordinate transformation data.

[0022] On the other hand, the candidate calibration data includes first candidate calibration data and second candidate calibration data;

[0023] The first candidate calibration data represents the coordinate transformation relationship between the radar coordinate system and the vehicle coordinate system, while the second candidate calibration data represents the coordinate transformation relationship between the vehicle coordinate system and the camera coordinate system.

[0024] The coordinate transformation module is used to perform coordinate transformation processing on the first coordinate data based on the first candidate calibration data, the first reference coordinate transformation data, the second reference coordinate transformation data, and the second candidate calibration data, to obtain the third coordinate data of the point cloud data in the camera coordinate system.

[0025] On the other hand, the adjustment module is used to adjust the candidate calibration data according to the second coordinate data and the third coordinate data until the difference between the second coordinate data and the third coordinate data is within the preset difference range, and the candidate calibration data at the end of the adjustment is used as the target calibration data.

[0026] According to a third aspect of this application, an electronic device is provided, comprising a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by the processor to implement the data calibration method of the first aspect of this application.

[0027] According to a fourth aspect of this application, a computer storage medium is provided, which stores at least one instruction or at least one program, wherein the at least one instruction or at least one program is loaded and executed by a processor to implement the data calibration method of the first aspect of this application.

[0028] According to a fifth aspect of this application, a computer program product is provided, comprising at least one instruction or at least one program segment, wherein the at least one instruction or at least one program segment is loaded and executed by a processor to implement the data calibration method of the first aspect of this application.

[0029] The data calibration method, apparatus, electronic device, and storage medium provided in this application have the following technical effects:

[0030] By acquiring point cloud data collected by radar and image data collected by vehicle-mounted camera within a preset sampling time unit, the first coordinate data of the point cloud data in the radar coordinate system and the second coordinate data of the image data in the camera coordinate system are determined. The pose data set collected by positioning device within the preset sampling time unit is acquired, and interpolation processing is performed on the pose data set to obtain reference coordinate transformation data. The reference coordinate transformation data represents the coordinate transformation relationship between the vehicle coordinate system and the world coordinate system. Based on candidate calibration data and reference coordinate transformation data, coordinate transformation processing is performed on the first coordinate data to obtain the third coordinate data of the point cloud data in the camera coordinate system. The candidate calibration data represents the coordinate transformation relationship between the radar coordinate system and the vehicle coordinate system, as well as the coordinate transformation relationship between the vehicle coordinate system and the camera coordinate system. The candidate calibration data is adjusted according to the second and third coordinate data until the termination condition is met, thus obtaining the target calibration data. Based on the embodiments of this application, by changing the pose determination method and interpolating the pose data set to obtain reference coordinate transformation data, the problem of subsequent point cloud data and image data not coinciding due to time matching errors can be reduced, thereby improving the accuracy of the calibration data. Attached Figure Description

[0031] To more clearly illustrate the technical solutions and advantages in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 This is a schematic diagram of an application environment provided in an embodiment of this application;

[0033] Figure 2 This is an installation diagram of a data acquisition device;

[0034] Figure 3 It is a set of schematic diagrams of a visual interface for annotation;

[0035] Figure 4 This is a schematic diagram of a mapping link between point clouds and images;

[0036] Figure 5 This is a schematic diagram of time matching of multiple sensors;

[0037] Figure 6 This is a schematic diagram illustrating the soft time synchronization between a lidar and a vehicle-mounted camera.

[0038] Figure 7 This is a flowchart illustrating a data calibration method provided in an embodiment of this application;

[0039] Figure 8 This is a visual schematic diagram of an adjustable interface provided in an embodiment of this application;

[0040] Figure 9 This is a comparative diagram showing the candidate calibration data before and after adjustment, provided in an embodiment of this application.

[0041] Figure 10 This is a schematic diagram of the structure of a data calibration device provided in an embodiment of this application;

[0042] Figure 11 This is a schematic diagram of the hardware structure of an electronic device for implementing the data calibration method provided in this application embodiment. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely one embodiment of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0044] The term "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of this application. In the description of the embodiments of this application, it should be understood that the terms "first," "second," and "third," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first," "second," and "third," etc., may explicitly or implicitly include one or more of that feature. Furthermore, the terms "first," "second," and "third," etc., are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. In addition, the terms "comprising," "having," and "being," and any variations thereof, are intended to cover non-exclusive inclusion.

[0045] It is understood that in the specific implementation of this application, data such as indicator data are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0046] The embodiments of the present invention can be applied to various scenarios, including but not limited to intelligent transportation and intelligent vehicle-road cooperative systems.

[0047] Intelligent Traffic System (ITS), also known as Intelligent Transportation System, effectively integrates advanced science and technology (information technology, computer technology, data communication technology, sensor technology, electronic control technology, automatic control theory, operations research, artificial intelligence, etc.) into transportation, service control, and vehicle manufacturing. It strengthens the connection between vehicles, roads, and users, thereby forming a comprehensive transportation system that ensures safety, improves efficiency, improves the environment, and saves energy.

[0048] Intelligent Vehicle Infrastructure Cooperative Systems (IVICS) are a development direction of Intelligent Transportation Systems (ITS). IVICS utilizes advanced wireless communication and next-generation Internet technologies to implement comprehensive, real-time dynamic information exchange between vehicles and infrastructure. Based on the collection and fusion of dynamic traffic information across all times and spaces, it conducts active vehicle safety control and cooperative road management, fully realizing effective collaboration between people, vehicles, and roads. This ensures traffic safety, improves traffic efficiency, and ultimately forms a safe, efficient, and environmentally friendly road traffic system.

[0049] Please see Figure 1 , Figure 1 This is a schematic diagram of an application environment provided in an embodiment of this application. This application environment may include a vehicle-mounted camera 10, a radar 20, a positioning device 30, and a server 40. The vehicle-mounted camera 10, radar 20, positioning device 30, and server 20 can be directly or indirectly connected via wired or wireless communication.

[0050] In some possible implementations, the vehicle-mounted camera 10, radar 20, and positioning device 30 can send image data, point cloud data, and pose data to the server 40, respectively, and the server 40 can provide data annotation services.

[0051] Server 40 can be a standalone physical server, a service cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Server 40 may include network communication units, processors, and memory, etc.

[0052] Figure 2This is an installation diagram of a data acquisition device, including 6 vehicle-mounted cameras, 1 LiDAR (Laser Radar), and 5 radar radars. For each data acquisition scene, only a portion of the keyframes can be selected for annotation. The annotation rate can be 2Hz, and a total of 23 categories of objects' 3D bounding boxes, category information, and other attribute information, such as visibility and status, can be annotated. Figure 3 This is a schematic diagram of a set of annotation visualization interfaces. In the annotation process, the same object presented in the image and point cloud at the same time is required to be annotated simultaneously to form a set of data. Usually, the 3D BOX of the object in the point cloud is manually annotated, and then the mapping relationship between the 3D BOX and the image in each viewpoint is established through a mapping matrix. This mapping relationship can be determined manually through calibration.

[0053] Existing radar and image joint annotation schemes require establishing a mapping relationship between any point in the point cloud and any pixel in the image. Figure 4 This is a schematic diagram of a mapping link between point clouds and images. There are two main types of links that map the coordinates of points in a point cloud to the image in an image. One method directly maps the point cloud at time T1 into the image captured by the vehicle-mounted camera at time T2 using a rigid body transformation matrix. This method requires T1 and T2 to be sufficiently close, but due to the low frequencies of the vehicle-mounted camera and radar, its matching accuracy is only 25ms, resulting in a large mapping error. The other method maps the point cloud data acquired at time T1 to the vehicle coordinate system using a transformation matrix between the radar coordinate system and the vehicle coordinate system. Then, based on the vehicle pose at time T1', the point cloud data is transformed to the world coordinate system, and then again based on the vehicle pose at time T2'. Finally, it is mapped to the camera coordinate system using a transformation matrix between the vehicle coordinate system and the camera coordinate system. This method requires T1' and T1, and T2' and T2 to be sufficiently close, but its matching accuracy is only 10ms, and mapping errors still exist.

[0054] However, existing joint annotation of radar and images faces the problem of time matching between multiple devices. Figure 5 This is a schematic diagram of time matching across multiple sensors, including a combined signal timestamp distribution from six vehicle-mounted cameras, one point cloud dataset, and one pose dataset. Figure 5 As can be seen, there is a misalignment of more than 10ms, which will cause the labeled objects to be unable to be accurately matched.

[0055] Multi-sensor fusion involves time synchronization, including hard synchronization and soft synchronization. Soft synchronization primarily utilizes timestamps to match data from different sensors. Typically, data from each sensor is unified to the sensor with the longer scanning period and lower frequency. For example, for a 4-line LiDAR with a sampling frequency of approximately 12.5Hz and an automotive camera with a sampling frequency of 30Hz, the LiDAR's sampling period is obviously longer, so matching can be based on the LiDAR's sampling frequency. Figure 6 This is a schematic diagram illustrating the soft synchronization of time between a LiDAR and an onboard camera. The upper dashed box represents the sampling frequency of the LiDAR, and the lower dashed box represents the sampling frequency of the onboard camera. The horizontal axis represents a unified timestamp. Each sampling moment of the sensor can be recorded on a unified time series. When the LiDAR completes a sampling, it can find the image closest to its sampling moment, thus achieving time matching between the two types of data. Figure 6 The shaded area is shown in the diagram. However, if the time interval between the sampling time of a LiDAR sample and the sampling time of the image closest to its sampling time is large, the final synchronization effect will still be poor. Furthermore, as the amount of time-synchronized data increases, the difficulty of matching inevitably increases, and the matching accuracy also decreases significantly. Hard synchronization refers to using dedicated hardware to connect all sensors, sending pulses to all sensors simultaneously, and receiving their respective timestamps. Triggering sensors through hardware, such as triggering vehicle cameras, ensures that multiple vehicle cameras trigger almost simultaneously (the time from receiving the pulse to completing the sampling process under the same exposure time), and assigns a hardware-based timestamp to the sampling results of the vehicle cameras. While hard synchronization can reduce the timestamp differences between different sensors, it has high hardware requirements, increasing vehicle manufacturing costs.

[0056] The following describes a specific embodiment of a data calibration method according to this application. Figure 7 This is a flowchart illustrating a data calibration method provided in an embodiment of this application. This specification provides method operation steps as shown in the embodiments or flowcharts, but based on conventional or non-inventive labor, more or fewer operation steps may be included. The order of steps listed in the embodiments is merely one of many execution orders and does not represent the only execution order. In actual execution, the method can be executed in the order shown in the embodiments or drawings, or in parallel (e.g., in a parallel processor or multi-threaded processing environment).

[0057] Specifically, such as Figure 7 As shown, the data calibration method may include:

[0058] S701: Acquire point cloud data collected by the radar and image data collected by the vehicle camera within a preset sampling time unit, and determine the first coordinate data of the point cloud data in the radar coordinate system and the second coordinate data of the image data in the camera coordinate system.

[0059] In this embodiment, point cloud data collected by radar at the first sampling time and image data collected by vehicle-mounted camera at the second sampling time can be acquired. Both the first and second sampling times are within a preset sampling time unit, and the time difference between the first and second sampling times is within a preset time difference interval.

[0060] In some possible implementations, it is assumed that soft and hard synchronization are ineffective, the sampling frequency of the vehicle-mounted camera is 30Hz, the sampling frequency of the LiDAR is 10Hz, and the positioning device may include a high-precision inertial measurement unit (IMU) and a real-time kinematic (RTK) unit with a sampling frequency of 100Hz. The preset sampling time unit can be the length of time during which the vehicle-mounted camera, LiDAR, and positioning device can acquire corresponding data based on their respective sampling frequencies. For example, point cloud data can be acquired at sampling time T1, and image data can be acquired at sampling time T2. Then, the coordinates of each point in the point cloud data in the radar coordinate system corresponding to the LiDAR can be determined to obtain the first coordinate data. Simultaneously, the coordinates of each pixel in the image data in the camera coordinate system can be determined to obtain the second coordinate data.

[0061] S703: Obtain the pose data set collected by the positioning device within the preset sampling time unit, perform interpolation processing on the pose data set, and obtain reference coordinate transformation data.

[0062] In this embodiment, the reference coordinate transformation data can characterize the coordinate transformation relationship between the vehicle coordinate system and the world coordinate system.

[0063] In this embodiment, the positioning device can acquire a first pose data set collected within a first preset sampling time unit corresponding to the first sampling time, and a second pose data set collected within a second preset sampling time unit corresponding to the second sampling time. Then, interpolation processing can be performed on the first pose data set to determine the first reference coordinate transformation data corresponding to the first pose data set, and interpolation processing can be performed on the second pose data set to determine the second reference coordinate transformation data corresponding to the second pose data set, thus obtaining the reference coordinate transformation data.

[0064] In the embodiments of the present application, the first pose data group collected by the positioning device within the first preset sampling time unit corresponding to the first sampling moment can be obtained. Then, interpolation processing can be performed on the first pose data group to obtain the first reference coordinate conversion data corresponding to the first pose data group. Then, inverse processing can be performed on the first reference coordinate conversion data to obtain the second reference coordinate conversion data corresponding to the first pose data group. By changing the determination method of the pose and performing interpolation processing on the pose data group to obtain the reference coordinate conversion data, the problem that the subsequent point cloud data and image data cannot coincide caused by time matching errors can be reduced, and the accuracy of the calibration data can be improved.

[0065] In some possible implementation manners, the first pose data group can be collected at sampling moments T1' and T1", and the first pose data group can also be collected at sampling moments T2' and T2". Among them, T1' < T1 < T1", and T2' < T2 < T2". Then, interpolation processing can be performed on the first pose data group collected at sampling moment T1' and sampling moment T1" to obtain the first reference coordinate conversion data Tx_I_M, and interpolation processing can be performed on the second pose data group collected at sampling moment T2' and sampling moment T2" to obtain the second reference coordinate conversion data Tx_M_I.

[0066] In some possible implementation manners, interpolation processing can be performed on the first pose data group collected at sampling moment T1' and sampling moment T1" to obtain the first reference coordinate conversion data Tx_I_M, and then inverse processing can be performed on the first reference coordinate conversion data Tx_I_M to obtain the second reference coordinate conversion data Tx_M_I.

[0067] S705: Perform coordinate conversion processing on the first coordinate data based on the candidate calibration data and the reference coordinate conversion data to obtain the third coordinate data of the point cloud data in the camera coordinate system.

[0068] In the embodiments of the present application, the candidate calibration data represents the coordinate conversion relationship between the radar coordinate system and the vehicle body coordinate system and the coordinate conversion relationship between the vehicle body coordinate system and the camera coordinate system.

[0069] In the embodiments of the present application, the candidate calibration data can include the first candidate calibration data and the second candidate calibration data. The first candidate calibration data can represent the coordinate conversion relationship between the radar coordinate system and the vehicle body coordinate system, and the second candidate calibration data can represent the coordinate conversion relationship between the vehicle body coordinate system and the camera coordinate system.

[0070] In the embodiments of the present application, coordinate conversion processing can be performed on the first coordinate data based on the first candidate calibration data, the first reference coordinate conversion data, the second reference coordinate conversion data, and the second candidate calibration data to obtain the third coordinate data of the point cloud data in the camera coordinate system.

[0071] In some possible implementations, the first coordinate data P_L can be transformed based on the parameterized first candidate calibration data Tx_L_I, which represents the coordinate transformation relationship between the radar coordinate system and the vehicle coordinate system, to obtain the coordinates of the point cloud in the vehicle coordinate system. Then, the coordinates of the point cloud in the vehicle coordinate system can be transformed based on the first reference coordinate transformation data Tx_I_M, which represents the coordinate transformation relationship between the vehicle coordinate system and the world coordinate system, to obtain the coordinates P_M of the point cloud in the world coordinate system. Next, the coordinates of the point cloud in the world coordinate system can be transformed based on the second reference coordinate transformation data Tx_M_I, which represents the coordinate transformation relationship between the world coordinate system and the vehicle coordinate system, to obtain the coordinates of the point cloud in the vehicle coordinate system. Finally, the coordinates of the point cloud in the vehicle coordinate system can be transformed based on the parameterized second candidate calibration data Tx_I_C, which represents the coordinate transformation relationship between the vehicle coordinate system and the camera coordinate system, to obtain the coordinates P_C of the point cloud in the camera coordinate system, i.e., the third coordinate data. The specific coordinate transformation process is as follows:

[0072] P_M=Tx_I_M●Tx_L_I●P_L

[0073] Wherein, P_M can represent the coordinates of the point cloud in the world coordinate system, P_L can represent the coordinates of the point cloud in the radar coordinate system, Tx_I_M can represent the coordinate transformation relationship between the vehicle coordinate system and the world coordinate system at sampling time T1, and Tx_L_I can represent the coordinate transformation relationship between the radar coordinate system and the vehicle coordinate system.

[0074] P_C=Tx_I_C●Tx_M_I●P_M

[0075] Wherein, P_C can represent the coordinates of the point cloud in the camera coordinate system, P_M can represent the coordinates of the point cloud in the world coordinate system, Tx_M_I can represent the coordinate transformation relationship between the world coordinate system and the vehicle coordinate system at sampling time T2, and Tx_I_C can represent the coordinate transformation relationship between the vehicle coordinate system and the camera coordinate system.

[0076] According to the associative law of matrices, P_C = Tx_I_C●Tx_M_I●Tx_I_M●Tx_L_I●P_L

[0077] Let Tx=Tx_I_C●Tx_M_I●Tx_I_M●Tx_L_I

[0078] Then P_C = Tx●P_L.

[0079] S707: Adjust the candidate calibration data based on the second and third coordinate data until the termination condition is met, and then end the adjustment to obtain the target calibration data.

[0080] The existing manual calibration process for a single radar mainly involves finding a large, flat location, placing a regular obstacle directly in front of the radar, converting and displaying the calibrated point cloud, and manually adjusting parameters such as pitch, yaw, roll, and X-axis and Y-axis displacements by observing the calibrated point cloud. However, the calibration results are often inaccurate.

[0081] In this embodiment, candidate calibration data can be adjusted based on second and third coordinate data until the difference between the second and third coordinate data falls within a preset difference range. The candidate calibration data at the end of the adjustment can be used as the target calibration data. During the adjustment process, for static objects, their positions in the world coordinate system are theoretically consistent regardless of when they are collected. If two moments are sufficiently close, it can be assumed that all objects have no displacement at those two moments.

[0082] Figure 8 This is a visual schematic diagram of an adjustable interface provided in an embodiment of this application. Figure 9 This is a comparative diagram showing the candidate calibration data before and after adjustment, as provided in an embodiment of this application. Figure 9 It can be clearly seen that the adjusted point cloud data can overlap with the image data.

[0083] By adjusting the values ​​of the first and second candidate calibration data based on a visual interface, point cloud data with a sampling time difference within a preset time difference range can be mapped to the corresponding image data. This can reduce the problem of subsequent point cloud data and image data not coinciding due to time matching errors and improve the accuracy of calibration data.

[0084] The data calibration method provided in this application, by changing the way the pose is determined and interpolating the pose data group to obtain reference coordinate transformation data, can reduce the problem of subsequent point cloud data and image data not coinciding due to time matching errors, thereby improving the accuracy of the calibration data. By adjusting the values ​​of the first candidate calibration data and the second candidate calibration data based on a visual interface, point cloud data with a sampling time difference within a preset time difference interval can be mapped to the corresponding image data, which can reduce the problem of subsequent point cloud data and image data not coinciding due to time matching errors, thereby improving the accuracy of the calibration data.

[0085] This application also provides a data calibration device. Figure 10 This is a schematic diagram of the structure of a data calibration device provided in an embodiment of this application, as shown below.Figure 10 As shown, the data calibration device may include:

[0086] The first acquisition module 1001 is used to acquire point cloud data collected by the radar and image data collected by the vehicle camera within a preset sampling time unit, and to determine the first coordinate data of the point cloud data in the radar coordinate system and the second coordinate data of the image data in the camera coordinate system.

[0087] The second acquisition module 1003 is used to acquire the pose data set collected by the positioning device within a preset sampling time unit, and to perform interpolation processing on the pose data set to obtain reference coordinate transformation data; the reference coordinate transformation data represents the coordinate transformation relationship between the vehicle coordinate system and the world coordinate system.

[0088] The coordinate transformation module 1005 is used to perform coordinate transformation processing on the first coordinate data based on the candidate calibration data and the reference coordinate transformation data to obtain the third coordinate data of the point cloud data in the camera coordinate system; the candidate calibration data represents the coordinate transformation relationship between the radar coordinate system and the vehicle coordinate system, as well as the coordinate transformation relationship between the vehicle coordinate system and the camera coordinate system.

[0089] The adjustment module 1007 is used to adjust the candidate calibration data according to the second coordinate data and the third coordinate data until the termination condition is met to end the adjustment and obtain the target calibration data.

[0090] In some possible implementations, the first acquisition module is used to acquire point cloud data collected by the radar at the first sampling time and image data collected by the vehicle-mounted camera at the second sampling time.

[0091] The first sampling time and the second sampling time are both within a preset sampling time unit, and the time difference between the first sampling time and the second sampling time is within a preset time difference interval.

[0092] In some possible implementations, the second acquisition module is used to acquire the first pose data set collected by the positioning device within the first preset sampling time unit corresponding to the first sampling time and the second pose data set collected within the second preset sampling time unit corresponding to the second sampling time; the first preset sampling time unit and the second preset sampling time unit are within the preset sampling time unit.

[0093] Interpolation processing is performed on the first pose data group and the second pose data group respectively to determine the first reference coordinate transformation data corresponding to the first pose data group and the second reference coordinate transformation data corresponding to the second pose data group, thus obtaining the reference coordinate transformation data.

[0094] In some possible implementations, the second acquisition module is used to acquire the first pose data group collected by the positioning device within a first preset sampling time unit corresponding to the first sampling time; the first preset sampling time unit is within the preset sampling time unit;

[0095] Interpolate the first pose data set to obtain the first reference coordinate transformation data corresponding to the first pose data set;

[0096] The first reference coordinate transformation data is inverted to obtain the second reference coordinate transformation data corresponding to the first pose data group;

[0097] The first and second reference coordinate transformation data are integrated to obtain the reference coordinate transformation data.

[0098] In some possible implementations, the candidate calibration data includes first candidate calibration data and second candidate calibration data;

[0099] The first candidate calibration data represents the coordinate transformation relationship between the radar coordinate system and the vehicle coordinate system, while the second candidate calibration data represents the coordinate transformation relationship between the vehicle coordinate system and the camera coordinate system.

[0100] The coordinate transformation module is used to perform coordinate transformation processing on the first coordinate data based on the first candidate calibration data, the first reference coordinate transformation data, the second reference coordinate transformation data, and the second candidate calibration data, to obtain the third coordinate data of the point cloud data in the camera coordinate system.

[0101] In some possible implementations, the adjustment module is used to adjust the candidate calibration data according to the second coordinate data and the third coordinate data until the difference between the second coordinate data and the third coordinate data is within a preset difference range, and the candidate calibration data at the end of the adjustment is used as the target calibration data.

[0102] The apparatus and method embodiments in this application are based on the same application concept.

[0103] This application provides an electronic device including a processor and a memory. The memory stores at least one instruction or at least one program segment, which is loaded and executed by the processor to implement the data calibration method provided in the above method embodiments.

[0104] Figure 11 This is a schematic diagram of the hardware structure of an electronic device for implementing the data calibration method provided in this application embodiment. The electronic device can participate in or include the data calibration device provided in this application embodiment. Figure 11As shown, the electronic device may include one or more processors 1101 (shown as 1101a and 1101b in the figure) 1101 (processor 1101 may include, but is not limited to, a microprocessor 1101 MCU or a programmable logic device FPGA, etc.), a memory 1103 for storing data, and a transmission device 1105 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, and / or a power supply. Those skilled in the art will understand that... Figure 11 The structure shown is for illustrative purposes only and does not limit the structure of the electronic device described above. For example, the electronic device may also include components that are more... Figure 11 The more or fewer components shown, or having the same Figure 11 The different configurations shown.

[0105] It should be noted that the aforementioned one or more processors 1101 and / or other data processing circuits are generally referred to as "data processing circuits" in this application. The data processing circuit can be embodied, in whole or in part, in software, hardware, firmware, or any other combination. Furthermore, the data processing circuit can be a single, independent processing module, or it can be integrated, in whole or in part, into any other element within an electronic device (or mobile device). As involved in the embodiments of this application, the data processing circuit acts as a processor 1101 control (e.g., selection of a variable resistor termination path connected to an interface).

[0106] The memory 1103 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the data calibration method in this embodiment. The processor 1101 implements the above-mentioned data calibration method by running the software programs and modules stored in the memory 1103 and executing various functional applications and data processing. The memory 1103 may include high-speed random access memory, and may also include non-volatile random access memory 1103, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory 1103. In some possible embodiments, the memory 1103 may further include remotely configured memory 1103 relative to the processing, which can be connected to electronic devices via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0107] The transmission device 1105 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the electronic device. In one example, the transmission device 1105 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 1105 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0108] The display can be, for example, a touchscreen liquid crystal display (LED), which allows users to interact with the user interface of an electronic device (or mobile device).

[0109] This application provides a computer-readable storage medium that can be disposed in an electronic device to store at least one instruction or at least one program related to implementing a data calibration method in the method embodiment. The at least one instruction or at least one program is loaded and executed by the processor to implement the data calibration method provided in the above method embodiment.

[0110] Optionally, in this embodiment, the storage medium may be located at at least one of the multiple network servers in a computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0111] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, while this specification describes specific embodiments, other embodiments are also within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in the order shown in different embodiments and still achieve the desired results. Additionally, the processes depicted in the drawings do not necessarily require a specific order or sequence of connections to achieve the desired results; in some implementations, parallel processing of multiple tasks is possible or may be advantageous.

[0112] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, embodiments of apparatus and electronic devices are described simply because they are based on similar method embodiments; relevant parts can be referred to the descriptions of the method embodiments.

[0113] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A data calibration method, characterized in that, include: Acquire point cloud data collected by radar and image data collected by vehicle camera within a preset sampling time unit, and determine the first coordinate data of the point cloud data in the radar coordinate system and the second coordinate data of the image data in the camera coordinate system; The pose data set collected by the positioning device within the preset sampling time unit is acquired, and the pose data set is interpolated to obtain reference coordinate transformation data; the reference coordinate transformation data represents the coordinate transformation relationship between the vehicle coordinate system and the world coordinate system. Based on the candidate calibration data and the reference coordinate transformation data, the first coordinate data is transformed to obtain the third coordinate data of the point cloud data in the camera coordinate system. The candidate calibration data characterizes the coordinate transformation relationship between the radar coordinate system and the vehicle coordinate system, as well as the coordinate transformation relationship between the vehicle coordinate system and the camera coordinate system; The candidate calibration data is adjusted based on the second coordinate data and the third coordinate data until the termination condition is met, thus obtaining the target calibration data.

2. The method according to claim 1, characterized in that, The acquisition of point cloud data collected by radar and image data collected by vehicle-mounted camera within a preset sampling time unit includes: The point cloud data collected by the radar at the first sampling time and the image data collected by the vehicle-mounted camera at the second sampling time are acquired. Wherein, both the first sampling time and the second sampling time are within the preset sampling time unit, and the time difference between the first sampling time and the second sampling time is within the preset time difference interval.

3. The method according to claim 2, characterized in that, The step of acquiring the pose data set collected by the positioning device within the preset sampling time unit, and performing interpolation processing on the pose data set to obtain reference coordinate transformation data includes: The positioning device acquires a first pose data set collected within a first preset sampling time unit corresponding to the first sampling time and a second pose data set collected within a second preset sampling time unit corresponding to the second sampling time; the first preset sampling time unit and the second preset sampling time unit are within the preset sampling time unit. Interpolation processing is performed on the first pose data group and the second pose data group respectively to determine the first reference coordinate transformation data corresponding to the first pose data group and the second reference coordinate transformation data corresponding to the second pose data group, thereby obtaining the reference coordinate transformation data.

4. The method according to claim 2, characterized in that, The step of acquiring the pose data set collected by the positioning device within the preset sampling time unit, and performing interpolation processing on the pose data set to obtain reference coordinate transformation data includes: The positioning device acquires the first pose data group collected within a first preset sampling time unit corresponding to the first sampling time; the first preset sampling time unit is within the preset sampling time unit. Interpolation processing is performed on the first pose data group to obtain the first reference coordinate transformation data corresponding to the first pose data group; The first reference coordinate transformation data is inverted to obtain the second reference coordinate transformation data corresponding to the first pose data group. The first reference coordinate transformation data and the second reference coordinate transformation data are integrated to obtain the reference coordinate transformation data.

5. The method according to claim 3 or 4, characterized in that, The candidate calibration data includes first candidate calibration data and second candidate calibration data; The first candidate calibration data represents the coordinate transformation relationship between the radar coordinate system and the vehicle coordinate system, and the second candidate calibration data represents the coordinate transformation relationship between the vehicle coordinate system and the camera coordinate system; The step of performing coordinate transformation processing on the first coordinate data based on the candidate calibration data and the reference coordinate transformation data to obtain the third coordinate data of the point cloud data in the camera coordinate system includes: Based on the first candidate calibration data, the first reference coordinate transformation data, the second reference coordinate transformation data, and the second candidate calibration data, coordinate transformation processing is performed on the first coordinate data to obtain the third coordinate data of the point cloud data in the camera coordinate system.

6. The method according to claim 1, characterized in that, The step of adjusting the candidate calibration data based on the second coordinate data and the third coordinate data until the termination condition is met to obtain the target calibration data includes: The candidate calibration data is adjusted based on the second coordinate data and the third coordinate data until the difference between the second coordinate data and the third coordinate data is within a preset difference range. The candidate calibration data at the end of the adjustment is then used as the target calibration data.

7. A data calibration device, characterized in that, include: The first acquisition module is used to acquire point cloud data collected by the radar and image data collected by the vehicle camera within a preset sampling time unit, and to determine the first coordinate data of the point cloud data in the radar coordinate system and the second coordinate data of the image data in the camera coordinate system. The second acquisition module is used to acquire the pose data set collected by the positioning device within the preset sampling time unit, and to perform interpolation processing on the pose data set to obtain reference coordinate transformation data; the reference coordinate transformation data represents the coordinate transformation relationship between the vehicle coordinate system and the world coordinate system. The coordinate transformation module is used to perform coordinate transformation processing on the first coordinate data based on the candidate calibration data and the reference coordinate transformation data to obtain the third coordinate data of the point cloud data in the camera coordinate system; the candidate calibration data represents the coordinate transformation relationship between the radar coordinate system and the vehicle coordinate system, as well as the coordinate transformation relationship between the vehicle coordinate system and the camera coordinate system. The adjustment module is used to adjust the candidate calibration data according to the second coordinate data and the third coordinate data until the termination condition is met to end the adjustment and obtain the target calibration data.

8. An electronic device, characterized in that, The electronic device includes a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the data calibration method as described in any one of claims 1-6.

9. A computer storage medium, characterized in that, The storage medium stores at least one instruction or at least one program, which is loaded and executed by a processor to implement the data calibration method as described in any one of claims 1-6.

10. A computer program product, characterized in that, The computer program product includes at least one instruction or at least one program segment, which is loaded and executed by a processor to implement the data calibration method as described in any one of claims 1-6.

Images