Multi-mode road environment sensing data acquisition system and method

Abstract

The invention discloses a multi-mode road environment perception data acquisition system and method, the system comprises a data acquisition hardware device and a data acquisition software device, the data acquisition hardware device is composed of a mounting frame, a camera, a laser radar, an IMU device, an RTK positioning module, an industrial personal computer, a laser radar adapter and a power supply, point cloud data, image data, attitude data and positioning data can be synchronously acquired through various sensors; and the data acquisition software device is deployed on the industrial personal computer and consists of a driving program, a calibration program, a synchronization program, an acquisition program and a visualization program. The method has the advantages of high integration, accurate data synchronization, efficient data processing and storage and the like, the environment perception capability and decision accuracy of an automatic driving system are remarkably improved, and the method is suitable for the safe driving requirement in a complex traffic environment.

Description

Technical Field

[0001] The present invention relates to the field of data acquisition for autonomous driving perception, and in particular to a multimodal road environment perception data acquisition system and method. Background Art

[0002] With the rapid development of autonomous driving technology, the demand for environmental perception and data collection has increased significantly. Autonomous driving systems rely on a variety of sensors, such as lidar, cameras, IMU devices, and RTK positioning modules, to achieve accurate perception and positioning of the surrounding environment. These sensors can provide a wealth of environmental information, including 3D point cloud data, image data, vehicle posture data, and high-precision positioning data, providing critical data support for autonomous vehicles' path planning, obstacle detection, and decision-making. Efficient and reliable multimodal data acquisition systems play a vital role in improving the safety, stability, and intelligence level of autonomous vehicles. They can effectively cope with complex and changing traffic environments and ensure accurate vehicle response and operation in various driving scenarios.

[0003] However, existing multi-sensor data acquisition devices suffer from numerous deficiencies in integration, data synchronization, and data processing efficiency, hindering the further development of autonomous driving technology. For example, existing systems typically utilize independent sensor modules and lack a unified integration platform, increasing overall system complexity and making installation and maintenance difficult. Imperfect spatial and temporal coordinate synchronization mechanisms between different sensors result in mismatches during data fusion, impacting the accuracy of environmental perception and decision-making. Furthermore, the volume of data generated by autonomous vehicles is enormous, and existing data acquisition systems face bottlenecks in data transmission, processing, and storage, making it difficult to meet real-time and high-efficiency requirements.

[0004] These technical defects urgently need to be addressed through innovative multimodal data acquisition systems and methods to enhance the environmental perception capabilities and decision-making accuracy of autonomous vehicles and meet the needs of safe driving in complex traffic environments. Summary of the Invention

[0005] In order to solve the above problems, the present invention proposes a multimodal road environment perception data acquisition system with the advantages of high integration, accurate data synchronization, efficient data processing and storage, etc., which significantly improves the environmental perception ability and decision-making accuracy of the autonomous driving system and is suitable for safe driving needs in complex traffic environments.

[0006] The specific plan is as follows:

[0007] On the one hand, a multimodal road environment perception data acquisition system includes: a data acquisition hardware device and a data acquisition software device; the data acquisition hardware device includes four sensors: a camera, a lidar, an inertial measurement unit (IMU) device, and an RTK positioning module, for respectively acquiring image data, point cloud data, attitude data, and positioning data; and further includes an industrial computer and a lidar adapter; the camera and IMU device are connected to the industrial computer, the lidar is connected to the industrial computer via the lidar adapter, and the RTK positioning module is connected to the industrial computer via the IMU device; the data acquisition software device is deployed on the industrial computer and includes a driver, a calibration program, a synchronization program, an acquisition program, and a visualization program; the driver is used to drive the normal use of the camera, lidar, IMU device, and RTK positioning module; the calibration program includes camera intrinsic parameter calibration and camera and lidar joint calibration functions, and outputs the camera's intrinsic parameters and the camera and lidar's extrinsic parameters based on the calibration data acquired by the camera and lidar; the synchronization program is used to achieve sensor clock source unification and interpolation synchronization of sensor data; the acquisition program is used to control communication between each sensor and the industrial computer to achieve transmission and storage of sensor data; and the visualization program is used to achieve visual display of sensor data.

[0008] Preferably, the data acquisition hardware device also includes a power supply; the power supply is connected to the industrial computer and the laser radar adapter respectively to supply power to the data acquisition hardware device.

[0009] Preferably, the data acquisition hardware device also includes a mounting frame; the camera, lidar, IMU device and lidar adapter are installed on the vehicle body through the mounting frame, the RTK positioning module is installed separately on the vehicle body, and the industrial computer and power supply are installed in the vehicle body.

[0010] Preferably, the mounting frame consists of an aluminum profile frame, a frame mounting plate, a bottom plate, a top plate and a camera fixing plate. The frame mounting plate is symmetrically installed at the four corners of the bottom of the aluminum profile frame, the bottom plate is installed inside the aluminum profile frame, and the top plate is installed above the aluminum profile frame.

[0011] Preferably, a laser radar and an IMU device are fixedly mounted on the top plate, a camera fixing plate is fixedly mounted above the laser radar, and a camera is mounted on the camera fixing plate; and a laser radar adapter is mounted on the bottom plate.

[0012] Preferably, a lidar adapter mounting hole is reserved on the bottom plate according to the hardware size; a lidar mounting hole and an IMU mounting hole are reserved on the top plate according to the sensor size; and a lidar mounting hole and a camera mounting hole are reserved on the camera fixing plate according to the sensor size.

[0013] In another aspect, a multimodal road environment perception data acquisition method includes the multimodal road environment perception data acquisition system, and the method includes:

[0014] Equipment deployment steps: correctly physically install and electrically connect the camera, lidar, IMU device, RTK positioning module, lidar adapter, industrial computer, and power supply;

[0015] Sensor configuration steps: Install the drivers for the camera, lidar, IMU device, and RTK positioning module, and configure the corresponding parameters for each sensor;

[0016] The sensor calibration step is to perform intrinsic parameter calibration on the camera to obtain the camera's intrinsic parameter matrix; the camera and lidar are jointly calibrated to obtain the camera and lidar's extrinsic parameter matrices, and the camera coordinate system is transformed to the lidar coordinate system through the extrinsic parameter matrix;

[0017] The sensor time synchronization step ensures that when the camera, lidar, IMU device, and RTK positioning module collect data, the timestamp of the recorded data frame is synchronized by the industrial computer, ensuring that the sensors use a unified clock source. The IMU data and RTK data at the lidar acquisition time point are calculated using the lidar timestamp as the benchmark using interpolation.

[0018] The data collection steps include sensor data transmission, specifically the camera, lidar, IMU device and RTK positioning module communicate with the industrial computer through their respective data lines and obtain timing, and then transmit the sensor data to the industrial computer for processing in the data acquisition software device.

[0019] Preferably, the data acquisition step also includes sensor data display, specifically, after the data acquisition software device receives the sensor data, the sensor data is displayed on the software interface, wherein the IMU data and RTK data are displayed in the format of quaternion, three-axis acceleration, three-axis angular velocity and longitude and latitude coordinates respectively; the image data is displayed as a video stream; and the laser point cloud data is visualized to display the three-dimensional spatial distribution.

[0020] Preferably, the data acquisition step also includes sensor data preservation, specifically, after the data acquisition software device completes the acquisition of sensor data, the sensor data is saved as a data set in the ROSBAG format, and the original data and timestamps collected by all sensors are recorded; from the ROSBAG format data set, the lidar data can be exported into a PCD point cloud file, the camera data can be exported into a PNG image file, and the IMU data and RTK data can be exported into an EXCEL data file.

[0021] The present invention adopts the above technical solution and has the following beneficial effects:

[0022] (1) The present invention integrates four sensors, namely, a camera, a lidar, an IMU device, and an RTK positioning module, through a unified installation framework, thereby reducing interference and space occupation between sensors, simplifying the installation and use of sensors, reducing the overall complexity and maintenance cost of the system, and improving the overall integration and stability of the system;

[0023] (2) The present invention adopts a sensor joint calibration and time synchronization algorithm to ensure the consistency of multiple sensors in time and space, reducing the data fusion error caused by the temporal and spatial misalignment between different sensors, effectively improving the accuracy of the data and the overall performance of the system, and providing a solid foundation for the reliable operation of the autonomous driving system;

[0024] (3) The present invention integrates a driver, a calibration program, a visualization program, an acquisition program, and a visualization program. By comprehensively acquiring point cloud data, image data, posture data, and positioning data, it realizes the real-time transmission, display, and storage of sensor data, supports the export of multiple data formats, and provides more comprehensive and accurate environmental perception information, meeting the needs of autonomous driving vehicles for efficient data processing and storage. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 Schematic diagram of the overall architecture of the multimodal road environment perception data acquisition system of the present invention;

[0026] Figure 2 It is a three-dimensional diagram of a portion of the structure of the data acquisition hardware device of the present invention;

[0027] Figure 3 It is a partial structural front view of the data acquisition hardware device of the present invention;

[0028] Figure 4 It is a partial structural top view of the data acquisition hardware device of the present invention;

[0029] Figure 5 is a flow chart of the multimodal road environment perception data acquisition method of the present invention;

[0030] Figure 6 This is a unified schematic diagram of the clock source of the present invention;

[0031] In the figure: 1. Data acquisition hardware device; 2. Data acquisition software device; 11. Mounting frame; 12. Camera; 13. LiDAR; 14. IMU device; 15. RTK positioning module; 16. Industrial computer; 17. LiDAR adapter; 18. Power supply; 21. Driver; 22. Calibration program; 23. Synchronization program; 24. Acquisition program; 25. Visualization program; 111. Aluminum profile frame; 112. Frame mounting plate; 113. Bottom plate; 114. Top plate; 115. Camera fixing plate. DETAILED DESCRIPTION

[0032] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0033] See also Figures 1 to 4 As shown, the present invention provides a multimodal data acquisition system, which includes a data acquisition hardware device 1 and a data acquisition software device 2.

[0034] The data acquisition hardware device 1 consists of a mounting frame 11, a camera 12, a laser radar 13, an IMU device 14, an RTK positioning module 15, an industrial computer 16, a laser radar adapter 17, and a power supply 18. The camera 12 and the IMU device 14 are connected to the industrial computer 16, the laser radar 13 is connected to the industrial computer 16 through the laser radar adapter 17, and the RTK positioning module 15 is connected to the industrial computer 16 through the IMU device 14; the power supply 18 is connected to the industrial computer 16 and the laser radar adapter 17 to supply power to the data acquisition hardware device 1; the camera 12, the laser radar 13, the IMU device 14, and the laser radar adapter 17 are installed on the vehicle body through the mounting frame 11, the RTK positioning module 15 is installed on the vehicle body separately, and the industrial computer 16 and the power supply 18 are installed in the vehicle body.

[0035] The data acquisition software device 2 is deployed on the industrial computer 16 and consists of a driver program 21, a calibration program 22, a synchronization program 23, an acquisition program 24, and a visualization program 25. The driver program 21 includes drivers for various sensor devices, and is used to enable the normal use of sensors such as the camera 12, the lidar 13, the IMU device 14, and the RTK positioning module 15. The calibration program 22 includes functions for camera intrinsic parameter calibration and camera and lidar joint calibration. Based on the calibration data collected by the camera and lidar, it outputs the camera's intrinsic parameters and the camera and lidar's extrinsic parameters. The synchronization program 23 is used to achieve sensor clock source unification and interpolation synchronization of sensor data; the acquisition program 24 is responsible for controlling the communication between each sensor and the industrial computer 16, enabling the transmission and storage of sensor data; and the visualization program 25 is used to visualize the sensor data, displaying it in real time on the software interface.

[0036] In this embodiment, the mounting frame 11 consists of an aluminum profile frame 111, a frame mounting plate 112, a bottom plate 113, a top plate 114, and a camera fixing plate 115. The frame mounting plate 112 is symmetrically installed at the four corners of the bottom of the aluminum profile frame 111, the bottom plate 113 is installed inside the aluminum profile frame 111, and the top plate 114 is installed above the aluminum profile frame 111.

[0037] In this embodiment, a laser radar 13 and an IMU device 14 are fixedly mounted on the top plate 114, a camera fixing plate 115 is fixedly mounted above the laser radar 13, and a camera 12 is mounted on the camera fixing plate 115. The camera fixing plate 115 can keep the relative positions of the camera 12 and the laser radar 13 fixed and non-interfering with each other; a laser radar adapter 17 is mounted on the bottom plate 113; sensors such as the camera 12, the laser radar 13, and the IMU device 14 can be quickly deployed through the mounting frame 11, and the relative positions of the camera 12, the laser radar 13, and the IMU device 14 can be kept fixed, thereby avoiding repeated sensor calibration work.

[0038] In this embodiment, a lidar adapter mounting hole is reserved on the bottom plate 113 according to the hardware size; a lidar mounting hole and an IMU mounting hole are reserved on the top plate 114 according to the sensor size; and a lidar mounting hole and a camera mounting hole are reserved on the camera fixing plate 115 according to the sensor size.

[0039] In this embodiment, the multimodal data acquisition system 1 can simultaneously collect four types of data, including point cloud data, image data, attitude data, and positioning data, providing more comprehensive and accurate environmental perception information for autonomous driving; the point cloud data is collected by the lidar 13, the image data is collected by the camera 12, the attitude data is collected by the IMU device 14, and the positioning data is collected by the RTK positioning module 15.

[0040] See also Figure 5 As shown, the present invention also provides a multimodal data acquisition method, which includes five steps:

[0041] In the device deployment step 501 , the camera 12 , the lidar 13 , the IMU device 14 , the RTK positioning module 15 , the lidar adapter 17 , the industrial computer 16 , and the power supply 18 are properly physically installed and electrically connected;

[0042] In the sensor configuration step 502 , the drivers of the camera 12 , the lidar 13 , the IMU device 14 , and the RTK positioning module 15 are installed, and the corresponding parameters of each sensor are configured;

[0043] In the sensor calibration step 503, the camera 12 is calibrated with internal parameters to obtain the camera's internal parameter matrix; the camera 12 and the lidar 13 are jointly calibrated to obtain the camera's and lidar's external parameter matrices, and the camera coordinate system is transformed into the lidar coordinate system through the external parameter matrix;

[0044] In the sensor time synchronization step 504, when the camera 12, lidar 13, IMU device 14, and RTK positioning module 15 collect data, the timestamp of the data frame is recorded by the industrial computer 16 to ensure that the sensors use a unified clock source. The IMU data and RTK data at the time of the lidar collection are calculated using the lidar timestamp as a reference using interpolation.

[0045] The data acquisition step 505 includes sensor data transmission, specifically the camera 12, lidar 13, IMU device 14 and RTK positioning module 15 communicate with the industrial computer 16 through their respective data lines and obtain timing, and then transmit the sensor data to the industrial computer for processing in the data acquisition software device 2.

[0046] In this embodiment, the device deployment step 501 is specifically as follows: the camera 12, the lidar 13, the IMU device 14, and the lidar adapter 17 are installed on the top of the vehicle to be tested through the mounting frame 11, and the fixing method is to fix the four frame mounting plates 112 to the top of the vehicle to be tested by bolts or adhesive; the RTK positioning module 15 is fixed on the top of the vehicle body and connected to the IMU device 14 through a data cable; the industrial computer 16 and the power supply 18 are installed inside the vehicle body, and the industrial computer is connected to the mounting frame 11; the integrity of the data acquisition hardware device 1 is detected to ensure that all data cables are correctly connected; ensure that the power supply 18 is connected normally, the power supply 18 provides stable voltage, and the normal power supply requirements of the equipment are guaranteed. After power-on, each hardware device works normally.

[0047] In this embodiment, the sensor configuration step 502 is specifically as follows: confirming that the platform and system version of the industrial computer 16 meet the requirements and can operate normally; installing sensor drivers that meet the platform and system version of the industrial computer according to the device manual and official documents of the sensor equipment, including lidar driver, camera driver, IMU driver, and RTK driver; setting the network configuration of the industrial computer 16 according to the requirements of the lidar 13, setting the IP address of the industrial computer 16 to a static IP and being in the same network segment as the lidar 13, and modifying the gateway and subnet mask to ensure that the lidar 13 can transmit point cloud data to the industrial computer and that the field of view is not blocked by the vehicle body; setting the configuration parameters of the camera 12, including parameters such as resolution and frame rate, to ensure that the camera can transmit image data to the industrial computer and that the image data is not blocked by the vehicle body; setting the configuration parameters of the IMU device 14 and the RTK positioning module 15 to ensure that the attitude data and positioning data of the IMU and RTK are accurate and can be transmitted to the industrial computer.

[0048] In this embodiment, the sensor calibration step 503 includes: camera intrinsic parameter calibration, camera and lidar joint calibration.

[0049] The camera intrinsic parameter calibration is specifically as follows: the camera 12 is used to collect calibration plate images at different distances and angles, and the camera's focal length, principal point coordinates, distortion coefficient and other internal parameters are obtained through the calibration image. The image data is corrected according to the output camera intrinsic parameter matrix to ensure the geometric correction and accuracy of the camera-taken images.

[0050] The joint calibration of the camera and lidar is specifically as follows: the point cloud data and image data of the same scene are collected simultaneously by the camera 12 and the lidar 13, and the data are collected again after changing different distances and angles. After calibration, the external parameter matrix of the camera 12 and the lidar 13 is obtained, and the transformation relationship from the camera coordinate system to the lidar coordinate system is realized through the external parameter matrix, so as to realize the one-to-one mapping of image data and point cloud data, which can provide important support for multi-sensor data fusion.

[0051] In this embodiment, the sensor time synchronization step 504 includes: unifying sensor clock sources and synchronizing data interpolation.

[0052] See also Figure 6 As shown, the sensor clock source is unified as follows: when the camera 12, lidar 13, IMU device 14 and RTK positioning module 15 collect data, they synchronously obtain the industrial computer time at that moment and record it as the timestamp of the data frame to ensure that the sensors use a unified clock source.

[0053] The specific data interpolation synchronization is as follows: when the multimodal data acquisition system collects sensor data, the IMU sequence data and RTK sequence data are based on the timestamp of the lidar, and the IMU data and RTK data at the time point of the lidar collection are calculated using the interpolation method; the timestamp of the series data to be synchronized is recorded as t s,1 ,t s,2 ,…,t s,n , the corresponding data value is D s (t s,1 ),D s (t s,2 ),…,D s (t s,j ),…D s (t s,n ), the timestamp of the lidar is t L,1 ,t L,2 ,…,t L,m For each radar timestamp t L,i , find s Two time points t in s,j and t s,j+1 Satisfaction: t s,j ≤t L,i ≤t s,j+1 , and then calculate the synchronized data value D S (t L,i ):

[0054]

[0055] The data collection step 505 shown includes: sensor data transmission, sensor data display and sensor data storage.

[0056] The sensor data transmission is specifically as follows: when the multimodal data acquisition system collects sensor data, the camera 12, lidar 13, IMU device 14 and RTK positioning module 15 communicate with the industrial computer 16 through their respective data lines and obtain timing, and then transmit four types of data such as image data, point cloud data, attitude data, positioning data and corresponding timestamps to the industrial computer for processing in the data acquisition software device 2.

[0057] The sensor data display is specifically as follows: after the data acquisition software device 2 receives the sensor data, it displays the sensor data in real time on the software interface, where the IMU data is displayed as quaternion data representing the posture and three-axis acceleration and three-axis angular velocity representing the motion state; the RTK data shows the current latitude and longitude coordinates; the image data is played frame by frame according to the timestamp to form a video stream; the laser point cloud data is visualized to display the three-dimensional spatial distribution, and the points in the space are rendered into the three-dimensional coordinate system with color, size or intensity information.

[0058] The sensor data is saved specifically as follows: after the data acquisition software device 2 completes the acquisition of sensor data, the sensor data can be saved as a data set in the ROSBAG format, recording the original data and timestamps collected by all sensors; the data set in the ROSBAG format can be further exported into other data formats, for example, lidar data can be exported into PCD point cloud files, camera data can be exported into PNG image files, and IMU and RTK data can be exported into EXCEL data files.

[0059] The above is only a specific implementation of the present invention, but the design concept of the present invention is not limited to this. Any non-substantial changes to the present invention using this concept shall be deemed as an infringement of the protection scope of the present invention.

Claims

1. A multimodal road environment perception data acquisition system, characterized in that: include: A data acquisition hardware device (1) and a data acquisition software device (2); the data acquisition hardware device (1) includes four sensors, namely a camera (12), a laser radar (13), an IMU device (14) and an RTK positioning module (15), for respectively collecting image data, point cloud data, attitude data and positioning data; and further includes an industrial computer (16) and a laser radar adapter (17); the camera (12) and the IMU device (14) are connected to the industrial computer (16), the laser radar (13) is connected to the industrial computer (16) via the laser radar adapter (17), and the RTK positioning module (15) is connected to the industrial computer (16) via the IMU device (14); the data acquisition software device (2) is deployed on the industrial computer (16), and includes a driver (21), a calibration program (22 ), synchronization program (23), acquisition program (24) and visualization program (25); the driver program (21) is used to drive the normal use of the camera (12), laser radar (13), IMU device (14) and RTK positioning module (15); the calibration program (22) includes the camera internal parameter calibration and the camera and laser radar joint calibration function, and outputs the internal parameters of the camera (12) and the external parameters of the camera (12) and the laser radar (13) according to the calibration data collected by the camera (12) and the laser radar (13); the synchronization program (23) is used to realize the unification of the sensor clock source and the interpolation synchronization of the sensor data; the acquisition program (24) is used to control the communication between each sensor and the industrial control computer (16) to realize the transmission and storage of the sensor data; the visualization program (25) is used to realize the visual display of the sensor data.

2. The multimodal road environment perception data acquisition system according to claim 1, characterized in that: The data acquisition hardware device (1) further includes a power supply (18); the power supply (18) is connected to the industrial computer (16) and the laser radar adapter (17) respectively to supply power to the data acquisition hardware device (1).

3. The multimodal road environment perception data acquisition system according to claim 2, characterized in that: The data acquisition hardware device (1) further includes a mounting frame (11); a camera (12), a laser radar (13), an IMU device (14) and a laser radar adapter (17) are mounted on a vehicle body through the mounting frame (11); an RTK positioning module (15) is separately mounted on the vehicle body; and an industrial computer (16) and a power supply (18) are mounted inside the vehicle body.

4. The multimodal road environment perception data acquisition system according to claim 3, characterized in that: The mounting frame (11) is composed of an aluminum profile frame (111), a frame mounting plate (112), a bottom plate (113), a top plate (114) and a camera fixing plate (115), wherein the frame mounting plate (112) is symmetrically mounted at the four corners of the bottom of the aluminum profile frame (111), the bottom plate (113) is mounted inside the aluminum profile frame (111), and the top plate (114) is mounted above the aluminum profile frame (111).

5. The multimodal road environment perception data acquisition system according to claim 4, characterized in that: A laser radar (13) and an IMU device (14) are fixedly mounted on the top plate (114); a camera fixing plate (115) is fixedly mounted above the laser radar (13); and a camera (12) is mounted on the camera fixing plate (115); and a laser radar adapter (17) is mounted on the bottom plate (113).

6. The multimodal road environment perception data acquisition system according to claim 4, characterized in that: The bottom plate (113) is provided with a laser radar adapter mounting hole according to the hardware size; the top plate (114) is provided with a laser radar mounting hole and an IMU mounting hole according to the sensor size; and the camera fixing plate (115) is provided with a laser radar mounting hole and a camera mounting hole according to the sensor size.

7. A multimodal road environment perception data acquisition method, characterized in that: The multimodal road environment perception data acquisition system according to any one of claims 1 to 6, wherein the method comprises: Equipment deployment step: correctly physically install and electrically connect the camera (12), lidar (13), IMU device (14), RTK positioning module (15), lidar adapter (17), industrial computer (16) and power supply (18); The sensor configuration step includes installing the drivers of the camera (12), lidar (13), IMU device (14) and RTK positioning module (15), and configuring the corresponding parameters of each sensor; The sensor calibration step is to perform internal parameter calibration on the camera (12) to obtain the internal parameter matrix of the camera; the camera (12) and the laser radar (13) are jointly calibrated to obtain the external parameter matrices of the camera and the laser radar, and the camera coordinate system is transformed to the laser radar coordinate system through the external parameter matrix; The sensor time synchronization step is to realize that when the camera (12), laser radar (13), IMU device (14) and RTK positioning module (15) collect data, the timestamp of the data frame is recorded by the industrial control computer (16) to ensure that the sensors use a unified clock source; based on the timestamp of the laser radar, the IMU data and RTK data at the time point of the laser radar collection are calculated using the interpolation method; The data acquisition step includes sensor data transmission, specifically, the camera (12), laser radar (13), IMU device (14) and RTK positioning module (15) communicate with the industrial computer (16) through their respective data lines and obtain timing, and then transmit the sensor data to the industrial computer for processing in the data acquisition software device (2).

8. The multimodal road environment perception data acquisition method according to claim 7, characterized in that: The data acquisition step also includes sensor data display, specifically, after the data acquisition software device (2) receives the sensor data, the sensor data is displayed on the software interface, wherein the IMU data and RTK data are displayed in the format of quaternion, three-axis acceleration, three-axis angular velocity and latitude and longitude coordinates respectively; the image data is displayed as a video stream; and the laser point cloud data is visualized to display the three-dimensional spatial distribution.

9. The multimodal road environment perception data acquisition method according to claim 7, characterized in that: The data acquisition step also includes sensor data storage, specifically, after the data acquisition software device (2) completes the acquisition of sensor data, the sensor data is saved as a data set in the ROSBAG format, and the original data and timestamps collected by all sensors are recorded; From the ROSBAG format dataset, you can export lidar data into PCD point cloud files, camera data into PNG image files, and IMU data and RTK data into EXCEL data files.

Images