Universal handheld multi-sensor data acquisition device and method

Abstract

The invention belongs to the technical field of portable multi-sensor data acquisition, and discloses a universal handheld multi-sensor data acquisition device and method.The universal handheld multi-sensor data acquisition device comprises a main body rack, a power supply module, a driving acquisition module and a sensor module, and the power supply module, the driving acquisition module and the sensor module are installed on the main body rack; according to the device, original data sets of frame synchronization of all sensors in a sensor module are collected in a handheld mode, positioning coordinates of a Body coordinate system in a world coordinate system are obtained through a laser radar inertia SLAM algorithm, and universal data sets of various types of high-precision frame synchronization can be collected. According to the universal handheld multi-sensor data acquisition device and method provided by the invention, all common sensors in the fields of automatic driving and autonomous robots can be carried, the structure is compact and reliable, the manufacturing and the use are simple, and the acquired data can be used for various purposes such as multi-modal deep learning, algorithm evaluation and the like.

Description

Technical Field

[0001] The present invention belongs to the technical field of portable multi-sensor data acquisition, and in particular relates to a universal handheld multi-sensor data acquisition device and method. Background Art

[0002] With the development of intelligent transportation, autonomous driving, robotics and industrial automation, multi-sensor data plays a vital role in achieving high-precision perception and environmental understanding. Traditional single sensor systems often fail to meet the diverse needs in complex environments.

[0003] In order to overcome the limitations of a single sensor, multi-sensor fusion technology has become an inevitable choice. By comprehensively processing the data of multiple sensors such as millimeter-wave radar, lidar, camera, IMU and depth camera, the robustness and perception ability of the system can be significantly improved. For example, in autonomous driving, millimeter-wave radar and lidar can complement each other, responsible for long-distance and high-precision target detection respectively; IMU and depth camera are used for real-time posture estimation and close-range obstacle perception.

[0004] The development of multi-sensor fusion technology inevitably requires a large number of multi-sensor data sets, and portable, accurate, and easy-to-use multi-sensor data acquisition software and hardware systems are crucial. However, existing multi-sensor data acquisition software and hardware systems often cannot meet the requirements of rich sensor types, high data acquisition accuracy, portability, and easy assembly and use. How to efficiently and compactly integrate these sensors and ensure the consistency and accuracy of the data is a problem that needs to be solved urgently. Summary of the invention

[0005] The object of the present invention is to provide a universal handheld multi-sensor data acquisition device and method to solve the above-mentioned technical problems.

[0006] In order to solve the above technical problems, the specific technical solutions of a universal handheld multi-sensor data acquisition device and method of the present invention are as follows: A universal handheld multi-sensor data acquisition device comprises a main frame, a power supply module, a drive acquisition module, and a sensor module. The power supply module, the drive acquisition module, and the sensor module are mounted on the main frame. The device collects frame-synchronized original data sets of all sensors in the sensor module in a handheld manner, obtains the positioning coordinates of the Body coordinate system in the world coordinate system through a laser radar inertial SLAM algorithm, and collects various types of high-precision frame-synchronized universal data sets.

[0007] Furthermore, the main frame includes an upper plate, a middle plate, a bottom plate and a handle, the upper plate and the middle plate are connected via a connecting column, the middle plate and the bottom plate are connected via a connecting column, and the bottom plate and the handle are connected via a fastener.

[0008] Furthermore, the upper board is fixedly installed with a laser radar and a depth camera; the middle board is installed with an edge computer, a millimeter-wave radar and an analog signal acquisition module, an inertial measurement unit IMU and a power distribution board; the bottom board is fixedly installed with a gigabit one-to-two network distributor; a battery compartment is fixedly installed between the upper board and the bottom board, and a 4S model aircraft battery is installed in the battery compartment.

[0009] Furthermore, the laser radar model is Livox Mid360, the depth camera model is Intel® RealSense™ D435i; the edge computer model is NVIDIA Jetson Orin NX 16GB; the millimeter wave radar is TI IWR1843BOOST; and the analog signal acquisition module model is DCA1000EVM.

[0010] Furthermore, the power supply module includes an inertial measurement unit IMU, a distribution board and a 4S model aircraft battery. The distribution board inputs the output of the 4S model aircraft battery and outputs two equal voltages, which respectively power the laser radar and the edge computer. At the same time, it outputs 5V voltage to power the millimeter wave radar and the analog signal acquisition module.

[0011] Furthermore, the driving and collecting module includes an edge computer, which is used to drive the laser radar, the inertial measurement unit IMU, the millimeter wave radar, the analog signal collecting module and the depth camera, and record the collected data sets.

[0012] Furthermore, the sensor module includes a laser radar, an inertial measurement unit IMU, a millimeter-wave radar and an analog signal acquisition module and a depth camera. The laser radar is used to collect laser radar point cloud information within a 360° horizontal field of view FOV, the inertial measurement unit IMU is used to collect 6-axis IMU data, including three-axis acceleration and three-axis angular velocity, the millimeter-wave radar and the analog signal acquisition module are used to collect millimeter-wave radar raw data, and the depth camera is used to record depth maps, RGB images, and grayscale images of left and right views.

[0013] The present invention also discloses a data acquisition method of a universal handheld multi-sensor data acquisition device, comprising the following steps: Through the lidar inertial SLAM algorithm, the collected IMU data and lidar point cloud raw data are used to obtain the six-dimensional pose of the body coordinate system of the handheld multi-sensor data acquisition system in the world coordinate system, and the time synchronization function of ROS is used to set different raw data buffers through the approximate time strategy. When there is a message less than the allowable time error in all data buffers, the callback function is triggered, and the data are saved as the same frame to achieve soft synchronization of multi-sensor data. Finally, the frame-aligned depth image, left and right eye grayscale image, RGB image, lidar point cloud, millimeter-wave radar point cloud, and global pose of the body coordinate system are obtained.

[0014] Furthermore, the lidar inertial SLAM algorithm is based on an iterative error state Kalman filter IESKF, which performs forward propagation through the IMU data, predicts the state error, and then records the predicted state error for each frame, and then uses the lidar point cloud data to extract feature points and adjacent feature planes, construct residual terms, and use the residual state observation equation to update the error state Kalman filter; the lidar point cloud data required for updating the iterative error state Kalman filter is the lidar point cloud original data obtained after the distortion is corrected by the predicted state error saved during forward propagation.

[0015] A universal handheld multi-sensor data acquisition device and method of the present invention has the following advantages: A universal handheld multi-sensor data acquisition device and method of the present invention, comprising a variety of sensors such as millimeter wave radar, laser radar, depth camera, RGB camera, IMU, etc., and powering each sensor through a power distribution board, making the structure more compact and portable, the overall frame adopts a layered structure design, which is convenient for disassembly and assembly, and each sensor is modularly designed for easy replacement. Combined with the laser radar inertial SLAM technology, the multi-sensor data acquisition platform is positioned in real time. The time synchronization between multiple sensor frames is performed through the ROS time synchronizer, and the inter-frame synchronized raw data and synchronized positioning information of multiple sensors are obtained.

[0016] This method can quickly, concisely, and cost-effectively construct a variety of high-precision data sets, and can quickly and conveniently collect various types of sensor data for application in multiple fields of autonomous driving and autonomous robot research, thereby reducing research costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a structural diagram of the handheld multi-sensor data acquisition software and hardware system in an embodiment of the present invention.

[0018] Figure 2 This is the specific implementation process of the present invention.

[0019] Figure 3 This is the specific implementation process of the laser radar inertial odometer described in the present invention.

[0020] Explanation of the marks in the figure: 1. LiDAR; 2. Edge computer; 3. Upper board; 4. 4S model aircraft battery; 5. Inertial measurement unit IMU and power distribution board; 6. Middle board; 7. Battery compartment; 8. Gigabit one-to-two network distributor; 9. Bottom board; 10. Handle; 11. Millimeter wave radar and analog signal acquisition module; 12. Depth camera. DETAILED DESCRIPTION

[0021] In order to better understand the purpose, structure and function of the present invention, a general handheld multi-sensor data acquisition device and method of the present invention are further described in detail below with reference to the accompanying drawings.

[0022] like Figure 1 As shown, a universal handheld multi-sensor data acquisition device of the present invention includes a main frame, a power supply module, a drive acquisition module, and a sensor module. The device collects the frame-synchronized original data sets of all sensors in the sensor module in a handheld manner, and obtains the positioning coordinates of the Body coordinate system (IMU coordinate system) in the world coordinate system through a laser radar inertial SLAM algorithm. The greatest innovation is that it can simply collect various types of high-precision frame-synchronized universal data sets.

[0023] The main frame includes an upper plate 3, a middle plate 6, a bottom plate 9 and a handle 10. The upper plate 3 and the middle plate 6 are connected by connecting columns, the middle plate 6 and the bottom plate 9 are connected by connecting columns, and the bottom plate 9 and the handle 10 are connected by screws. The upper plate 3 is fixedly mounted with a laser radar 1 and a depth camera 12; the middle plate 6 is installed with an edge computer 2, a millimeter-wave radar and an analog signal acquisition module 11, an inertial measurement unit IMU and a power distribution board 5; the bottom plate 9 is fixedly mounted with a gigabit one-to-two network distributor 8; a battery compartment 7 is fixedly mounted between the upper plate 3 and the bottom plate 9, and a 4S aircraft model battery 4 is installed in the battery compartment 7. Preferably, the laser radar 1 is of Livox Mid360 model, the depth camera 12 is of Intel® RealSense™ D435i model; the edge computer 2 is of NVIDIA Jetson Orin NX 16GB model; the millimeter-wave radar is of TI IWR1843BOOST model; the analog signal acquisition module is of DCA1000EVM model; The power supply module includes an inertial measurement unit IMU, a distribution board and a 4S aircraft model battery 4. The distribution board input is the output of the 4S aircraft model battery 4, and the output is two equal voltages, which respectively power the laser radar 1 and the edge computer 2. At the same time, it outputs 5V voltage to power the millimeter wave radar and analog signal acquisition module 11.

[0024] The driving and collecting module includes an edge computer 2, which is used to drive the laser radar 1, the inertial measurement unit IMU, the millimeter wave radar and the analog signal collecting module 11 and the depth camera 12, and record the collected data sets.

[0025] The sensor module includes a laser radar 1, an inertial measurement unit IMU, a millimeter-wave radar and analog signal acquisition module 11, and a depth camera 12. The laser radar 1 is used to collect laser radar point cloud information within a 360° horizontal field of view (FOV). The inertial measurement unit IMU is used to collect 6-axis IMU data, including three-axis acceleration and three-axis angular velocity. The millimeter-wave radar and analog signal acquisition module 11 is used to collect millimeter-wave radar raw data. The depth camera 12 is used to record depth maps, RGB images, and grayscale images of left and right views.

[0026] like Figure 2 As shown, a general handheld multi-sensor data acquisition method of the present invention comprises the following steps: The LiDAR inertial SLAM algorithm uses the collected IMU data and LiDAR point cloud raw data to obtain the six-dimensional pose (position and attitude) of the Body coordinate system (IMU coordinate system) of the handheld multi-sensor data acquisition system in the world coordinate system, and uses the time synchronization function of ROS (Robot Operating System) to set different raw data buffers through the Approximate Time Policy. When there is a message less than the time tolerance in all data buffers, the callback function is triggered, and these data are saved as the same frame to achieve multi-sensor data soft synchronization. Finally, the frame-aligned depth image, left and right eye grayscale image, RGB image, LiDAR point cloud, millimeter-wave radar point cloud, and the global pose of the Body coordinate system are obtained.

[0027] like Figure 3 As shown, the LiDAR inertial SLAM algorithm is based on an iterative error state Kalman filter (IESKF), which performs forward propagation through IMU data, predicts state errors, and then records the predicted state errors for each frame. Then, the LiDAR point cloud data is used to extract feature points and adjacent feature planes, construct residual terms, and update the error state Kalman filter using the residual state observation equation. The LiDAR point cloud data required for the Iterative Error State Kalman Filter (IESKF) to update is obtained by correcting the distortion of the LiDAR point cloud raw data through the predicted state error saved during forward propagation.

[0028] It is to be understood that the present invention is described by some embodiments, and it is known to those skilled in the art that various changes or equivalent substitutions may be made to these features and embodiments without departing from the spirit and scope of the present invention. In addition, under the teachings of the present invention, these features and embodiments may be modified to adapt to specific circumstances and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the scope of protection of the present invention.

Claims

1. A universal handheld multi-sensor data acquisition device, characterized in that: The device comprises a main frame, a power supply module, a drive acquisition module and a sensor module, wherein the power supply module, the drive acquisition module and the sensor module are installed on the main frame. The device collects the frame-synchronized original data sets of all sensors in the sensor module in a handheld manner, obtains the positioning coordinates of the Body coordinate system in the world coordinate system through a laser radar inertial SLAM algorithm, and collects various types of high-precision frame-synchronized general data sets.

2. The universal handheld multi-sensor data acquisition device according to claim 1, characterized in that: The main frame comprises an upper plate (3), a middle plate (6), a bottom plate (9) and a handle (10); the upper plate (3) and the middle plate (6) are connected via a connecting column, the middle plate (6) and the bottom plate (9) are connected via a connecting column, and the bottom plate (9) and the handle (10) are connected via a fastener.

3. The universal handheld multi-sensor data acquisition device according to claim 2, characterized in that: The upper plate (3) is fixedly mounted with a laser radar (1) and a depth camera (12); the middle plate (6) is mounted with an edge computer (2), a millimeter wave radar and analog signal acquisition module (11), an inertial measurement unit (IMU) and a power distribution board (5); the bottom plate (9) is fixedly mounted with a gigabit one-to-two network distributor (8); a battery compartment (7) is fixedly mounted between the upper plate (3) and the bottom plate (9), and a 4S model aircraft battery (4) is mounted in the battery compartment (7).

4. The universal handheld multi-sensor data acquisition device according to claim 3, characterized in that: The laser radar (1) is of model Livox Mid360, the depth camera (12) is of model Intel® RealSense™ D435i; the edge computer (2) is of model NVIDIA Jetson Orin NX 16GB; the millimeter wave radar is TIIWR1843BOOST; and the analog signal acquisition module is of model DCA1000EVM.

5. The universal handheld multi-sensor data acquisition device according to claim 1, characterized in that: The power supply module comprises an inertial measurement unit (IMU), a power distribution board and a 4S model aircraft battery (4); the power distribution board input is the output of the 4S model aircraft battery (4), and the output is two equal voltages, which respectively power a laser radar (1) and an edge computer (2); and at the same time, a 5V voltage is output to power a millimeter wave radar and an analog signal acquisition module (11).

6. The universal handheld multi-sensor data acquisition device according to claim 1, characterized in that: The driving and collecting module comprises an edge computer (2) for driving a laser radar (1), an inertial measurement unit (IMU), a millimeter wave radar, an analog signal collecting module (11) and a depth camera (12), and recording the collected data set.

7. The universal handheld multi-sensor data acquisition device according to claim 1, characterized in that: The sensor module comprises a laser radar (1), an inertial measurement unit (IMU), a millimeter wave radar and analog signal acquisition module (11) and a depth camera (12); the laser radar (1) is used to collect laser radar point cloud information within a 360° horizontal field of view (FOV); the inertial measurement unit (IMU) is used to collect 6-axis IMU data, including three-axis acceleration and three-axis angular velocity; the millimeter wave radar and analog signal acquisition module (11) is used to collect millimeter wave radar raw data; and the depth camera (12) is used to record a depth map, an RGB image and grayscale images of left and right views.

8. A data acquisition method for a universal handheld multi-sensor data acquisition device according to any one of 1 to 7, characterized in that: The steps include: Through the lidar inertial SLAM algorithm, the collected IMU data and lidar point cloud raw data are used to obtain the six-dimensional pose of the body coordinate system of the handheld multi-sensor data acquisition system in the world coordinate system, and the time synchronization function of ROS is used to set different raw data buffers through the approximate time strategy. When there is a message less than the allowable time error in all data buffers, the callback function is triggered, and the data are saved as the same frame to achieve soft synchronization of multi-sensor data. Finally, the frame-aligned depth image, left and right eye grayscale image, RGB image, lidar point cloud, millimeter-wave radar point cloud, and global pose of the body coordinate system are obtained.

9. The data collection method according to claim 8, characterized in that: The described laser radar inertial SLAM algorithm is based on an iterative error state Kalman filter IESKF, which performs forward propagation through IMU data, predicts state errors, and then records the predicted state errors of each frame. Then, the laser radar point cloud data is used to extract feature points and adjacent feature planes, and residual terms are constructed. The error state Kalman filter is updated using the residual state observation equation. The laser radar point cloud data required for updating the iterative error state Kalman filter is obtained after the laser radar point cloud raw data is distorted by correcting the predicted state error saved during forward propagation.

Images