Multi-mode ground data acquisition unmanned vehicle platform

Abstract

The invention discloses a multi-modal ground data acquisition unmanned vehicle platform, which is characterized in that a mobile carrying platform of the platform is divided into three layers, a plurality of preformed holes are reserved on the mobile carrying platform, the positions of sensors can be adjusted as required, and different sensors are mounted on each layer of platform and comprise a bottom layer event camera, a binocular camera and a depth camera; the fisheye camera, the inertial measurement unit, the single-line laser radar and the microphone array are arranged on the middle layer, the GNSS-RTK positioning module, the UWB positioning module, the environment sensor and the monocular camera pointing to the sky are arranged on the top layer, time synchronization is carried out on data collected by all the sensors through timestamps, space synchronization is carried out by recording pose information, and the single-line laser radar and the microphone array are arranged on the top layer. The system further comprises an NUC control module and a power supply device. Each sensor is connected to the main control unit through a data line for power supply, data transmission and data acquisition. The data acquisition platform solves the problems that the number and scale of sensors of an existing data acquisition platform are small, and the position relation is not clear, has high environmental adaptability, and can improve the richness of acquired data.

Description

Technical Field

[0001] The invention belongs to the technical field of mobile robots and data acquisition platforms, and relates to a multi-sensor data acquisition platform for an unmanned vehicle. Background Art

[0002] At present, in the face of the requirements for unmanned and autonomous combat equipment, it is necessary to pre-train the different purposes and functions of intelligent equipment in advance before the combat equipment is put into combat. This requires the establishment of training scenes similar to the real environment, containing rich environmental information. Generally, for a sensor, the environmental information collected is only one type of image information, sound information or point cloud information. Therefore, when establishing a training environment, the environmental information collected by a single or a few sensors is far from enough. The construction of a multi-sensory data acquisition platform provides a platform for the collection of simulation data sets for multi-modal, multi-environmental scenarios and even simulated interference situations. In addition, the popularization of deep learning in the pre-training process of intelligent devices requires a large amount of data for training. Therefore, sensor data collection for a variety of environmental information is very necessary, but the existing data acquisition platform has the following defects:

[0003] The data collection platform for unmanned vehicles has a single collection scenario, and can only collect data for a single scene outdoors or indoors, lacking flexibility. The sensor type is single, and the need for photoelectric sensor data fusion is not taken into account. In the data set collection for simultaneous positioning and mapping, the sound source positioning is not taken into account. When there is a lack of visual information or when encountering occlusion, the positioning accuracy will decrease. Most sensors are fixed in position, and the positional relationship between each sensor is not clear enough, which makes it inconvenient for subsequent users to change and adjust the position of the sensor on the platform. Most sensors cannot be turned on and off for specific environments or for data types collected for special purposes, resulting in invalid opening of sensors that collect redundant data types, increasing the power consumption of the data collection platform; in addition, redundant data types, such as RGB image information collected in a completely dark environment, may cause data redundancy in the data set and occupy storage space. Summary of the invention

[0004] The present invention proposes a multimodal ground data acquisition unmanned vehicle platform to solve the problems of the existing data acquisition platform such as single acquisition scenario, poor flexibility and single sensor type, and realize the optimized design of the multimodal data acquisition platform.

[0005] The technical solution adopted by the present invention is:

[0006] A multimodal ground data acquisition unmanned vehicle platform, comprising a mobile carrier platform, a multi-sensor integrated platform, a control module and a power supply device; the mobile carrier platform is composed of an unmanned remote-controlled electric vehicle, a metal connecting rod, a metal plate and a wireless remote control module, and comprises three layers, each layer is connected by a metal connecting rod, and holes are reserved on each platform, on which sensors are either directly installed or connected by metal studs; the wireless remote control module controls the route and direction of the unmanned remote-controlled electric vehicle through a wireless transmitting circuit; the multi-sensor integrated platform is composed of an event camera, a binocular camera, a depth camera, a fisheye camera, an inertial measurement unit, a single-line laser radar, a microphone array, a GNSS-RTK positioning module, a UWB positioning module, an environmental sensor and a monocular camera pointing to the sky, the event camera is installed on the center line of the bottom metal plate, the binocular camera is installed on the center line of the bottom metal plate and behind the event camera, and the depth camera is installed on the center line of the bottom metal plate and behind the event camera Below, the fisheye cameras are respectively installed on the left front, left rear, right front, right rear and rear midlines of the middle metal plate, the inertial measurement unit is closely installed on the midline of the middle metal plate, the single-line laser radar is placed on the midline of the middle metal plate and supported to a certain height by metal studs, the microphone array is installed on the midline of the middle metal plate, the GNSS-RTK positioning module is installed on the top metal plate and supported to a certain height by metal studs, the UWB positioning module is installed on the top metal plate and supported to a certain height by metal studs, the environmental sensor is installed on the midline of the top metal plate, the monocular camera pointing to the sky is installed on the midline of the top metal plate, the control module is connected to the multi-sensor integrated platform and installed on the rear side of the bottom metal plate, and is used to control the opening and closing of the sensors in the above-mentioned multi-sensor integrated platform, and to save and transmit data, and the power supply device is connected to the control module for powering the control module and the sensor.

[0007] Preferably, the chassis of the unmanned remote-controlled car is an Ackerman structure, and its size is 68*42*76cm.

[0008] Preferably, the microphone array is a six-microphone circular array mounted on the intermediate metal plate, and the sound source localization can be performed after subsequent processing using the data set.

[0009] Preferably, the GNSS-RTK module has two antennas on the mobile carrier platform to achieve communication between the mobile terminal and the satellite, and cooperates with the base station to achieve differential RTK positioning.

[0010] Preferably, the UWB positioning module has two antennas on the mobile carrier platform to realize sending and receiving UWB signals, and cooperates with the base station to realize UWB ranging using the two-way time-of-flight method.

[0011] Preferably, the environmental sensor integrates sensors such as temperature, humidity, PM1.0, PM2.5, PM10, atmospheric pressure, light and noise, and is installed on the top metal plate to obtain richer information about the surrounding environment.

[0012] Preferably, the monocular camera pointing to the sky is installed on the top metal plate to facilitate monitoring whether the top of the unmanned vehicle is blocked, and then choose whether to use the data collected by the GNSS-RTK module as the accurate value of outdoor positioning.

[0013] Preferably, the multi-sensor integrated platform is connected to the control module, and the control module is connected to the power supply module. The control unit used is NUC, which is connected to external devices in the same local area network during use to save and transmit collected data. The battery used is an outdoor battery to ensure sufficient voltage and endurance to power the sensor and NUC, and is installed on the bottom metal plate.

[0014] Compared with the existing technology, the multi-modal ground data acquisition unmanned vehicle platform of the present invention has the following beneficial effects:

[0015] The present invention can perform indoor and outdoor positioning through the GNSS-RTK module and the UWB module, and is not limited by the use scenario, so that the collected data has good applicability. The sensor types are rich, and its distinctive features are: it has a monocular camera pointing to the sky, which can monitor whether the top of the unmanned vehicle is blocked outdoors; it has a microphone array sensor, which can obtain the sound of the surrounding environment, perform sound positioning and determine the sound source; it has an environmental sensor, which can detect factors such as PM2.5, PM10 and noise in the environment, which makes it possible to collect rich data information in complex environments such as battlefields.

[0016] The present invention reserves extra holes on the multimodal ground data acquisition unmanned vehicle platform, and the user can adjust the position of the sensor as needed, and the height of the sensor can be adjusted by adjusting the length of the metal connecting rod. After the adjustment, the collected information can be subsequently processed according to the clear sensor position information. The required sensors can be selected to be turned on according to the data acquisition type of the specific environment. The sensor is mainly turned on and off by adjusting the presence or absence of the sensor name in the launch file under the ROS working environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 The structural principle diagram of the embodiment of the present invention is

[0018] Figure 2 A side view of an embodiment of the present invention

[0019] Figure 3 A front view of an embodiment of the present invention

[0020] Figure 4 A top view of an embodiment of the present invention

[0021] Among them, 1-unmanned remote-controlled electric vehicle, 2-metal connecting rod, 3-metal plate, 4-event camera, 5-binocular camera, 6-depth camera, 7-fisheye camera, 8-inertial measurement unit, 9-single-line lidar, 10-microphone array, 11-GNSS-RTK positioning module, 12-UWB positioning module, 13-environmental sensor, 14-monocular camera pointing to the sky, 15-NUC, 16-outdoor power supply. DETAILED DESCRIPTION

[0022] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and through specific implementations. The implementations described herein are only used to illustrate and explain the present invention and are not used to limit the present invention.

[0023] See also Figure 1-4The present embodiment is described below. A multimodal ground data acquisition unmanned vehicle platform described in the present embodiment includes a mobile carrier platform, a multi-sensor integrated platform, a control module and a power supply device; the mobile carrier platform is composed of an unmanned remote-controlled electric vehicle 1, a metal connecting rod 2, a metal plate 3 and a wireless remote control module. The chassis of the unmanned remote-controlled vehicle 1 is an Ackerman structure with a size of 68*42*76cm; the metal connecting rod 2 and the metal plate 3 are installed on the unmanned remote-controlled vehicle to fix the multi-sensor integrated platform; the wireless remote control module is used to receive the radio signal of the remote controller to control the movement of the unmanned remote-controlled vehicle 1 during multimodal data acquisition. The multi-sensor integrated platform consists of an event camera 4, a binocular camera 5, a depth camera 6, a fisheye camera 7, an inertial measurement unit 8, a single-line laser radar 9, a microphone array 10, a GNSS-RTK positioning module 11, a UWB positioning module 12, an environmental sensor 13 and a monocular camera 14 pointing to the sky. The event camera 4 is installed in front of the metal plate of the stratum to capture the events in front, the binocular camera 5 is installed on the back side of the event camera 4, and the depth camera 6 is installed directly below the event camera 4 to reduce the mutual interference with the GNSS-RTK module 11 on the top layer; five Fisheye cameras 7 are respectively installed on the left and right sides and rear of the unmanned remote control car 1; the inertial measurement unit 8 is closely installed on the center line of the middle metal plate; the single-line laser radar 9 is installed on the center line of the middle metal plate and is raised with a support rod to reduce obstruction; the microphone array 10 is a six-microphone circular array, installed on the center line of the middle metal plate, for collecting external sounds; the GNSS-RTK positioning module 11 has two antennas, which are installed on the top metal plate to reduce mutual interference with the depth camera, and the antenna is raised; the UWB positioning module 12 is installed on the top metal plate to reduce the metal plate from interfering with the depth camera. The reflection of UWB signal raises the UWB positioning module 12; the environmental sensor 13 integrates temperature, humidity, PM1.0, PM2.5, PM10, atmospheric pressure, light and noise sensors, and is installed on the top metal plate to obtain more abundant information about the surrounding environment; the monocular camera 14 pointing to the sky is installed on the center line of the top metal plate; the control module uses NUC15, which is used to save and transmit the collected data and is installed on the bottom metal plate; the power supply module uses outdoor power supply 16, which is used to ensure sufficient voltage and endurance of the multi-sensor integrated platform and is installed on the bottom metal plate. The data collected by each sensor is synchronized in time through timestamps and in space by recording posture information.

[0024] Before this system officially works, you need to turn on the control module NUC15, connect NUC15 and the laptop to the same local area network, remotely control NUC15 through the software Nomachine, turn on and off the sensor, and monitor the data output in real time.

[0025] According to the data collection scenario, determine the type of sensor that needs to be enabled, write a launch file in the ROS environment, and write the name of the sensor that needs to be enabled into the launch file.

[0026] Before collecting data, it is necessary to perform signal detection on the GNSS-RTK module 11 to check whether the signal quality meets the positioning requirements, calibrate the UWB positioning module 12, build the base station indoors in a rectangular position, and then calibrate the base station position using the UWB positioning module 12. Generally, the GNSS-RTK positioning module 11 and the UWB positioning module 12 are not used at the same time. The GNSS-RTK positioning module 11 is used in outdoor scenes, and the UWB positioning module 12 is used in indoor scenes.

[0027] After setting up the GNSS-RTK positioning module 11 and the UWB positioning module 12, it is necessary to check the names of the remaining sensors to ensure that each sensor is recognized by the NUC 15 to avoid the loss of subsequent collected data. The main checks are for camera devices and USB serial port devices.

[0028] Use the ROS command to obtain the name of the sensor device that starts normally. After confirming that each sensor can be recognized by NUC15, reopen the workspace and use the launch file to open the nodes of each sensor device. If there is no error message, it means that the sensor device has been opened correctly. Then you can use the software RVIZ on the laptop to view the output data of each sensor in real time.

[0029] After confirming that the sensor equipment is operating normally, use ROS commands to record the data collected by all sensors. The file format being saved is .bag.active, and the final file format is .bag.

[0030] After the system starts to run formally, the remote controller can be used to control the running track and direction of the unmanned remote control car 1, and the multi-sensory data can be collected according to the specified data collection route.

[0031] After the ground data collection is completed, the data is post-processed, which mainly includes the calibration of monocular cameras, binocular cameras and fisheye cameras, and the joint calibration of inertial sensor units and cameras. The collected data is time synchronized through timestamps and spatially synchronized by recording posture information.

[0032] The specific embodiments described above further illustrate the purpose and technical solutions of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention, and the protection scope of the present invention should be based on the attached claims.

Claims

1. A multi-modal ground data acquisition unmanned vehicle platform, characterized by: It includes a mobile carrier platform, a multi-sensor integrated platform, a control module and a power supply device. The multi-sensor integrated platform is a platform that integrates multiple sensors and is used to collect ground data, including image information, sound information, and point cloud information data. The control module is connected to the multi-sensor integrated platform through a data line to control the opening and closing of the sensors in the multi-sensor integrated platform, as well as the transmission and storage of collected data. The power supply device provides power for the operation of the multi-sensor platform and the control module. The mobile carrier platform provides a mounting platform for the multi-sensor integrated platform, the control module and the power supply device, and controls the direction and path of the unmanned vehicle when collecting ground data. The data collected by the multi-sensor integrated platform are calibrated as needed, and the timestamps of each data are synchronized, and spatial synchronization is performed by recording the position information of the sensor.

2. The multimodal ground data acquisition unmanned vehicle platform according to claim 1, characterized in that: The mobile transport platform is composed of an unmanned remote-controlled electric vehicle (1), a metal connecting rod (2), a metal plate (3) and a wireless remote controller, and includes three layers, each layer is connected by a metal connecting rod (2), and holes are reserved on each platform, on which sensors are either directly installed or connected by metal studs; the wireless remote controller controls the route and direction of the unmanned remote-controlled electric vehicle (1) through a wireless transmission circuit.

3. The multimodal ground data acquisition unmanned vehicle platform according to claim 1, characterized in that: The multi-sensor integrated platform is composed of an event camera (4), a binocular camera (5), a depth camera (6), a fisheye camera (7), an inertial measurement unit (8), a single-line laser radar (9), a microphone array (10), a GNSS-RTK positioning module (11), a UWB positioning module (12), an environmental sensor (13) and a monocular camera (14) pointing to the sky. The event camera (4) is installed on the center line of the bottom metal plate, the binocular camera (5) is installed on the center line of the bottom metal plate and behind the event camera (4), the depth camera (6) is installed on the center line of the bottom metal plate and below the event camera (4), the fisheye camera (7) is installed on the left front, left rear, right front, right rear and rear center lines of the middle metal plate, and the inertial measurement unit (8) is installed on the left front, left rear, right front, right rear and rear center lines of the middle metal plate. The unit (8) is closely mounted on the center line of the middle metal plate, the single-line laser radar (9) is placed on the center line of the middle metal plate and supported at a certain height by metal studs, the microphone array (10) is mounted on the center line of the middle metal plate, the GNSS-RTK positioning module (11) is mounted on the top metal plate and supported at a certain height by metal studs, the UWB positioning module (12) is mounted on the top metal plate and supported at a certain height by metal studs, the environmental sensor (13) is mounted on the center line of the top metal plate, and the monocular camera (14) pointing to the sky is mounted on the center line of the top metal plate. The data collected by each sensor is synchronized in time through a timestamp, and spatial synchronization is performed by recording the position information of the sensor.

4. The multimodal ground data acquisition unmanned vehicle platform according to claim 1, characterized in that: The control unit used by the control module is a NUC (15), which is connected to the multi-sensor integrated platform and installed on the rear side of the bottom metal plate, and is used for controlling, storing and transmitting data of the multi-sensor integrated platform.

5. The multi-modal ground data acquisition unmanned vehicle platform according to claim 1, characterized in that: The battery used in the power supply device is an outdoor battery (16) to ensure sufficient voltage and endurance required for the operation of the multi-sensor integrated platform and the control module. The power supply device is connected to the control module and is installed on the bottom metal plate to supply power to the control module and the sensor.

6. The multi-modal ground data acquisition unmanned vehicle platform according to claim 3, characterized in that: The chassis of the unmanned remote-controlled electric vehicle is an Ackerman structure, with a size of 68*42*76cm.

7. The multi-modal ground data acquisition unmanned vehicle platform according to claim 3, characterized in that: The microphone array is a six-microphone circular array installed on the middle metal plate.

8. The multi-modal ground data acquisition unmanned vehicle platform according to claim 3, characterized in that: The GNSS-RTK module has two antennas on the mobile carrier platform to realize communication between the mobile terminal and the satellite, and cooperates with the base station to realize differential RTK positioning.

9. The multi-modal ground data acquisition unmanned vehicle platform according to claim 3, characterized in that: The UWB positioning module has two antennas on the mobile carrier platform to send and receive UWB signals, and is matched with the base station to realize UWB ranging using the two-way time-of-flight method.

10. The multi-modal ground data acquisition unmanned vehicle platform according to claim 3, characterized in that: The environmental sensor integrates temperature, humidity, PM1.0, PM2.5, PM10, atmospheric pressure, light and noise sensors, and is installed on the top metal plate to obtain richer information about the surrounding environment. The monocular camera pointing to the sky is installed on the top metal plate.

Images