A multi-radar fusion perception system and method for large vehicles

Abstract

The present invention relates to the technical field of intelligent driving perception systems, and provides a multi-radar fusion perception system and method for large vehicles. The method includes: storing the data of each sensor in a domain controller to establish a cache area; regularly reading the data in the cache area to obtain synchronized data; and inputting it into the perception software system module; extracting boundaries from the laser point cloud data, performing point cloud clustering on obstacle points and suspected obstacle points; and constructing obstacle information; fusing the traditional laser radar module with the laser deep learning module to obtain the perception results and the millimeter wave radar perception results, and fusing the two to obtain the final detection results. The present invention ensures the accuracy of the perception system, reduces the impact of the system on the harsh environment, improves the accuracy and recall rate of obstacle detection, and improves the detection reliability.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent driving perception systems, and in particular to a multi-radar fusion perception system and method for large vehicles. Background Art

[0002] The realization of intelligent driving is inseparable from the detection of surrounding environment information by the perception system. An accurate perception system plays a decisive role in the functional realization and functional safety of intelligent driving. The perception system first obtains the surrounding environment data of the equipment through detection equipment such as laser radar, millimeter wave radar, camera, and ultrasound. Then, the corresponding perception algorithm is used to fuse the environmental data of multiple sensors and obtain key information such as pedestrians, vehicles, and road boundaries in the environmental data.

[0003] Among the detection equipment in the perception system, the laser radar has high detection accuracy and a large detection range, but is easily affected by dust, rain, fog, and snow. The millimeter-wave radar is less affected by dust, rain, fog, and snow, but has low detection accuracy. The camera has high accuracy for environmental attributes and 2D detection, but low 3D position detection, and is easily affected by bad weather such as light, dust, rain, fog, and snow. Therefore, a system that integrates multiple sensors is necessary for the stability and accuracy of detection.

[0004] Intelligent driving has developed rapidly and has a more complete theoretical system in small cars, but large vehicles are large in size, require a large driving area, and have large blind spots to consider; large vehicles have harsh operating scenarios and often need to operate in rain, snow, fog, and large dusty environments; large vehicles have slow power response and are easily affected by potholes, road slopes, and stones. All these have raised the requirements for the perception system. Summary of the invention

[0005] In view of this, the present invention provides a large vehicle multi-radar fusion perception system and method to solve the problems in the prior art of solving the blind spots of large vehicle detection, coping with the interference of rain, snow, fog and heavy dust, and meeting the detection of special obstacles when large vehicles operate on unstructured roads.

[0006] A first aspect of the present invention provides a large vehicle multi-radar fusion perception system, comprising:

[0007] Hardware systems and software systems,

[0008] The hardware system includes a laser radar system, a millimeter wave radar system, a GPS-IMU module, a domain controller and a power module. The software system includes a driver module and a core algorithm module. The software system is embedded in the hardware system.

[0009] The laser radar system is distributed in four directions of the vehicle, and is composed of one laser radar or a combination of multiple laser radars at different positions and angles, and is used to detect the entire three-dimensional space in the four directions of the vehicle by setting the first arrangement condition, and feed back the obtained laser radar system detection data to the domain controller;

[0010] The millimeter wave radar system is composed of two or more millimeter wave radars, and is used to detect the front and rear directions of the vehicle, and feeds back the obtained millimeter wave radar system detection data to the domain controller based on the set arrangement condition 2;

[0011] The GPS-IMU module is used to provide data on the location and heading angle of the vehicle and data on the vehicle body posture, and feed back the GPS-IMU module detection data to the domain controller;

[0012] The domain controller synchronously processes the received detection data of the laser radar system and the millimeter wave radar system, the data of the location and heading angle of the vehicle, and the data of the vehicle body posture to realize obstacle detection, and outputs the processing results to other modules outside the perception system, wherein the other modules include a control execution module and a platform management module;

[0013] The power module is used to provide power for the laser radar system, the millimeter wave radar system, the GPS-IMU module and the domain controller;

[0014] The driving system includes a data analysis module, a data time synchronization module and a data space synchronization module of each sensor, wherein each sensor includes each laser radar, each millimeter wave radar and a GPS-IMU module;

[0015] The data time synchronization module uses the same time reference to add timestamp information to the data from various sensors at different times, wherein the various sensors include a laser radar system, a millimeter wave radar system and a GPS-IMU module;

[0016] The data space synchronization module is used to calibrate the parameters of the detection data of each sensor to the center of the rear axle of the vehicle, and convert the coordinate center of each sensor data to the center of the rear axle of the vehicle;

[0017] The data analysis module analyzes the detection data of each sensor and puts them into their respective buffer areas. The data time synchronization module periodically reads the buffer areas of each sensor to obtain data with a set time matching degree. The data space synchronization module converts the data coordinate system of each sensor to a unified standard and inputs the synchronized data into the core algorithm module respectively.

[0018] Furthermore, the arrangement condition 1 includes:

[0019] Forward LiDAR arrangement: 1) Short-distance area: The horizontal field of view angle coverage is not less than 180 degrees, the vertical field of view angle is not less than 90 degrees, and the coverage distance is not less than 20 meters horizontally and vertically; it is covered by one LiDAR, or several LiDARs are used to perform coverage detection through installation combinations at different positions or angles; 2) Long-distance area: The horizontal field of view angle coverage is not less than 120 degrees, the vertical field of view angle is not less than 20 degrees, and the coverage area is not less than 100 meters vertically. It is covered by one LiDAR, or several LiDARs are used to perform coverage detection through installation combinations at different positions or angles;

[0020] Lateral LiDAR arrangement: The horizontal field of view of the LiDARs on the left and right sides of the vehicle shall be no less than 180 degrees, the vertical field of view shall be no less than 90 degrees, and the coverage distance shall be no less than 20 meters in the horizontal and vertical directions. Coverage detection shall be performed by one LiDAR, or by using several LiDARs installed at different positions or angles.

[0021] Rear-facing laser radar arrangement: The horizontal field of view of the rear-facing laser radar is not less than 180 degrees, the vertical field of view is not less than 90 degrees, and the coverage distance is not less than 20 meters horizontally and vertically. Coverage detection is performed by one laser radar, or several laser radars are used to perform coverage detection through installation combinations at different positions or angles.

[0022] Furthermore, the second arrangement condition is set, including:

[0023] Coverage detection is performed by a millimeter-wave radar with a set horizontal viewing angle, or by using several millimeter-wave radars installed in different positions or angles to achieve a front horizontal detection angle of not less than 180 degrees and a detection distance of not less than 100 meters; the rear horizontal angle is greater than or equal to 120, and the detection distance is not less than 40 meters.

[0024] Furthermore, the laser radar system is a system in which each laser radar transmits the detection data of the laser radar system to the domain controller via a network cable or an in-vehicle Ethernet;

[0025] The millimeter wave radar system is that each millimeter wave radar transmits the detection data of the millimeter wave radar system to the domain controller through a CAN line, a network cable or an in-vehicle Ethernet;

[0026] The GPS-IMU module connects the data of the vehicle's position and heading angle as well as the vehicle body posture to the domain controller module via a data line.

[0027] Furthermore, the core algorithm module includes a traditional laser radar module, a laser deep learning module, and a sensor data fusion module.

[0028] The traditional laser radar module and the laser deep learning module process the input laser radar point cloud, and fuse the traditional laser radar module and the laser deep learning module to obtain a perception result. The millimeter wave radar detection module processes the input millimeter wave radar perception result. The sensor data fusion module fuses the perception result and the millimeter wave radar perception result to obtain a final detection result.

[0029] Furthermore, the traditional module of the laser radar includes: boundary extraction of laser point cloud, point cloud ROI extraction, ground filtering, rain, snow, fog and dust detection, point cloud clustering, obstacle construction, and obstacle tracking.

[0030] The boundary extraction of the laser point cloud includes extracting the point cloud within the boundary directly from the point cloud information based on the boundary information, or extracting the point cloud within the boundary from the prior map based on the boundary information and the positioning information provided by the GPS-IMU module 3;

[0031] The ground filtering is used to filter ground points from the point cloud within the extraction boundary, convert the filtered ground points into a grid ground information list, determine whether the grid ground information belongs to an obstacle, ground or a pit and record the category;

[0032] The rain, snow, fog and dust detection is used to filter out noise points interfered by rain, snow, fog and dust, and the filtered information includes confirmed obstacle point clouds and suspected obstacle point clouds, which are subsequently verified with the millimeter wave radar.

[0033] Furthermore, the laser deep learning module is used to regress one or more point cloud target detection models to the position, size, and heading of obstacles in the laser radar point cloud.

[0034] Furthermore, the sensor data fusion module includes a laser radar fusion module, a millimeter wave radar fusion module, and a millimeter wave radar and laser radar fusion module.

[0035] The laser radar fusion module is used to fuse the obstacle information determined by the output of the traditional laser radar module and the output result of the laser deep learning module;

[0036] The millimeter wave radar fusion module is used to fuse multiple millimeter wave radar detection results;

[0037] The millimeter-wave radar and lidar fusion module is used to further fuse the results of the lidar fusion module and the millimeter-wave radar fusion module. The determined obstacle information and the fusion result of the laser deep learning module are directly fused. The information of suspected obstacles is verified by the millimeter-wave radar fusion result. The objects detected by both are identified as obstacles at the same position, and the results are output. Target information with only lidar but no millimeter-wave radar is identified as false targets.

[0038] Furthermore, the data time synchronization module sorts the working cycles of the sensors from long to short, selects the first one as the main sensor, and when the first sensor fails, selects the next sensor as the main sensor for synchronization.

[0039] A second aspect of the present invention provides a large vehicle multi-radar fusion perception method, comprising:

[0040] Step 1: Store the data of each laser radar, each millimeter wave radar and GPS-IMU module into the domain controller to establish a cache area;

[0041] Step 2: The data time synchronization module periodically reads the buffer area, obtains data with a set time matching degree, converts the data coordinate system of each sensor to a unified standard, and obtains synchronized data;

[0042] The present invention performs master sensor time prediction on the same time base for synchronized data, thereby greatly reducing the differences and separation of data among multiple sensors.

[0043] Step 3: input the synchronization data into the perception software system module respectively;

[0044] Step 4: The traditional laser radar module extracts the boundary based on the input laser point cloud data, and extracts the laser radar point cloud of ROI based on the boundary; the ROI point cloud is ground filtered, and the filtered points are divided into ground points and non-ground points. The ground points construct a grid ground information list, and the grid size is set as required. The obstacle points and suspected obstacle points are respectively subjected to point cloud clustering, obstacle construction, and obstacle tracking, where ROI, the full spelling of region of interest, is the region of interest;

[0045] Step 5: The laser deep learning module processes the input laser radar point cloud and constructs obstacle information;

[0046] Step 6: The sensor fusion module fuses the traditional laser radar module and the laser deep learning module to obtain the perception result;

[0047] The present invention improves the accuracy and recall rate of obstacle detection by simultaneously integrating the detection of millimeter-wave radar and lidar.

[0048] Step 7: At the same time, the sensor fusion module processes the millimeter wave radar detection module to obtain the millimeter wave radar perception result;

[0049] In Step 8, the sensor fusion module fuses the perception result with the millimeter-wave radar perception result to obtain the final detection result.

[0050] Compared with the prior art, the present invention has the following beneficial effects:

[0051] 1. The system equipment is built to provide 360-degree coverage of large vehicles, and it also provides comprehensive coverage of the three-dimensional space outside the vehicle, not just the top-down coverage. It is an effective solution to deal with the blind spots and dead angles of large vehicles. It has the ability to detect long- and short-distance piles, potholes, stones, pedestrians, and vehicles.

[0052] 2. This system uses a combination of two sensors with different characteristics: a high-precision laser radar and a millimeter-wave radar with strong anti-interference properties. The design of the perception system solution can not only ensure the accuracy of the perception system, but also reduce the impact of the system on the harsh environment of rain, snow, fog and dust.

[0053] 3. The time synchronization adopted by this system completes the synchronization before the data is generated and directly obtains the data after it is generated, thus ensuring real-time performance. In the process of synchronizing data, the comprehensive maximum overlap of the real scene of the data is guaranteed, and the synchronized data is predicted with the same time reference (main sensor time), which greatly reduces the difference and separation of data among multiple sensors.

[0054] 4. This system uses a variety of methods to process lidar point clouds, such as traditional lidar modules and laser deep learning modules, and integrates millimeter-wave radar and lidar detection to improve the accuracy and recall rate of obstacle detection.

[0055] 5. The grid ground information established by this system in the laser point cloud processing is a good environmental marker. It can well mark the location of surrounding obstacles, potholes and road boundaries, as well as the height and slope information of the surrounding ground, and can well measure the height of obstacles without being affected by the road slope.

[0056] 6. This system uses the rain, snow, fog and dust detection module to distinguish the laser point cloud, and tracks and fuses the confirmed obstacles and suspected obstacles separately to improve the detection reliability. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions in the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0058] Figure 1 It is a schematic diagram of a large vehicle multi-radar fusion perception system provided by the present invention;

[0059] Figure 2It is a flow chart of a multi-radar fusion perception method for large vehicles provided by the present invention;

[0060] Figure 3 It is a schematic diagram of the core algorithm processing of a large vehicle multi-radar fusion perception system provided by the present invention. DETAILED DESCRIPTION

[0061] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present invention. However, it should be clear to those skilled in the art that the present invention may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present invention.

[0062] A large vehicle multi-radar fusion perception system and method according to the present invention will be described in detail below with reference to the accompanying drawings.

[0063] Figure 1 Schematic diagram of a multi-radar fusion perception system for large vehicles provided by the present invention. Figure 1 As shown, the multi-radar fusion perception system includes:

[0064] Hardware systems and software systems,

[0065] The hardware system includes the LiDAR system, millimeter-wave radar system, GPS-IMU module, domain controller and power module. The software system includes the driver module and core algorithm module. The software system is embedded in the hardware system.

[0066] The laser radar system is distributed in four directions of the vehicle and is composed of one laser radar or a combination of multiple laser radars at different positions and angles. It is used to detect the entire three-dimensional space in the four directions of the vehicle by setting the layout condition one, and feed back the obtained laser radar system detection data to the domain controller;

[0067] The present invention provides 360-degree viewing of large vehicles. 0 It not only provides comprehensive coverage, but also comprehensive coverage of the three-dimensional space outside the vehicle, not just coverage of the overhead plane, which is more effective in dealing with the blind spots and dead angles of large vehicles.

[0068] Arrangement Condition 1 includes:

[0069] Forward LiDAR arrangement: 1) Short-distance area: The horizontal field of view angle coverage is not less than 180 degrees, the vertical field of view angle is not less than 90 degrees, and the coverage distance is not less than 20 meters horizontally and vertically; it is covered by one LiDAR, or several LiDARs are used to perform coverage detection through installation combinations at different positions or angles; 2) Long-distance area: The horizontal field of view angle coverage is not less than 120 degrees, the vertical field of view angle is not less than 20 degrees, and the coverage area is not less than 100 meters vertically. It is covered by one LiDAR, or several LiDARs are used to perform coverage detection through installation combinations at different positions or angles;

[0070] Lateral LiDAR arrangement: The horizontal field of view of the LiDARs on the left and right sides of the vehicle shall be no less than 180 degrees, the vertical field of view shall be no less than 90 degrees, and the coverage distance shall be no less than 20 meters in the horizontal and vertical directions. Coverage detection shall be performed by one LiDAR, or by using several LiDARs installed at different positions or angles.

[0071] Rear-facing laser radar arrangement: The horizontal field of view of the rear-facing laser radar is not less than 180 degrees, the vertical field of view is not less than 90 degrees, and the coverage distance is not less than 20 meters horizontally and vertically. Coverage detection is performed by one laser radar, or several laser radars are used to perform coverage detection through installation combinations at different positions or angles.

[0072] Prioritize the use of a laser radar that meets the set conditions for coverage detection.

[0073] The millimeter wave radar system is composed of two or more millimeter wave radars, and is used to detect the front and rear directions of the vehicle, and feeds back the obtained millimeter wave radar system detection data to the domain controller based on the set arrangement condition 2;

[0074] Set the second arrangement condition, including:

[0075] Coverage detection is performed by a millimeter-wave radar with a set horizontal viewing angle, or by using several millimeter-wave radars installed in different positions or angles to achieve a front horizontal detection angle of not less than 180 degrees and a detection distance of not less than 100 meters; the rear horizontal angle is greater than or equal to 120, and the detection distance is not less than 40 meters.

[0076] A millimeter-wave radar that meets the set conditions is given priority for coverage detection.

[0077] The GPS-IMU module is used to provide data on the vehicle's location, heading angle, and body posture, and to feed back the GPS-IMU module detection data to the domain controller;

[0078] The data of the vehicle's location include latitude and longitude data, and the data of the heading angle include the angle data between the front of the vehicle and the north direction.

[0079] The domain controller synchronously processes the received laser radar system detection data and millimeter wave radar system detection data, the vehicle's location and heading angle data, and the body posture data to realize obstacle detection, and outputs the processing results to other modules outside the perception system. Among them, other modules include control execution module and platform management module;

[0080] Power module, used to provide power for the LiDAR system, millimeter wave radar system, GPS-IMU module and domain controller;

[0081] Among them, the power module provides direct current of different voltages such as 24v and 12v.

[0082] The driving system includes a data analysis module, a data time synchronization module and a data space synchronization module of each sensor, wherein each sensor includes each laser radar, each millimeter wave radar and a GPS-IMU module;

[0083] The data time synchronization module uses the same time reference to add timestamp information to the data from various sensors at different times. The sensors include the lidar system, millimeter wave radar system and GPS-IMU module.

[0084] The data space synchronization module is used to calibrate the parameters of the detection data of each sensor to the center of the rear axle of the vehicle, and convert the coordinate center of each sensor data to the center of the rear axle of the vehicle;

[0085] The data analysis module analyzes the detection data of each sensor and puts them into their respective buffers. The data time synchronization module periodically reads the buffers of each sensor to obtain data with a set time matching degree. The data space synchronization module converts the data coordinate system of each sensor to a unified standard and inputs the synchronized data into the core algorithm module.

[0086] The data time synchronization module sorts the working cycles of each sensor from long to short, and selects the first one as the main sensor. When the first sensor fails, the next sensor is selected as the main sensor for synchronization.

[0087] After synchronization is completed, wait for the data to arrive, and directly obtain the data after it arrives.

[0088] Core algorithm modules, including traditional LiDAR module, laser deep learning module, sensor data fusion module,

[0089] The traditional lidar module and the laser deep learning module process the input lidar point cloud and fuse the traditional lidar module and the laser deep learning module to obtain the perception result. The millimeter wave radar detection module processes the input millimeter wave radar perception result. The sensor data fusion module fuses the perception result and the millimeter wave radar perception result to obtain the final detection result.

[0090] Traditional LiDAR modules: including laser point cloud boundary extraction, point cloud ROI extraction, ground filtering, rain, snow, fog and dust detection, point cloud clustering, obstacle construction and obstacle tracking.

[0091] Boundary extraction of the laser point cloud, including extracting the point cloud within the boundary directly from the point cloud information based on the boundary information, or extracting the point cloud within the boundary from the prior map based on the boundary information and the positioning information provided by the GPS-IMU module 3;

[0092] Ground filtering is used to filter ground points from the point cloud within the extraction boundary, convert the filtered ground points into a grid ground information list, determine whether the grid ground information belongs to obstacles, ground or pits, and record the category;

[0093] Rain, snow, fog and dust detection is used to filter out noise points caused by rain, snow, fog and dust. The filtered information includes confirmed obstacle point clouds and suspected obstacle point clouds, which are then verified with the millimeter-wave radar.

[0094] The laser deep learning module is used to regress one or more point cloud object detection models to the position, size, and heading of obstacles in the lidar point cloud.

[0095] Sensor data fusion module: including lidar fusion module, millimeter-wave radar fusion module, and millimeter-wave radar and lidar fusion module.

[0096] The laser radar fusion module is used to fuse the obstacle information determined by the output of the traditional laser radar module and the output results of the laser deep learning module;

[0097] Millimeter-wave radar fusion module, used to fuse the detection results of multiple millimeter-wave radars;

[0098] The millimeter-wave radar and lidar fusion module is used to further fuse the results of the lidar fusion module and the millimeter-wave radar fusion module. The determined obstacle information is directly integrated with the fusion result of the laser deep learning module. The information of suspected obstacles is verified by the millimeter-wave radar fusion result. Objects detected by both are identified as obstacles at the same position, and the results are output. Target information with only lidar but no millimeter-wave radar is identified as false targets.

[0099] Based on the same concept, the present invention also provides a large vehicle multi-radar fusion perception method, comprising:

[0100] Step 1: Store the data of each laser radar, each millimeter wave radar and GPS-IMU module into the domain controller to establish a cache area;

[0101] Step 2 The data time synchronization module reads the buffer area regularly, obtains the data with the set time matching degree, converts the data coordinate system of each sensor to a unified standard, and obtains the synchronized data;

[0102] Step 3: Input the synchronization data into the perception software system module respectively;

[0103] Step 4 The traditional laser radar module extracts boundaries based on the incoming laser point cloud data, and extracts the laser radar point cloud of ROI based on the boundaries; the ground is filtered out for the ROI point cloud, and the filtered points are divided into ground points and non-ground points. The ground points build a grid ground information list, and the grid size is set as required. Point cloud clustering, obstacle construction, and obstacle tracking are performed on obstacle points and suspected obstacle points respectively; ROI, the full spelling of region of interest, is the area of ​​interest.

[0104] For example, it is set to (0.3X0.3 meters or 0.5X0.5 meters); the non-ground points are detected by rain, snow, fog and dust, the determined noise points are filtered out, and the remaining laser point cloud is divided into obstacle points and suspected obstacle points.

[0105] Step 5: The laser deep learning module processes the input laser radar point cloud and constructs obstacle information;

[0106] Step 6: The sensor fusion module integrates the traditional LiDAR module and the laser deep learning module to obtain the perception result;

[0107] Step 7: At the same time, the sensor fusion module processes the millimeter-wave radar detection module to obtain the millimeter-wave radar perception result;

[0108] Step 8 The sensor fusion module fuses the perception results with the millimeter-wave radar perception results to obtain the final detection results.

[0109] The present invention provides 360-degree viewing of large vehicles. 0 It not only covers the entire space outside the vehicle, but also covers the entire space outside the vehicle. It is not just the coverage of the top-down plane. It is an effective solution to deal with the blind spots and dead angles of large vehicles. It has the ability to detect obstacles, pedestrians, and vehicles at long and short distances.

[0110] The present invention adopts a combination of two sensors with different characteristics, a high-precision laser radar and a millimeter-wave radar with strong anti-interference, to ensure the accuracy of the perception system and reduce the impact of the system on the harsh environment of rain, snow, fog and dust; the time synchronization adopted completes the synchronization before the data is generated, and the data is directly acquired after it is generated, which ensures real-time performance and the comprehensive maximum overlap of the real scene of the data; at the same time, the detection of millimeter-wave radar and laser radar is integrated to improve the accuracy and recall rate of obstacle detection, and can well measure the height of the obstacle without being affected by the road slope. The confirmed obstacles and suspected obstacles are tracked and integrated respectively, thereby improving the detection reliability.

[0111] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present application, which will not be described one by one here.

[0112] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0113] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention.

Claims

1. A large vehicle multi-radar fusion perception system, characterized in that: include: Hardware systems and software systems, The hardware system includes a laser radar system, a millimeter wave radar system, a GPS-IMU module, a domain controller and a power module. The software system includes a driver module and a core algorithm module. The software system is embedded in the hardware system. The laser radar system is distributed in four directions of the vehicle, and is composed of one laser radar or a combination of multiple laser radars at different positions and angles, and is used to detect the entire three-dimensional space in the four directions of the vehicle by setting the first arrangement condition, and feed back the obtained laser radar system detection data to the domain controller; The millimeter wave radar system is composed of two or more millimeter wave radars, and is used to detect the front and rear directions of the vehicle, and feeds back the obtained millimeter wave radar system detection data to the domain controller based on the set arrangement condition 2; The GPS-IMU module is used to provide data on the location and heading angle of the vehicle and data on the vehicle body posture, and feed back the GPS-IMU module detection data to the domain controller; The domain controller synchronously processes the received detection data of the laser radar system and the millimeter wave radar system, the data of the location and heading angle of the vehicle, and the data of the vehicle body posture to realize obstacle detection, and outputs the processing results to other modules outside the perception system, wherein the other modules include a control execution module and a platform management module; The power module is used to provide power for the laser radar system, the millimeter wave radar system, the GPS-IMU module and the domain controller; The driving module includes a data analysis module, a data time synchronization module and a data space synchronization module of each sensor, wherein each sensor includes each laser radar, each millimeter wave radar and a GPS-IMU module; The data time synchronization module uses the same time reference to add timestamp information to the data from various sensors at different times, wherein the various sensors include a laser radar system, a millimeter wave radar system and a GPS-IMU module; The data space synchronization module is used to calibrate the parameters of the detection data of each sensor to the center of the rear axle of the vehicle, and convert the coordinate center of each sensor data to the center of the rear axle of the vehicle; The data analysis module analyzes the detection data of each sensor and puts them into their respective buffers. The data time synchronization module periodically reads the buffers of each sensor to obtain data with a set time matching degree. The data space synchronization module converts the data coordinate system of each sensor to a unified standard and inputs the synchronized data into the core algorithm module. The core algorithm modules include traditional laser radar module, laser deep learning module and sensor data fusion module. The traditional laser radar module and the laser deep learning module process the input laser radar point cloud, and fuse the traditional laser radar module and the laser deep learning module to obtain a perception result. The millimeter wave radar system processes the input millimeter wave radar perception result, and the sensor data fusion module fuses the perception result and the millimeter wave radar perception result to obtain a final detection result.

2. The multi-radar fusion perception system according to claim 1, characterized in that: The arrangement condition 1 includes: Forward LiDAR arrangement: 1) Short-distance area: horizontal field of view angle coverage is not less than 180 degrees, vertical field of view angle is not less than 90 degrees, and coverage distance is not less than 20 meters horizontally and vertically; one LiDAR is used for coverage detection, or several LiDARs are used to perform coverage detection through installation combinations at different positions or angles; 2) Long-distance area: horizontal field of view angle coverage is not less than 120 degrees, vertical field of view angle is not less than 20 degrees, and coverage area is not less than 100 meters vertically; one LiDAR is used for coverage detection, or several LiDARs are used to perform coverage detection through installation combinations at different positions or angles; Lateral LiDAR arrangement: The horizontal field of view of the LiDARs on the left and right sides of the vehicle shall be no less than 180 degrees, the vertical field of view shall be no less than 90 degrees, and the coverage distance shall be no less than 20 meters in the horizontal and vertical directions. Coverage detection shall be performed by one LiDAR, or by using several LiDARs installed at different positions or angles. Rear-facing laser radar arrangement: The horizontal field of view of the rear-facing laser radar is not less than 180 degrees, the vertical field of view is not less than 90 degrees, and the coverage distance is not less than 20 meters horizontally and vertically. Coverage detection is performed by one laser radar, or several laser radars are used to perform coverage detection through installation combinations at different positions or angles.

3. The multi-radar fusion perception system according to claim 1, characterized in that: The second setting arrangement condition includes: Coverage detection is performed by a millimeter-wave radar with a set horizontal viewing angle, or by using several millimeter-wave radars installed in different positions or angles to achieve a front horizontal detection angle of not less than 180 degrees and a detection distance of not less than 100 meters; the rear horizontal angle is greater than or equal to 120, and the detection distance is not less than 40 meters.

4. The multi-radar fusion perception system according to claim 1, characterized in that: The laser radar system is a system in which each laser radar transmits the detection data of the laser radar system to the domain controller through a network cable or an in-vehicle Ethernet; The millimeter wave radar system is that each millimeter wave radar transmits the detection data of the millimeter wave radar system to the domain controller through a CAN line, a network cable or an in-vehicle Ethernet; The GPS-IMU module connects the data of the vehicle's position and heading angle as well as the vehicle body posture to the domain controller module via a data line.

5. The multi-radar fusion perception system according to claim 1, characterized in that: The traditional modules of the laser radar include: boundary extraction of laser point cloud, point cloud ROI extraction, ground filtering, rain, snow, fog and dust detection, point cloud clustering, obstacle construction and obstacle tracking; The boundary extraction of the laser point cloud includes extracting the point cloud within the boundary directly from the point cloud information based on the boundary information, or extracting the point cloud within the boundary from the prior map based on the boundary information and the positioning information provided by the GPS-IMU module; The ground filtering is used to filter ground points from the point cloud within the extraction boundary, convert the filtered ground points into a grid ground information list, determine whether the grid ground information belongs to an obstacle, ground or a pit, and record it; The rain, snow, fog and dust detection is used to filter out noise points interfered by rain, snow, fog and dust, and the filtered information includes confirmed obstacle point clouds and suspected obstacle point clouds, which are subsequently verified with the millimeter wave radar.

6. The multi-radar fusion perception system according to claim 1, characterized in that: The laser deep learning module is used to regress one or more point cloud target detection models to the position, size, and heading of obstacles in the laser radar point cloud.

7. The multi-radar fusion perception system according to claim 1, characterized in that: The sensor data fusion module includes a laser radar fusion module, a millimeter wave radar fusion module, and a millimeter wave radar and laser radar fusion module. The laser radar fusion module is used to fuse the obstacle information determined by the output of the traditional laser radar module and the output result of the laser deep learning module; The millimeter wave radar fusion module is used to fuse multiple millimeter wave radar detection results; The millimeter-wave radar and lidar fusion module is used to further fuse the results of the lidar fusion module and the millimeter-wave radar fusion module. The determined obstacle information and the fusion result of the laser deep learning module are directly fused. The information of suspected obstacles is verified by the millimeter-wave radar fusion result. The objects detected by both are identified as obstacles at the same position, and the results are output. Target information with only lidar but no millimeter-wave radar is identified as false targets.

8. The multi-radar fusion perception system according to claim 1, characterized in that: The data time synchronization module sorts the working cycles of the sensors from long to short, selects the first one as the main sensor, and when the first sensor fails, selects the next sensor as the main sensor for synchronization.

9. The perception method of the multi-radar fusion perception system according to claim 5, comprising: Step 1: Store the data of each laser radar, each millimeter wave radar and GPS-IMU module into the domain controller to establish a cache area; Step 2: The data time synchronization module periodically reads the buffer area, obtains data with a set time matching degree, converts the data coordinate system of each sensor to a unified standard, and obtains synchronized data; The master sensor time prediction is performed on the synchronized data with the same time base, which greatly reduces the difference and separation of data between multiple sensors; Step 3: input the synchronization data into the perception software system module respectively; Step 4: The traditional laser radar module extracts the boundary based on the input laser point cloud data, and extracts the laser radar point cloud of ROI based on the boundary; the ROI point cloud is ground filtered, and the filtered points are divided into ground points and non-ground points. The ground points construct a grid ground information list, and the grid size is set as required. The obstacle points and suspected obstacle point clouds are respectively subjected to point cloud clustering, obstacle construction, and obstacle tracking, where ROI, the full spelling of region of interest, is the region of interest; Step 5: The laser deep learning module processes the input laser radar point cloud and constructs obstacle information; Step 6: The sensor fusion module fuses the traditional laser radar module and the laser deep learning module to obtain the perception result; By integrating the detection of millimeter-wave radar and lidar, the accuracy and recall rate of obstacle detection are improved; Step 7: At the same time, the sensor fusion module processes the millimeter wave radar detection module to obtain the millimeter wave radar perception result; In Step 8, the sensor data fusion module fuses the perception result with the millimeter-wave radar perception result to obtain the final detection result.

Images