Multi-sensing information fusion three-dimensional road information acquisition system

Through the multi-sensor information fusion system, combined with sensors such as RTK, camera, color sensor and lidar, the accuracy and reliability of road information collection in complex traffic environments are solved, and high-precision measurement and collection of three-dimensional roads are realized, eliminating the elimination of INS cumulative errors provide accurate road boundary and traffic sign line information.

CN119063743APending Publication Date: 2024-12-03CHINA UNIV OF GEOSCIENCES (BEIJING)
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202411171324.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-23
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The existing technology is difficult to achieve high-precision and high-reliability road information collection in complex traffic environments, especially the three-dimensional road information collection, absolute road coordinate imperfection, lack of visual and laser point cloud fusion, and INS-based road positioning The problem of cumulative error.

Method used

Adopt multi-sensor information fusion system, including RTK, camera, color sensor, lidar, IMU measurement unit, industrial control machine and lithium battery pack, and receives INS inertial navigation information, camera visual detection, landmark lines collected by color sensors and GNSS High-precision position information, data processing and fusion, a solid road database is established, and laser point clouds are used to refine road information to correct and supplement the data in the database.

Benefits of technology

High-precision measurement and acquisition of three-dimensional roads are realized, and the road boundaries and traffic sign lines are accurately positioned through color sensors, the laser point clouds and visual characteristics are integrated, and the INS cumulative errors are eliminated, which improves the accuracy and reliability of road information.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119063743A_ABST
    Figure CN119063743A_ABST
Patent Text Reader

Abstract

The invention discloses a multi-sensor information fusion three-dimensional road information acquisition system, which comprises an RTK, a camera, a color sensor, a laser radar, an IMU measurement unit, an industrial personal computer, a control panel and a lithium battery pack, and is characterized in that observation data is acquired through color, vision and INS, multi-sensor data fusion is carried out, and the real-time information of a road is acquired. The landmark extraction, the road information analysis, the lateral deviation analysis and the advancing reverse deviation analysis are realized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of multi-sensor fusion, and in particular to a three-dimensional road information collection system integrating multi-sensor information. Background Art

[0002] With the rapid development of intelligent transportation systems, the demand for accurate and real-time road information collection is growing. Traditional road information collection usually relies on a single type of sensor, which is difficult to meet the requirements of high accuracy and high reliability in complex traffic environments.

[0003] Multi-sensor information fusion technology effectively overcomes these limitations and achieves higher accuracy by integrating data from multiple sensors. Three-dimensional road information requires the collection of information about the road surface as well as environmental information on both sides and above the road, such as traffic signs. This three-dimensional information is crucial for implementing driver assistance systems and autonomous driving technologies.

[0004] Currently, multi-sensor fusion and three-dimensional road construction have the following problems:

[0005] Road construction incorporates absolute road coordinates but incomplete relative road information. It can be used for navigation, but is not yet perfect for road exploration and autonomous driving.

[0006] The road construction of vision and laser point cloud is not further integrated with the refined road feature sensing component;

[0007] The INS-based road positioning method has cumulative errors. Summary of the Invention

[0008] The purpose of the present invention is to provide a three-dimensional road information collection system with multi-sensor information fusion to solve the problems in the above-mentioned background technology.

[0009] To achieve the above objectives, the present invention provides a three-dimensional road information acquisition system with multi-sensor information fusion, including RTK, camera, color sensor, lidar, IMU measurement unit, industrial computer, control panel and lithium battery pack. The RTK, camera, color sensor, lidar, IMU inertial measurement unit, control panel and lithium battery pack are all connected to the industrial computer, and the RTK is connected to the GNSS antenna.

[0010] Preferably, the implementation method includes:

[0011] S1, receives INS inertial navigation information, landmark and road information detected by camera vision, landmark and road edge information collected by color sensor, and GNSS-based RTK high-precision position information, processes the received data, and obtains three-dimensional road center coordinate positioning information;

[0012] S2, receiving road information collected by the LiDAR and refining the road information viewpoint cloud;

[0013] S3: Fusing camera visual detection information with laser point cloud to construct a three-dimensional road database;

[0014] S4. Compare the standard road information obtained by using the RTK high-precision positioning information of GNSS with the 3D road database. If there is a mismatch, correct the 3D road information. If there is a match, supplement the 3D road database based on the center coordinate positioning information.

[0015] Preferably, processing the received data in S1 includes:

[0016] S11, screening the received data;

[0017] S12, pre-processing the filtered data, correlating the camera visual data and the color sensor data to obtain the travel direction and lateral direction reference;

[0018] S13: Use RTK high-precision position information to determine the effective value of the travel direction and lateral direction reference, which can eliminate the cumulative error of INS but does not meet the requirements of cyclic detection.

[0019] S14. Based on the IMU measurement unit, the INS data queue in the INS inertial navigation data is collected, and the INS data queue is aligned to the integration interval through differential interpolation. INS pre-integration is performed to obtain the visual data queue corrected for the relative position change between epochs. The random sampling consistency method is used to calculate the basic matrix and homography matrix, and the external points in the visual observation are eliminated. The geometric transformation analysis is performed to obtain landmark extraction and road information analysis.

[0020] Preferably, point cloud refinement in S2 includes extracting lidar planes and edge points, parameterizing them, and calculating different line-line distances and surface-surface distances to match line-surface features.

[0021] Preferably, the correction of the three-dimensional road information in S4 includes:

[0022] ZeroMQ is used to receive solution information, and a sliding window settlement of the factor graph is performed based on the visual observation factor, the inertial navigation pre-integration factor, and the color recognition road factor, and the posterior covariance information is calculated;

[0023] According to the standard road factor, the road standard value is updated, the updated road center coordinates are judged, and the standard road factor is integrated within the valid value using the sliding window solution result as a priori; the standard road factor is no longer directly integrated.

[0024] Preferably, the supplementation of the three-dimensional road database in S4 includes: extracting road center information based on standard road information, judging the extracted information by the travel direction and lateral direction benchmarks, correcting the posture of the matched three-dimensional road data after determining the road center information, and supplementing the road center coordinate positioning information to the three-dimensional road data for output.

[0025] Therefore, the present invention adopts the above-mentioned three-dimensional road information collection system with multi-sensor information fusion, which has the following beneficial effects:

[0026] (1) Divide road information into central coordinate positioning information and relative road information to achieve three-dimensional road modeling;

[0027] (2) Using color sensors to accurately locate road information relative to road boundaries and traffic signs as landmarks;

[0028] (3) Laser point cloud and visual features are integrated to extract accurate road information, and three-dimensional road data collection is achieved by combining central coordinate positioning information;

[0029] (4) The complementary advantages of multiple sensors enable high-precision measurement and collection of three-dimensional road information.

[0030] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 Provide a block diagram of the system;

[0032] Figure 2 This is a flowchart for multi-sensor three-dimensional road information data processing;

[0033] Figure 3 This is a three-dimensional road information processing flowchart. DETAILED DESCRIPTION

[0034] Example

[0035] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention.

[0036] The present invention provides a three-dimensional road information collection system based on multi-sensor information fusion. Figure 1, including RTK, camera, color sensor, lidar, IMU measurement unit, industrial computer, control panel and lithium battery pack. RTK, camera, color sensor, lidar, IMU inertial measurement unit, control panel and lithium battery pack are all connected to the industrial computer, and RTK is connected to the GNSS antenna.

[0037] Reference Figure 2 , implementation methods include:

[0038] S1. Use ZeroMQ to receive INS inertial navigation information, camera-based visual detection of landmarks and road information, color sensor-based landmarks and road edge information, and GNSS-based RTK high-precision position information. This data is then filtered. The filtered information is verified and invalid information is removed. The camera-based visual data and color sensor data are then correlated to obtain the direction of travel and lateral reference.

[0039] The effective value judgment of the travel direction and lateral direction reference is performed through RTK high-precision position information, which can eliminate the cumulative error of INS but does not meet the requirements of cyclic detection.

[0040] Based on the IMU measurement unit, the INS data queue in the INS inertial navigation data is collected, and the INS data queue is differentially interpolated and aligned to the integration interval. INS pre-integration is performed to obtain the visual data queue to correct the relative position change between epochs. The random sampling consistency method is used to calculate the basic matrix and homography matrix, and the external points in the visual observation are eliminated. The geometric transformation analysis is then performed to obtain landmark extraction and road information analysis, namely the three-dimensional road center coordinate positioning information.

[0041] S2. Receive road information collected by LiDAR and refine the road point cloud. LiDAR planes and edge points are extracted and parameterized. Line-to-line distances and surface-to-surface distances are calculated to match line-to-surface features. Points with low and high curvature in the original point cloud represent planes and lines in the data environment, respectively.

[0042] S3. Integrate camera visual detection information with laser point cloud, improve spatial positioning through point cloud, provide rich interpretation of point cloud with vision, obtain relative road information of 3D roads and traffic signs, and build a 3D road database.

[0043] S4. Compare the standard road information obtained by using GNSS RTK high-precision position information with the three-dimensional road database. If there is a mismatch, correct the three-dimensional road information. If there is a match, supplement the three-dimensional road database based on the center coordinate positioning information. Figure 3 shown.

[0044] Correction of 3D road information includes:

[0045] ZeroMQ is used to receive solution information. A factor graph is constructed based on the visual observation factor, inertial navigation pre-integration factor, and color recognition road factor. A sliding window is set to solve the parameters to be estimated and their posterior covariance information is calculated.

[0046] According to the standard road factor, the road standard value is updated, and the updated road center coordinates are judged. The standard road factor is integrated within the valid value using the sliding window solution result as a priori; the standard road factor is no longer directly integrated.

[0047] Sliding window: Marginalize and remove the earliest observation value in the window to obtain the settlement result of the next frame.

[0048] Supplementing the three-dimensional road database includes: extracting road center information based on standard road information, judging the extracted information through the travel direction and lateral direction benchmarks, correcting the posture of the matched three-dimensional road data after determining the road center information, and supplementing the road center coordinate positioning information to the three-dimensional road data for output.

[0049] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the same.

Claims

1. A three-dimensional road information collection system with multi-sensor information fusion, characterized by: Including RTK, camera, color sensor, lidar, IMU measurement unit, industrial computer, control panel and lithium battery pack. RTK, camera, color sensor, lidar, IMU inertial measurement unit, control panel and lithium battery pack are all connected to the industrial computer, and RTK is connected to the GNSS antenna.

2. The three-dimensional road information collection system of multi-sensor information fusion according to claim 1 is characterized in that: Implementation methods include: S1, receiving INS inertial navigation information, camera visual detection data, color sensor acquisition data and GNSS-based RTK high-precision position information, and processing the received data to obtain three-dimensional road center coordinate positioning information; S2, receiving the road information collected by the laser radar and refining the road information point cloud; S3, integrating camera visual detection data with laser point cloud to build a three-dimensional road database; S4. Compare the standard road information obtained from the RTK high-precision position information with the three-dimensional road database. If there is a mismatch, correct the three-dimensional road information. If there is a match, supplement the three-dimensional road database based on the center coordinate positioning information.

3. A three-dimensional road information collection system with multi-sensor information fusion according to claim 2, characterized in that: The processing of received data in S1 includes: S11, screening the received data; S12, pre-processing the filtered data, associating the camera vision data and the color sensor data to obtain the travel direction and the lateral direction reference; S13, using RTK high-precision position information to determine the effective value of the travel direction and lateral direction reference, which can eliminate the cumulative error of INS but does not meet the cycle detection requirement; S14. Based on the IMU measurement unit, the INS data queue in the INS inertial navigation information is collected, and the INS data queue is differentially interpolated and aligned to the integration interval. INS pre-integration is performed to obtain the visual data queue corrected for the relative position change between epochs. The random sampling consistency method is used to calculate the basic matrix and the homography matrix, and the external points in the visual observation are eliminated. The geometric transformation analysis is performed to obtain the landmark extraction and road information analysis.

4. The three-dimensional road information collection system of multi-sensor information fusion according to claim 3 is characterized by: Point cloud refinement in S2 includes extracting lidar planes and edge points, parameterizing them, and calculating line-surface features for matching different line-line distances and surface-surface distances.

5. The three-dimensional road information collection system of multi-sensor information fusion according to claim 4 is characterized in that: The correction of the three-dimensional road information in S4 includes: ZeroMQ is used to receive solution information, and factor graph sliding window settlement is performed based on visual observation factors, inertial navigation pre-integration factors, and color recognition road factors, and the posterior covariance information is calculated; According to the standard road factor, the road standard value is updated, the updated road center coordinates are judged, and the standard road factor is integrated with the sliding window settlement result as a priori within the valid value; the standard road factor is no longer directly integrated.

6. The three-dimensional road information collection system of multi-sensor information fusion according to claim 4 is characterized in that: The supplementation of the three-dimensional road database in S4 includes: extracting road center information based on standard road information, judging the extracted information by the travel direction and lateral direction benchmarks, correcting the posture of the matched three-dimensional road data after determining the road center information, and supplementing the road center coordinate positioning information to the three-dimensional road data for output.

Citation Information

Cited By

  • Chip combiner, method and device based on multi-sensor fusion, controller and medium

    CN119300533A

  • Die Bonding Machine, Method, Device, Controller and Medium Based on Multi-Sensor Fusion

    CN119300533B