A road surface singular feature localization system and method based on multi-source input fusion
By combining multiple information sources and employing a weighted function for feature matching and fusion, the system and method for locating singular road features solve the problem of inaccurate localization of singular road features in existing technologies and achieves high-precision road detection.
Patent Information
- Application Number
- CN202210881979.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-07-26
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2042-07-26
AI Technical Summary
Existing automated road surface detection technologies have significant limitations under complex driving conditions. Single-sensor or multi-sensor fusion methods struggle to accurately identify and locate unusual road surface features, especially in harsh environments where errors are substantial.
A road singular feature localization system and method based on multi-source input fusion is proposed. By acquiring vehicle position information, pose information, vibration information, speed information, road point cloud data and image information, and combining non-contact and response-based road singular feature recognition algorithms, feature matching and fusion are performed using weighted functions to improve localization accuracy.
It enables precise detection and location of unusual road surface features, reduces positioning errors, and improves detection accuracy.
Smart Images

Figure CN115273006B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of road surface positioning technology, and in particular to a road surface singular feature positioning system and method based on multi-source input fusion. Background Technology
[0002] By the end of 2020, China's total highway network mileage exceeded 5,010,000 km, ranking first in the world. With the gradual improvement of the highway network, the continuous increase in mileage, and the constant upgrading of technology, highway maintenance tasks have become increasingly important, and efficient and accurate pavement quality detection technology has become a crucial industry requirement. Existing automated pavement detection technologies are mainly divided into two categories: one is non-contact pavement anomaly detection and localization based on sensing sensors combined with algorithms, primarily using cameras, LiDAR, etc., to directly identify pavement anomalies remotely and non-contactly (e.g., CN106529593B, CN113674355A); the other is indirect identification based on vehicle response parameters collected by vibration acceleration sensors (e.g., CN107167580B). Single-sensor automated pavement detection under complex driving conditions has significant limitations, resulting in large errors in the localization of pavement anomalies. While identification methods based on vehicle response parameters can objectively reflect the pavement quality traversed by vehicle wheels based on vibration, they struggle to accurately identify and locate pavement anomalies. Existing multi-sensor fusion road surface recognition methods mostly employ joint identification using sensing sensors (radar and camera) (Patent 202110762668.X). However, in harsh environments, the vehicle's roll angle constantly changes, leading to significant errors in the localization of road surface anomalies. Patent (202111220525.2) uses joint identification with sensing and vibration sensors to model road surface anomalies, but it does not address the accurate localization of these anomalies. Therefore, there is an urgent need in this field for a method or system capable of accurately detecting and localizing road surface anomalies. Summary of the Invention
[0003] To address the aforementioned problems in the existing technology, this invention provides a road surface singular feature localization system and method based on multi-source input fusion.
[0004] To achieve the above objectives, the present invention provides the following solution:
[0005] A road surface singular feature localization system based on multi-source input fusion includes:
[0006] The information acquisition module is used to acquire vehicle information, including: vehicle location information, vehicle pose information, vehicle tire position information, vehicle vibration information, vehicle speed information, and road surface point cloud data and road surface image information during vehicle operation.
[0007] A road surface anomaly feature recognition module, connected to the information acquisition module, is used to extract features based on the vehicle information to obtain first road surface anomaly feature location information and second road surface anomaly feature location information.
[0008] The road surface singular feature fusion module is connected to the road surface singular feature recognition module. It is used to determine whether the first road surface singular feature and the second road surface singular feature are the same road surface singular feature location information by using a matching algorithm, to obtain matching information, and to generate road surface singular feature positioning information based on the matching information.
[0009] Preferably, it further includes:
[0010] An error calculation module, connected to the road surface singular feature recognition module, is used to obtain the longitudinal measurement noise variance and radial measurement noise variance of the first road surface singular feature location information, as well as the radial positioning variance of the second road surface singular feature.
[0011] The weighting function generation module, connected to the error calculation module and the road surface singular feature fusion module, is used to construct a weighting function with the longitudinal measurement noise variance, the radial measurement noise variance and the radial positioning variance as inputs;
[0012] When the first road surface singular feature and the second road surface singular feature are the same road surface singular feature, the road surface singular feature fusion module calls the weighting function to perform weighted fusion of the location information of the first road surface singular feature and the location information of the second road surface singular feature, and outputs fused feature information; the fused feature information is the road surface singular feature positioning information.
[0013] Preferably, the information acquisition module includes:
[0014] The positioning system information acquisition submodule is used to acquire the vehicle's location information and the vehicle's pose information;
[0015] The vehicle-mounted sensor information acquisition submodule is used to acquire the vibration information and speed information of the vehicle.
[0016] The perception sensor data acquisition submodule is used to acquire road surface point cloud data and road surface image information during vehicle operation.
[0017] Preferably, the road surface anomaly recognition module includes:
[0018] A non-contact road surface singular feature fusion and recognition submodule is connected to the perception sensor data acquisition submodule and the positioning system information acquisition submodule, and is used to obtain the first road surface singular feature location information based on the vehicle's location information, the vehicle's pose information, the road surface point cloud data and the road surface image information;
[0019] The response-based road surface singular feature recognition submodule is connected to the vehicle-mounted sensor information acquisition submodule and is used to obtain the second road surface singular feature location information based on the vehicle's vibration information and speed information using the response-based road surface singular feature recognition algorithm.
[0020] Preferably, the non-contact road surface anomaly feature fusion and recognition submodule includes:
[0021] The identification and positioning unit is connected to the perception sensor data acquisition submodule and the positioning system information acquisition submodule, and is used to identify road surface singular features based on the vehicle's location information, the vehicle's pose information, the road surface point cloud data, and the road surface image information; the road surface singular features include: lidar road surface singular features and camera road surface singular features;
[0022] A fusion positioning unit, connected to the identification positioning unit, is used to output non-contact road surface singularity feature location information based on the identified road surface singularity features; the non-contact road surface singularity feature location information is the first road surface singularity feature location information.
[0023] Preferably, the positioning system information acquisition submodule includes:
[0024] The GPS positioning unit is used to obtain the location information of the vehicle;
[0025] The dead reckoning unit is used to obtain the vehicle's pose information.
[0026] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:
[0027] The road singular feature localization system based on multi-source input fusion provided by this invention obtains first road singular feature location information and second road singular feature location information based on vehicle information including vehicle position information, vehicle pose information, vehicle tire position information, vehicle vibration information, vehicle speed information, and road point cloud data and road image information during vehicle travel. Then, road singular feature localization information is generated according to the matching information between the first road singular feature location information and the second road singular feature location information, thereby realizing accurate detection and localization of road singular features.
[0028] Corresponding to the road surface singular feature localization system based on multi-source input fusion provided above, this invention provides a road surface singular feature localization method based on multi-source input fusion, the method comprising:
[0029] Acquire vehicle information; the vehicle information includes: vehicle location information, vehicle pose information, vehicle tire position information, vehicle vibration information, vehicle speed information, and road surface point cloud data and road surface image information during vehicle operation.
[0030] Based on the vehicle information, feature extraction is performed to obtain the location information of the first and second road surface singular features.
[0031] A matching algorithm is used to determine whether the first road surface singular feature and the second road surface singular feature are the same road surface singular feature location information, and the matching information is obtained;
[0032] Based on the matching information, road surface anomaly feature location information is generated.
[0033] Preferably, generating road surface singular feature localization information based on the matching information specifically includes:
[0034] The longitudinal measurement noise variance and radial measurement noise variance of the first road surface singular feature location information, and the radial positioning variance of the second road surface singular feature are obtained.
[0035] A weighting function is constructed using the longitudinal measurement noise variance, the radial measurement noise variance, and the radial positioning variance as inputs;
[0036] When the matching information is that the first road surface singular feature and the second road surface singular feature are the same road surface singular feature, the weighting function is used to perform weighted fusion of the location information of the first road surface singular feature and the location information of the second road surface singular feature, and then the fused feature information is output; the fused feature information is the road surface singular feature location information;
[0037] When the matching information indicates that the first road surface singular feature and the second road surface singular feature are not the same road surface singular feature, the location information of the first road surface singular feature and the location information of the second road surface singular feature are output respectively as road surface singular feature positioning information.
[0038] Preferably, the step of extracting features based on the vehicle information to obtain the first road surface singular feature location information and the second road surface singular feature location information specifically includes:
[0039] The location information of the first road surface singular feature is obtained based on the vehicle's location information, the vehicle's pose information, the road surface point cloud data, and the road surface image information.
[0040] The location information of the second road surface singular feature is obtained by using a response-based road surface singular feature recognition algorithm based on the vibration information and speed information of the vehicle.
[0041] Preferably, obtaining the first road surface singular feature location information based on the vehicle's location information, the vehicle's pose information, the road surface point cloud data, and the road surface image information specifically includes:
[0042] The road surface singular features are identified based on the vehicle's location information, the vehicle's pose information, the road surface point cloud data, and the road surface image information; the road surface singular features include: lidar road surface singular features and camera road surface singular features;
[0043] The location information of non-contact road surface singular features is output based on the identified road surface singular features; the location information of non-contact road surface singular features is the location information of the first road surface singular features.
[0044] Since the technical effect achieved by the road surface singular feature localization method based on multi-source input fusion provided by this invention is the same as that achieved by the road surface singular feature localization system based on multi-source input fusion provided above, it will not be described again here. Attached Figure Description
[0045] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0046] Figure 1 A structural diagram of the road surface singular feature localization system based on multi-source input fusion provided by the present invention;
[0047] Figure 2 The data processing flowchart of the non-contact road surface singular feature fusion and recognition submodule provided by the present invention;
[0048] Figure 3 The optimal matching result diagram provided by the embodiments of the present invention;
[0049] Figure 4 This is a schematic diagram of tire position provided for an embodiment of the present invention;
[0050] Figure 5 The flowchart shows the road surface singular feature localization method based on multi-source input fusion provided by the present invention.
[0051] Symbol explanation:
[0052] 1. Information Acquisition Module, 1-1 Positioning System Information Acquisition Submodule, 1-2 Vehicle Sensor Information Acquisition Submodule, 1-3 Perception Sensor Data Acquisition Submodule, 2. Road Singular Feature Recognition Module, 2-1 Non-Contact Road Singular Feature Fusion Recognition Submodule, 2-2 Response-Based Road Singular Feature Recognition Submodule, 3. Road Singular Feature Fusion Module, 4. Error Calculation Module, 5. Weighted Function Generation Module. Detailed Implementation
[0053] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0054] The purpose of this invention is to provide a road surface singular feature localization system and method based on multi-source input fusion, which can improve the accuracy of road surface localization.
[0055] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0056] like Figure 1 As shown, the road surface singular feature localization system based on multi-source input fusion provided by the present invention includes: an information acquisition module 1, a road surface singular feature recognition module 2, and a road surface singular feature fusion module 3.
[0057] Information acquisition module 1 is used to acquire vehicle information. Vehicle information includes: vehicle location information, vehicle pose information, vehicle tire position information, vehicle vibration information, vehicle speed information, and road surface point cloud data and road surface image information during vehicle operation. For example, information acquisition module 1 may include: positioning system information acquisition submodule 1-1, vehicle-mounted sensor information acquisition submodule 1-2, and perception sensor data acquisition submodule 1-3.
[0058] The positioning system information acquisition submodule 1-1 is used to acquire the vehicle's position and pose information. Specifically, the positioning system information acquisition submodule 1-1 can be equipped with a GPS (or BeiDou) positioning unit and a dead reckoning unit. The GPS positioning unit consists of a GPS base station and a GPS signal receiver. The dead reckoning (DR) unit consists of an electronic compass and an inertial measurement unit (IMU). The vehicle's center of mass position information is acquired in real time via GPS, and the vehicle's pose information is acquired in real time via the dead reckoning unit. Through the fusion calculation of the two positioning information (for example, using the "Vehicle Heading Angle Measurement Technology Based on RTK-GNSS and MEMS Gyroscope"), the vehicle's position information in the world coordinate system and its own pose information are acquired in real time. Simultaneously, combined with the vehicle's own parameters, the vehicle's tire position information is acquired in real time. The results obtained from both are then sent to the road surface singularity feature recognition module.
[0059] The vehicle-mounted sensor information acquisition submodule 1-2 is used to acquire vehicle vibration and speed information. Specifically, the vehicle-mounted sensor information acquisition submodule 1-2 may include an accelerometer and a wheel speed meter (or any vehicle speed measuring hardware). The accelerometer acquires vehicle vibration signals, and the wheel speed meter acquires vehicle speed information, enabling real-time detection of changes in the vehicle's vertical vibration characteristics and speed.
[0060] The perception sensor data acquisition submodule 1-3 is used to acquire road surface point cloud data and road surface image information during vehicle operation. Specifically, the perception sensor data acquisition submodule 1-3 may include a lidar and a road surface image data acquisition unit. The lidar acquires raw road surface point cloud data during vehicle operation, and the road surface image data acquisition unit (camera) acquires road surface image information during vehicle operation. The results obtained from the above two methods are then sent to the road surface singular feature recognition module 2.
[0061] The road surface anomaly recognition module 2 is connected to the information acquisition module 1. The road surface anomaly recognition module 2 is used to extract features based on vehicle information to obtain the location information of the first road surface anomaly feature and the location information of the second road surface anomaly feature.
[0062] To improve the accuracy of road surface singularity feature recognition, this invention incorporates a non-contact road surface singularity feature recognition and localization algorithm and a response-based road surface singularity feature recognition and localization algorithm into the road surface singularity feature recognition module 2. Specifically, the non-contact road surface singularity feature recognition and localization algorithm corresponds to the non-contact road surface singularity feature fusion recognition submodule 2-1, and the response-based road surface singularity feature recognition and localization algorithm corresponds to the response-based road surface singularity feature recognition submodule 2-2.
[0063] Specifically, the non-contact road surface singular feature fusion and recognition submodule 2-1 is connected to the perception sensor data acquisition submodule 1-3 and the positioning system information acquisition submodule 1-1. The non-contact road surface singular feature fusion and recognition submodule 2-1 is used to obtain the first road surface singular feature location information based on the vehicle's location information, vehicle's pose information, road surface point cloud data, and road surface image information.
[0064] like Figure 2 As shown, the non-contact road surface singular feature fusion recognition submodule 2-1 includes a recognition and positioning unit and a fusion and positioning unit.
[0065] The identification and positioning unit is connected to the perception sensor data acquisition submodule 1-3 and the positioning system information acquisition submodule 1-1. The identification and positioning unit is used to identify road surface singularities based on vehicle location information, vehicle pose information, road surface point cloud data, and road surface image information. Road surface singularities include: lidar road surface singularities and camera road surface singularities. Specifically, the identification and positioning unit includes a lidar-based road surface singularity feature identification and positioning algorithm (referred to as the lidar identification and positioning algorithm) and a camera-based road surface singularity feature identification and positioning algorithm (referred to as the camera identification and positioning algorithm). For example, the camera identification and positioning algorithm detects road surface singularities using a deep learning algorithm (such as YOLOv5), extracts left and right eye features using the ORB feature point extraction method, completes feature point matching, and then calculates the three-dimensional coordinates of the matching points based on stereo vision principles, performs road surface singularity feature clustering, and obtains the road surface singularities, but is not limited to this. The lidar identification and positioning algorithm identifies road surface singularities using a deep learning algorithm (such as Point-GNN), but is not limited to this.
[0066] The fusion positioning unit is connected to the identification positioning unit. The fusion positioning unit is used to output non-contact road surface singularity feature location information based on the identified road surface singularity features. The non-contact road surface singularity feature location information is the first road surface singularity feature location information. For example, the fusion positioning unit determines whether both the lidar and the camera output road surface singularity features, and designs an algorithm to fuse the identified road surface singularity features from different inputs to obtain fused non-contact road surface singularity features (hereinafter referred to as non-contact features), but is not limited to this.
[0067] Furthermore, the fusion positioning unit outputs non-contact location information of road surface singularities based on the identified road surface singularities, categorized into three cases:
[0068] 1. Only when the lidar identifies unusual feature information, the unusual feature of the road surface is transformed from the lidar coordinate system to the world coordinate system through coordinate transformation, and output as a non-contact identification of unusual road surface features.
[0069] 2. Only when the camera recognizes the singular feature information, the singular feature of the road surface is transformed from the image coordinate system to the world coordinate system through coordinate transformation, and output as a non-contact recognition of the singular feature of the road surface.
[0070] 3. When both the lidar and the camera output obstacle information, the data are first synchronized in time to obtain lidar point cloud data and camera data after feature extraction at the same time. Then, the lidar and camera are jointly calibrated to obtain the rotation and translation matrix between the lidar and the camera. The image target and the point cloud target are converted to the same coordinate system. Using the DS (Dempster-Shafer) evidence theory, sensor data fusion is performed to obtain the coordinates of the non-contact road surface singular feature center, which is mapped to the xoy plane and used as the input of the road surface singular feature fusion module 3.
[0071] The response-based road surface singularity feature recognition submodule 2-2 is connected to the vehicle-mounted sensor information acquisition submodule 1-2. The response-based road surface singularity feature recognition submodule 2-2 is used to obtain the location information of the second road surface singularity feature based on the vehicle's vibration and speed information using a response-based road surface singularity feature recognition algorithm. For example, the response-based road surface singularity feature recognition submodule 2-2 extracts features from the vehicle's vertical vibration system response and identifies road surface singularities using a disturbance observer or a threshold-based recognition method; the selected method for identifying road surface singularities is not limited to these.
[0072] The road surface singular feature fusion module 3 is connected to the road surface singular feature recognition module 2. The road surface singular feature fusion module 3 uses a matching algorithm to determine whether the first road surface singular feature and the second road surface singular feature are the same road surface singular feature location information, obtaining matching information, and uses this matching information to generate road surface singular feature location information. For example, the singular feature matching algorithm is:
[0073] Using a threshold value as the matching threshold, distance matching is performed on the response feature and non-contact features. When a response feature and multiple non-contact features satisfy a matching relationship, the matching result with the smallest feature distance is the optimal matching result. For example... Figure 3 As shown, represents the identified response features, represents the identified non-contact features, R is the matching threshold, and within the threshold, and satisfy the matching relationship. By calculation, and are less than and , and and are selected as the same road surface singular features.
[0074] To further improve the accuracy of road surface singular feature localization, the road surface singular feature localization system provided above also includes: an error calculation module 4 and a weighted function generation module 5.
[0075] Error calculation module 4 is connected to road surface singularity feature recognition module 2. Error calculation module 4 is used to obtain the longitudinal measurement noise variance δ of the first road surface singularity feature location information.x and radial measurement noise variance δ y And the radial positioning variance δ of the second road surface singularity features. x' .
[0076] The weighting function generation module 5 is connected to the error calculation module 4 and the road surface singular feature fusion module 3. The weighting function generation module 5 is used to construct a weighting function using the longitudinal measurement noise variance, radial measurement noise variance, and radial positioning variance as inputs. For example, the constructed weighting function is:
[0077]
[0078] Where (x',y') represents the location of the singular feature of the road surface after final fusion, (x X ,y X (x) represents the location of the response feature. F ,y F () represents the non-contact feature location. The weighting parameters w1 and w2 are:
[0079] When the first and second road surface singular features are the same singular feature, the road surface singular feature fusion module 3 invokes a weighting function to weight and fuse the location information of the first and second road surface singular features, and outputs the fused feature information. The fused feature information is the road surface singular feature location information. Specifically, the road surface singular feature fusion module 3 receives the location information of road surface singular features identified from multiple sources, and uses a matching algorithm to match non-contact identified road surface singular features (i.e., the first road surface singular feature) and response-based identified road surface singular features (i.e., the second road surface singular feature). Matched singular features are considered the same singular feature, and are weighted and fused using a weighting function to obtain the fused road surface singular feature with location information. Unmatched features output their respective location information.
[0080] The example provided describes a specific implementation process for data collection in an information collection module, but in practical applications, it is not limited to this.
[0081] Step 1: During normal vehicle operation, the dead reckoning unit and GPS positioning unit perform vehicle positioning and pose estimation. Onboard sensors begin collecting vehicle information, including but not limited to vehicle speed and vertical vibration information. Perception sensors collect road surface information; specifically, the lidar collects raw point cloud data of the road surface during vehicle movement, and the camera collects road surface image information during vehicle movement. All of this information serves as input to the road surface singularity feature recognition module 2.
[0082] Step 2: After obtaining the information input by the information acquisition module 1, the road surface singular feature identification module 2 can detect singular features by selecting a threshold based on the response road surface singular feature identification algorithm. By setting the singular feature threshold, whether it is a singular feature is determined by whether the vehicle z-axis acceleration input by the vehicle sensor is greater than the threshold. At the same time, the vehicle centroid position information collected in real time by the GPS positioning system and the vehicle heading angle θ and centroid sideslip angle β collected in real time by the dead reckoning system are used to calculate the vehicle yaw angle ψ as shown in formula (2), and the tire position, i.e., the response feature position information, is obtained in real time as shown in formula (3). Since it is impossible to determine which tire position passes through the singular feature center, the radial positioning variance of the response feature is half the tire width B / 2, as shown in formula (4). Taking the left front wheel as an example, the tire position is shown to be calculated.
[0083] ψ=θ-β (2)
[0084]
[0085]
[0086] Its schematic diagram is as follows Figure 4 As shown, where a is the distance from the vehicle's center of gravity to the front axle, b is the distance from the vehicle's center of gravity to the rear axle, and ψ is the vehicle's yaw angle. (x 车 ,y 车 ) is the location of the vehicle's center of gravity.
[0087] Corresponding to the road surface singular feature localization system based on multi-source input fusion provided above, this invention provides a road surface singular feature localization method based on multi-source input fusion, such as... Figure 5 As shown, the method includes:
[0088] Step 100: Obtain vehicle information. Vehicle information includes: vehicle location information, vehicle pose information, vehicle tire position information, vehicle vibration information, vehicle speed information, and road surface point cloud data and road surface image information during vehicle operation.
[0089] Step 101: Extract features based on vehicle information to obtain the location information of the first and second road surface singular features. This step specifically includes:
[0090] Step 1010: Obtain the first road surface singularity feature location information based on the vehicle's location information, vehicle's pose information, road surface point cloud data, and road surface image information. Specifically, identify road surface singularities based on the vehicle's location information, vehicle's pose information, road surface point cloud data, and road surface image information. Road surface singularities include: LiDAR road surface singularities and camera road surface singularities. Output non-contact road surface singularity feature location information based on the identified road surface singularities. The non-contact road surface singularity feature location information is the first road surface singularity feature location information.
[0091] Step 1011: The response-based road surface singular feature recognition algorithm is used to obtain the location information of the second road surface singular feature based on the vehicle's vibration information and speed information.
[0092] Step 102: Use a matching algorithm to determine whether the first road surface singular feature and the second road surface singular feature are the same road surface singular feature location information, and obtain matching information.
[0093] Step 103: Generate road surface singular feature localization information based on the matching information. This step specifically includes:
[0094] Step 1030: Obtain the longitudinal measurement noise variance and radial measurement noise variance of the location information of the first road surface singular feature, and the radial positioning variance of the second road surface singular feature.
[0095] Step 1031: Construct a weighting function using longitudinal measurement noise variance, radial measurement noise variance, and radial positioning variance as inputs.
[0096] Step 1032: When the matching information is that the first road surface singular feature and the second road surface singular feature are the same road surface singular feature, a weighted function is used to weight and fuse the location information of the first road surface singular feature and the location information of the second road surface singular feature, and the fused feature information is output. The fused feature information is the road surface singular feature localization information.
[0097] Step 1034: When the matching information is that the first road surface singular feature and the second road surface singular feature are not the same road surface singular feature, output the location information of the first road surface singular feature and the location information of the second road surface singular feature respectively, as the location information of the road surface singular feature.
[0098] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.
[0099] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.
Claims
1. A road surface singular feature localization system based on multi-source input fusion, characterized in that, include: The information collection module is used to acquire vehicle information; The vehicle information includes: vehicle location information, vehicle pose information, vehicle tire position information, vehicle vibration information, vehicle speed information, and road surface point cloud data and road surface image information during vehicle operation. A road surface anomaly feature recognition module, connected to the information acquisition module, is used to extract features based on the vehicle information to obtain first road surface anomaly feature location information and second road surface anomaly feature location information. A road surface singular feature fusion module, connected to the road surface singular feature recognition module, is used to determine whether a first road surface singular feature and a second road surface singular feature are the same road surface singular feature location information using a matching algorithm, obtaining matching information, and generating road surface singular feature positioning information based on the matching information. The road surface singular feature fusion module receives road surface singular feature location information recognized from multiple sources, matches the first road surface singular feature and the second road surface singular feature using a matching algorithm, and if the singular features that match are the same singular feature, they are weighted and fused using a weighting function to obtain the fused road surface singular feature with location information; if they do not match, their respective road surface singular feature location information is output. An error calculation module, connected to the road surface singular feature recognition module, is used to obtain the longitudinal measurement noise variance and radial measurement noise variance of the first road surface singular feature location information, as well as the radial positioning variance of the second road surface singular feature. The weighting function generation module is connected to the error calculation module and the road surface singular feature fusion module, and is used to construct the weighting function with the longitudinal measurement noise variance, the radial measurement noise variance and the radial positioning variance as inputs.
2. The road surface singular feature localization system based on multi-source input fusion according to claim 1, characterized in that, The information collection module includes: The positioning system information acquisition submodule is used to acquire the vehicle's location information and the vehicle's pose information; The vehicle-mounted sensor information acquisition submodule is used to acquire the vibration information and speed information of the vehicle. The perception sensor data acquisition submodule is used to acquire road surface point cloud data and road surface image information during vehicle operation.
3. The road surface singular feature localization system based on multi-source input fusion according to claim 2, characterized in that, The road surface anomaly feature recognition module includes: A non-contact road surface singular feature fusion and recognition submodule is connected to the perception sensor data acquisition submodule and the positioning system information acquisition submodule, and is used to obtain the first road surface singular feature location information based on the vehicle's location information, the vehicle's pose information, the road surface point cloud data and the road surface image information; The response-based road surface singular feature recognition submodule is connected to the vehicle-mounted sensor information acquisition submodule and is used to obtain the second road surface singular feature location information based on the vehicle's vibration information and speed information using the response-based road surface singular feature recognition algorithm.
4. The road surface singular feature localization system based on multi-source input fusion according to claim 3, characterized in that, The non-contact road surface singular feature fusion and recognition submodule includes: The identification and positioning unit is connected to the perception sensor data acquisition submodule and the positioning system information acquisition submodule, and is used to identify road surface singular features based on the vehicle's location information, the vehicle's pose information, the road surface point cloud data, and the road surface image information; the road surface singular features include: lidar road surface singular features and camera road surface singular features; A fusion positioning unit, connected to the identification positioning unit, is used to output non-contact road surface singularity feature location information based on the identified road surface singularity features; the non-contact road surface singularity feature location information is the first road surface singularity feature location information.
5. The road surface singular feature localization system based on multi-source input fusion according to claim 2, characterized in that, The positioning system information acquisition submodule includes: The GPS positioning unit is used to obtain the location information of the vehicle; The dead reckoning unit is used to obtain the vehicle's pose information.
6. A method for locating singular road features based on multi-source input fusion, characterized in that, include: Obtain vehicle information; The vehicle information includes: vehicle location information, vehicle pose information, vehicle tire position information, vehicle vibration information, vehicle speed information, and road surface point cloud data and road surface image information during vehicle operation. Based on the vehicle information, feature extraction is performed to obtain the location information of the first and second road surface singular features. A matching algorithm is used to determine whether the first road surface singular feature and the second road surface singular feature are the same road surface singular feature location information, and the matching information is obtained; Generating road surface singular feature location information based on the matching information includes: obtaining the longitudinal measurement noise variance and radial measurement noise variance of the first road surface singular feature location information, and the radial location variance of the second road surface singular feature; constructing a weighting function using the longitudinal measurement noise variance, the radial measurement noise variance, and the radial location variance as inputs; wherein, the first road surface singular feature and the second road surface singular feature are matched using a matching algorithm, and the matched singular features are the same singular feature, which is then weighted and fused using the weighting function to obtain the fused road surface singular feature with location information; the location information of the unmatched road surface singular features is output separately.
7. The method for locating singular road features based on multi-source input fusion according to claim 6, characterized in that, The step of extracting features based on the vehicle information to obtain the first road surface singular feature location information and the second road surface singular feature location information specifically includes: The location information of the first road surface singular feature is obtained based on the vehicle's location information, the vehicle's pose information, the road surface point cloud data, and the road surface image information. The location information of the second road surface singular feature is obtained by using a response-based road surface singular feature recognition algorithm based on the vibration information and speed information of the vehicle.
8. The method for locating singular road features based on multi-source input fusion according to claim 7, characterized in that, The step of obtaining the first road surface singular feature location information based on the vehicle's location information, the vehicle's pose information, the road surface point cloud data, and the road surface image information specifically includes: The road surface singular features are identified based on the vehicle's location information, the vehicle's pose information, the road surface point cloud data, and the road surface image information; the road surface singular features include: lidar road surface singular features and camera road surface singular features; The non-contact road surface singularity feature location information is output based on the identified road surface singularity feature; the non-contact road surface singularity feature location information is the first road surface singularity feature location information.
Citation Information
Patent Citations
Road surface distress detection methods and systems
CN106529593B
Road pothole detection method based on acceleration sensor and machine learning
CN107167580B
Target identification and positioning method based on camera and laser radar
CN113674355A
A method and system for road quality detection by fusing multi-source input signals
CN113962301B
Road surface quality detection method and system based on multi-source input signal fusion
CN113962301A
