A park lane line perception method and system based on multi-source data fusion and prior map constraint

Abstract

The application provides a kind of park lane line perception method and system based on multi-source data fusion and prior map constraint, it is related to map and positioning navigation technical field, this method includes: collecting multi-source data and pre-processing image;Extract color, edge and semantic feature for weighted fusion, generate candidate lane line heat map;Obtain high-precision map prior information and project to image coordinate system, spatial screening is carried out to candidate point;Adaptive model fitting is carried out to effective point set, and lane line observation value is obtained;Kalman filtering tracking and prediction are carried out in combination with vehicle kinematic model;Comprehensive multi-dimensional index calculates confidence and carries out hierarchical processing.The application improves the detection robustness in complex environment by multi-feature fusion, uses map prior to carry out flexible constraint to eliminate noise, introduces motion tracking to ensure continuity when shielding, and realizes abnormal response through confidence evaluation, effectively solves the perception problem caused by unclear park lane line feature and variable environment.

Description

Technical Field

[0001] This invention relates to the field of maps and positioning navigation, and in particular to a method and system for perceiving lane lines in a park based on multi-source data fusion and prior map constraints. Background Technology

[0002] As a crucial application scenario for intelligent connected vehicles, autonomous driving in industrial parks relies heavily on accurate lane line perception as a key prerequisite for achieving high-precision positioning and stable control. Lane line information provides lateral constraints and path guidance for vehicles, forming the foundation for higher-level functions such as path planning and decision-making control. However, unlike structured highways or urban roads, the closed or semi-closed environment of industrial parks presents unique complexities, posing significant challenges to lane line perception. Due to the poor standardization of road markings in industrial parks, long-term exposure to heavy vehicles and environmental erosion has resulted in widespread wear, fading, and blurring of lane lines, making it difficult for traditional visual detection algorithms to extract complete and clear line features. Furthermore, the drastic changes in lighting conditions within industrial parks, including shadows cast by buildings and trees, and uneven nighttime illumination, easily lead to over-segmentation or missed detection of lane lines. Frequent dynamic interference from pedestrians, low-speed logistics vehicles, and temporary obstacles within industrial parks severely obstructs lane lines, resulting in intermittent and unstable perception results.

[0003] To address the aforementioned issues, existing technologies primarily offer two types of solutions. One type is the pure visual perception solution, which heavily relies on the quality of the input image. In complex scenarios such as uneven lighting, blurred lane markings, or target occlusion, its detection performance drops sharply, especially after lane lines are briefly and completely obscured, making it difficult for the system to recover quickly and leading to a significant decrease in the reliability of the output results. The other type is the absolute constraint solution based on high-precision maps. This type directly projects pre-stored map lane line information into the image space as the perception result. It lacks the ability to adapt to real-time dynamic vehicle offsets, and when road construction or temporary changes to lane markings occur in the park without timely map updates, it outputs incorrect guidance information, posing serious safety hazards.

[0004] Therefore, there is an urgent need for a lane line perception method and system for parks based on multi-source data fusion and prior map constraints, which can effectively integrate vehicle motion state, high-precision map prior knowledge and visual multi-feature information to improve the robustness and accuracy of lane line detection in complex scenarios. Summary of the Invention

[0005] The main objective of this invention is to provide a method and system for sensing lane lines in a park based on multi-source data fusion and prior map constraints, which solves the problems of unclear lane line feature perception and complex and variable environment in the prior art, reduces environmental interference, reduces false detection and missed detection of lane lines, and maintains stable tracking of lane lines through kinematic model prediction extrapolation.

[0006] To solve the above-mentioned technical problems, the technical solution adopted in this invention is a park lane line perception method based on multi-source data fusion and prior map constraints, including the following steps:

[0007] S1. Collect multi-source data and preprocess the images. The multi-source data includes image data, vehicle motion state data, and vehicle positioning data. S2. Extract and fuse multiple visual features from the preprocessed image to generate a candidate lane line heatmap. S3. Obtain the prior information of lane lines in the high-precision map, combine it with the real-time positioning information, project the prior information of lane lines onto the image coordinate system, and perform spatial filtering on the candidate points in the candidate lane line heat map. S4. Perform model fitting on the filtered set of valid lane line points to obtain the lane line observation values ​​of the current frame; S5. By combining vehicle motion status and historical lane line information, the lane line status is tracked and predicted. S6. Calculate the confidence level of lane line detection by combining multiple dimensions of indicators, and perform graded processing based on the confidence level.

[0008] Preferably, in step S2, the various visual features include color features, edge features, and semantic features; Weighted fusion of color features, edge features, and semantic features is performed to generate a fused heatmap. Threshold segmentation and connected component analysis are then performed on the fused heatmap to obtain a set of candidate lane line pixels.

[0009] Preferably, in step S3, the lane line elements in the high-precision map are projected from the world coordinate system to the current image coordinate system to form a strip-shaped search area; Only the pixels that fall within the strip search area from the candidate points are retained to form the effective point set of the left and right lane lines.

[0010] Preferably, in step S4, principal component analysis is performed on the effective lane line point set, and a straight line model or a curve model is adaptively selected based on the analysis results. The random sampling consensus algorithm is used to estimate the parameters of the selected model, remove noise points, and output the lane line observation values ​​of the current frame.

[0011] Calculate the principal direction of the effective point set in the image coordinate system. If the eigenvalue of the second principal component is less than the preset threshold, it is determined to be a straight line and a straight line model is adopted; otherwise, a quadratic curve model is adopted.

[0012] Preferably, in step S5, a state vector containing lane line model parameters and their rate of change is defined; based on the vehicle kinematics model and the current vehicle speed and yaw rate, a priori prediction of the lane line state at the next moment is made. The lane line observations obtained in step S4 are used as update values. The predicted state is corrected by Kalman filtering, and the updated lane line state is output.

[0013] Preferably, in step S6, the confidence level is obtained by weighted summation based on the proportion of detected lane line segment lengths, semantic segmentation probability, inlier rate of random sampling consistency fitting, and trace of covariance matrix of state estimation during tracking.

[0014] Preferably, in step S6, when the confidence level is higher than the first threshold, it is determined to be a high-reliability mode, and the current lane line result is output; When the confidence level is between the first threshold and the second threshold, it is determined to be in downgrade mode, and a high-precision map prior is introduced to apply soft constraints to the tracking results. When the confidence level is lower than the second threshold, it is determined to be an abnormal mode, the predicted status is output and a downgrade warning is issued.

[0015] This invention provides a park lane perception system based on multi-source data fusion and prior map constraints, comprising: The data acquisition module is used to collect image data, vehicle motion status data, and vehicle positioning data; An image preprocessing module is used to perform distortion correction, noise reduction, and illumination equalization on the image data; The multi-feature fusion lane line detection module is used to extract and fuse multiple visual features to generate a candidate lane line heatmap. The high-precision map module is used to store and provide prior information on lane lines for roads within the park. The filtering and verification module is used to perform spatial filtering and verification of candidate lane line segments by combining the lane line prior information and the vehicle real-time positioning information. The lane line model fitting and optimization module is used to perform model fitting on the filtered set of valid lane line points to obtain lane line observations. The tracking module is used to track and predict lane line status by combining vehicle motion status and historical lane line information. The confidence assessment module is used to calculate the confidence level of lane line detection; The exception handling module is used to perform corresponding exception operations based on the confidence level; The control decision interface is used to output the final lane line information to the autonomous driving decision control system.

[0016] Preferably, the multi-feature fusion lane line detection module extracts multiple visual features including color features, edge features, and semantic features; the multi-feature fusion lane line detection module generates a candidate lane line heatmap by weighted fusion of the multiple visual features. Beneficial effects: 1. Significantly improves the robustness of perception in complex environments. By simultaneously extracting three complementary features—color, edge, and semantic—and performing weighted fusion, this invention effectively solves the problem of feature loss caused by severe wear of road markings and drastic changes in lighting in park scenarios. Even when a single feature is not obvious, accurate candidate heatmaps can still be generated by relying on redundant fusion of multiple features, thus reducing the false negative rate.

[0017] 2. Effectively reduces the false detection rate caused by environmental noise. This invention introduces high-precision map prior information adapted to autonomous driving, which is constructed offline. By projecting the map lane lines in the prior map onto the image space to construct a "strip search area", a large amount of background interference unrelated to lane lines is eliminated from the spatial dimension, so that the system can eliminate non-target noise points while retaining the vehicle's deflection flexibility.

[0018] 3. Ensures the continuity of perception when the target is occluded. Addressing the pain point of instantaneous occlusion caused by frequent pedestrians and vehicles in the park, this invention couples the vehicle kinematics model with the Kalman filter algorithm. In the event of short-term loss of visual observations, the system can use the vehicle's current longitudinal speed and yaw rate to extrapolate and predict the trajectory, ensuring the smoothness and temporal continuity of lane line parameter output and avoiding abrupt instructions from the decision layer due to perception interruption.

[0019] 4. Enhanced system adaptability and model fitting accuracy: By using PCA principal component analysis to determine the point cloud morphology online in real time, this invention achieves adaptive switching between straight line model and quadratic curve model. Compared with a fixed model, this scheme is more suitable for the road characteristics of alternating straight lines and gentle curves in the park. Combined with iterative optimization of the RANSAC algorithm, it further ensures the parameter fitting accuracy even with a small number of outlier noise points.

[0020] 5. A quantitative safety early warning mechanism is provided. This invention constructs a multi-dimensional quantitative confidence evaluation system, which comprehensively considers the fitting quality and tracking stability, providing an important reference indicator for the autonomous driving decision-making system. This enables the system to take timely degraded responses in low-confidence scenarios, significantly improving the safety and reliability of unmanned operation in the park. Attached Figure Description

[0021] The present invention will be further described below with reference to the accompanying drawings and embodiments: Figure 1 This is a schematic diagram of the multi-feature fusion lane line detection process of the present invention; Figure 2 This is a flowchart of the park lane line perception system based on multi-source data fusion and prior map constraints of the present invention. Detailed Implementation

[0022] Example 1 like Figure 1 As shown, a method for perceiving lane lines in a park based on multi-source data fusion and prior map constraints includes the following steps: S1. Multi-source data acquisition and preprocessing: acquire motion data and obtain images, and perform illumination equalization, noise reduction and correction on the images. S2. Lane detection with multi-feature fusion: The system simultaneously extracts three complementary features: color, edge, and semantics to cope with complex environments. The above feature maps are input into the fusion network and generated by weighting to form candidate heatmaps. In this example, the heatmap refers to the probability statistical map used to display confidence or distribution density during the construction / verification of high-precision maps. S3. Based on the prior and context of the high-precision map, the road direction and number of lanes in the high-precision map are projected onto the image coordinate system using real-time positioning information to verify the parallelism of the left and right candidate lane lines and eliminate false detection interference located in semantic areas such as pedestrians and vehicles. S4. Model adaptive fitting and optimization: Select the model based on the road curvature, apply the RANSAC (Random Sample Consensus) algorithm for parameter estimation, and iteratively eliminate outlier noise. S5, Kalman filter tracking with kinematic coupling, is based on the bicycle kinematic model. It combines the current vehicle speed and yaw rate to make prior predictions of the lane line parameters at the next moment and uses the fitting results of S4 as observations to update the filter state. S6. Quantitative Confidence Assessment and Anomaly Response: The system calculates the comprehensive confidence level and evaluates the system's high reliability mode, degradation mode, and anomaly mode based on weighting factors such as detection length ratio, semantic probability, fitting inlier rate, and filter certainty.

[0023] According to an embodiment of the present invention, in step S1, the data acquisition unit synchronously acquires the following data at a frequency of 100Hz: Image data is obtained by using a forward-facing camera with a resolution of 1920×1080, a field of view of 120°, and a frame rate of 30fps to capture raw RGB images for subsequent lane line feature extraction.

[0024] The pose data is fused with GNSS+RTK and IMU to output the vehicle's pose (X, Y, Z, Roll, Pitch, Yaw) in the ground coordinate system. The X direction refers to the longitudinal direction of the vehicle, the Y direction refers to the lateral direction of the vehicle, and the Z direction refers to the vertical direction. Roll is the roll around the X axis, Pitch is the pitch around the Y axis, and Yaw is the yaw around the Z axis.

[0025] Motion data is obtained from the vehicle's longitudinal speed via the CAN bus, derived from wheel speed pulse signals. and yaw rate from IMU .

[0026] All data are tagged with a uniform timestamp and interpolated and aligned based on the timestamp to ensure the spatiotemporal synchronization of multi-source data. In this embodiment, the synchronization error between image data, pose data, and motion data is controlled within ±5ms, ensuring the accuracy of subsequent data fusion.

[0027] According to an embodiment of the present invention, in step S2, the image data is preprocessed, and distortion correction is performed using the camera intrinsic parameter matrix K and distortion coefficients [k1, k2, p1, p2] through a Brownian model. The method of the present invention applies Gaussian filtering (kernel size 5×5, σ=1.5) for noise reduction and uses the CLAHE algorithm (Clip Limit=2.0, Tile Grid Size=8×8) for illumination equalization. Simultaneously, three features are extracted and fused in parallel: color features convert the image to HSV space, set thresholds ([0, 0, 200], [180, 30, 255]) for white lane lines, and set thresholds ([15, 100, 100], [35, 255, 255]) for yellow lane lines, and perform threshold segmentation to obtain a binary mask; edge features apply Canny edge detection (low threshold) to the grayscale image. High threshold Then, morphological closing operations are performed to connect the breakpoints; semantic features are obtained using the lightweight segmentation network MobileNet through forward inference to obtain the probability map of lane lines; feature fusion uses weighted fusion to generate the final heatmap. As shown in equation (1): (1) in, For color characteristics, For edge features, Semantic features, weight coefficients , , ,and .right Threshold segmentation and connected component analysis are performed to obtain the candidate lane line pixel set. .

[0028] According to an embodiment of the present invention, in step S3, based on the prior candidate area filtering of the high-precision map, the high-precision map of the park is loaded. Based on the current vehicle pose, the Lanelet elements of the current lane and adjacent lanes in the map are queried to obtain the prior position information of the left and right lane lines. The left and right lane lines LineString in the map are then transferred from the world coordinate system... Transforming to the current image coordinate system yields two prior search region bands. and Bandwidth set to The pixels form a band-shaped search region of a certain width, rather than a single line constraint. The candidate point set obtained in step S2... Filter and retain only those that fall under or The pixels within the region constitute the effective point sets for the left and right lane lines, respectively. and This step utilizes map priors to eliminate over 90% of background noise interference, such as road stains, tire tracks, and shadow edges, while retaining the actual lane line points when the vehicle deviates from the lane center, thus avoiding rigid matching failure caused by absolute projection constraints.

[0029] According to an embodiment of the present invention, in step S4, considering that the roads in the park are mostly straight lines or gentle curves, an adaptive model selection strategy is adopted. The principal direction of the effective point set P in the image coordinate system is calculated through PCA analysis. If the second principal component eigenvalue... , If the line is straight, it is determined to be a straight line, and the straight line model is as shown in equation (2): (2) Otherwise, the quadratic curve model is used as shown in equation (3): (3) right and RANSAC fitting was performed separately. For the linear model, two points were randomly selected in each iteration to calculate the linear parameters. Interior point distance threshold Pixels, maximum number of iterations Finally, the model with the highest interior point ratio was selected as the optimal solution. For the curve model, three points were randomly selected in each iteration to calculate the curve parameters. Interior point distance threshold Pixels. Output the parameters corresponding to the model with the highest inlier rate, denoted as the current observation value. .

[0030] According to an embodiment of the present invention, in step S5, a vehicle kinematics model is introduced to realize the temporal prediction and update of lane line states, solving the detection interruption problem caused by short-term occlusion. The state vector is defined to include lane line model parameters and their rates of change; for a straight line model... ,in The slope The intercept is... , For the parameter change rate, for the curve model The prediction based on the vehicle kinematics model is shown in equation (4): (4) in, The state transition matrix is ​​formed by fusing the vehicle's longitudinal velocity. and yaw rate , The lateral offset variation in the linear model represents process noise. Relationship with vehicle motion As shown in equation (5): (5) in, For vehicle heading angle deviation, The sampling period.

[0031] The observed values ​​obtained by fitting S4 With predicted state Perform fusion and calculate Kalman gain. Update the optimal state estimate Covariance Matrix As shown in equation (6): (6) in, To predict the covariance matrix, For the observation matrix, To observe the noise covariance, Kalman filtering is used to smoothly fuse the detection results of the current frame with historical states, improving the continuity and stability of lane line estimation.

[0032] According to an embodiment of the present invention, in step S6, the lane line detection confidence level is calculated by comprehensively considering multi-dimensional indicators. As shown in equation (7): (7) in, , , , , Based on experience, The RANSAC in-point ratio reflects the quality of the fit. The mean square error of the fit. The trace of the state estimation covariance matrix reflects the tracking uncertainty.

[0033] Based on confidence level Adopt a tiered processing strategy: High confidence level ( and ): Outputs high-confidence lane line results, enabling the control system to perform automatic lane keeping; Medium confidence level (either side) ): Outputs a result with medium confidence, triggering the map prior assistance mode. The lane line model from the high-precision map is used to apply soft constraints to the tracking results to prevent state estimation divergence, while simultaneously prompting the control system to drive cautiously; Low confidence (either side) ): Determined as a detection failure, output predicted status. It will immediately issue a downgrade warning, prompting the control system to reduce driving speed, increase the distance from the vehicle in front, and switch to manual assistance mode when appropriate.

[0034] Example 2 like Figure 2 As shown, a park lane perception system based on multi-source data fusion and prior map constraints includes the following modules: The data acquisition module includes a forward-facing camera, IMU, wheel speed odometer, and GPS+RTK positioning module, used to collect environmental image data and vehicle motion status data. The image preprocessing module, connected to the data acquisition module, is used to perform distortion correction, noise reduction, and illumination equalization on the image. The multi-feature fusion lane detection module is connected to the image preprocessing module and is used to extract color, edge and semantic features and fuse them to generate candidate lane heatmaps. The high-precision map module stores and provides prior information on lane lines and positioning references for roads within the park. The screening and verification module is connected to the multi-feature fusion lane line detection module and the high-precision map module, respectively, and is used to screen and verify candidate lane line segments by combining the prior information of the high-precision map and contextual constraints. The lane line model fitting and optimization module is connected to the screening and verification module and is used to perform model fitting and optimization on valid lane line segments. The tracking module is connected to the module fitting and optimization module and the data acquisition module, respectively. It is used to track and predict by combining the vehicle motion state and historical lane line information, and output a smooth trajectory. The confidence assessment module, connected to the lane line tracking module, is used to assess the confidence level and trigger anomaly handling when the confidence level is low. The exception handling module is connected to the confidence assessment module. It marks the status of the confidence assessment module and performs exception handling according to different confidence levels. The control decision interface, connected to the confidence assessment module, is used to output the final lane line information to the autonomous driving decision control system.

[0035] The above embodiments are merely preferred technical solutions of the present invention and should not be considered as limitations on the present invention. The scope of protection of the present invention should be limited to the technical solutions described in the claims, including equivalent substitutions of the technical features described in the claims. That is, equivalent substitutions and improvements within this scope are also within the scope of protection of the present invention.

Claims

1. A park lane line perception method based on multi-source data fusion and prior map constraint, characterized in that, Includes the following steps: S1. Collect multi-source data and preprocess the images. The multi-source data includes image data, vehicle motion state data, and vehicle positioning data. S2. Extract and fuse multiple visual features from the preprocessed image to generate a candidate lane line heatmap. S3. Obtain the prior information of lane lines in the high-precision map, combine it with the real-time positioning information, project the prior information of lane lines onto the image coordinate system, and perform spatial filtering on the candidate points in the candidate lane line heat map. S4. Perform model fitting on the filtered set of valid lane line points to obtain the lane line observation values ​​of the current frame; S5. By combining vehicle motion status and historical lane line information, the lane line status is tracked and predicted. S6. Calculate the confidence level of lane line detection by combining multiple dimensions of indicators, and perform graded processing based on the confidence level.

2. The method of claim 1, wherein: In step S2, various visual features include color features, edge features, and semantic features; Weighted fusion of color features, edge features, and semantic features is performed to generate a fused heatmap. Threshold segmentation and connected component analysis are then performed on the fused heatmap to obtain a set of candidate lane line pixels.

3. The method of claim 1, wherein: In step S3, the lane line features in the high-precision map are projected from the world coordinate system to the current image coordinate system to form a strip-shaped search area; Only the pixels that fall within the strip search area from the candidate points are retained to form the effective point set of the left and right lane lines.

4. The method of claim 1, wherein: In step S4, principal component analysis is performed on the effective lane line point set, and a straight line model or a curve model is adaptively selected based on the analysis results. The random sampling consensus algorithm is used to estimate the parameters of the selected model, remove noise points, and output the lane line observation values ​​of the current frame.

5. The method of claim 4, wherein: Calculate the principal direction of the effective point set in the image coordinate system. If the eigenvalue of the second principal component is less than the preset threshold, it is determined to be a straight line and a straight line model is adopted; otherwise, a quadratic curve model is adopted.

6. The method of claim 1, wherein: In step S5, a state vector containing lane line model parameters and their rate of change is defined; based on the vehicle kinematics model and the current vehicle speed and yaw rate, a priori prediction of the lane line state at the next moment is made. The lane line observations obtained in step S4 are used as update values. The predicted state is corrected by Kalman filtering, and the updated lane line state is output.

7. The method of claim 1, wherein: In step S6, the confidence level is obtained by weighted summation based on the proportion of detected lane line segment lengths, semantic segmentation probability, inlier rate of random sampling consistency fitting, and trace of covariance matrix of state estimation during tracking.

8. The method according to claim 1, characterized in that: In step S6, when the confidence level is higher than the first threshold, it is determined to be a high-reliability mode, and the current lane line result is output. When the confidence level is between the first threshold and the second threshold, it is determined to be in downgrade mode, and a high-precision map prior is introduced to apply soft constraints to the tracking results. When the confidence level is lower than the second threshold, it is determined to be an abnormal mode, the predicted status is output and a downgrade warning is issued.

9. A park lane line perception system based on multi-source data fusion and prior map constraints, characterized in that, include: The data acquisition module is used to collect image data, vehicle motion status data, and vehicle positioning data; An image preprocessing module is used to perform distortion correction, noise reduction, and illumination equalization on the image data; The multi-feature fusion lane line detection module is used to extract and fuse multiple visual features to generate a candidate lane line heatmap. The high-precision map module is used to store and provide prior information on lane lines for roads within the park. The filtering and verification module is used to perform spatial filtering and verification of candidate lane line segments by combining the lane line prior information and the vehicle real-time positioning information. The lane line model fitting and optimization module is used to perform model fitting on the filtered set of valid lane line points to obtain lane line observations. The tracking module is used to track and predict lane line status by combining vehicle motion status and historical lane line information. The confidence assessment module is used to calculate the confidence level of lane line detection; The exception handling module is used to perform corresponding exception operations based on the confidence level; The control decision interface is used to output the final lane line information to the autonomous driving decision control system.

10. The system according to claim 9, characterized in that: The multi-feature fusion lane line detection module extracts various visual features, including color features, edge features, and semantic features; The multi-feature fusion lane line detection module generates a candidate lane line heatmap by weighted fusion of multiple visual features.

Images