Multi-feature fusion adaptive filtering lane line detection and front vehicle distance measurement method
Through the multi-feature fusion adaptive filtering method, combined with edge, color and texture characteristics, the accuracy and stability problems of existing lane line detection in complex environments are solved, and efficient lane line detection and forward vehicle distance measurement are achieved.
Patent Information
- Application Number
- CN202510390161.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-31
- Publication Date
- 2025-07-18
AI Technical Summary
The existing lane line detection methods have low detection accuracy under the influence of light changes, shadows, lane line wear and other factors, and it is difficult to maintain stable detection performance in complex road environments, and have failed to make full use of historical information processing abnormal situations.
The multi-feature fusion adaptive filtering method is used to detect lane line by combining edge, color and texture features. The lane line model parameters are updated using adaptive filtering algorithm, and abnormal situations are processed through historical frame matching to calculate the distance in front of the car.
It improves the accuracy and stability of lane line detection, enhances the ability to handle abnormal situations, and realizes reliable lane line detection and forward vehicle distance measurement in complex environments.
Smart Images
Figure BDA0005337193020000021 
Figure BDA0005337193020000031 
Figure BDA0005337193020000032
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of vehicle assisted driving, and particularly relates to a multi-feature fusion adaptive filtering lane line detection and vehicle distance measurement method for the vehicle ahead. Background Art
[0002] With the development of intelligent transportation systems, vehicle assisted driving technology has become a research hotspot. Among them, lane line detection is one of the key technologies to realize advanced driving assistance functions such as lane departure warning and adaptive cruise control.
[0003] Existing lane line detection methods mainly rely on image segmentation and line detection, and are easily affected by factors such as illumination changes, shadows, and lane line wear, resulting in low detection accuracy. At the same time, in complex road environments, such as curves, unclear lane lines, vehicle occlusion, etc. And existing methods are difficult to maintain stable detection performance, and most methods only rely on the current frame image for detection, failing to fully utilize the historical information of lane line detection, resulting in insufficient ability to handle abnormal situations. Summary of the Invention
[0004] In order to solve the deficiencies existing in the prior art, the present application proposes a multi-feature fusion adaptive filtering lane line detection and vehicle distance measurement method for the vehicle ahead to overcome the problems existing in the prior art and improve the stability of detection performance and processing ability.
[0005] The technical solution adopted by the present invention is as follows:[[]]
[0006] A multi-feature fusion adaptive filtering lane line detection method includes the following steps:[[]]
[0007] Step 1, obtaining the current frame lane line image and performing image preprocessing;
[0008] Step 2, respectively extracting lane line features, color features, and texture features based on the preprocessed image;
[0009] Step 3, performing weighted fusion on the edge features, color features, and texture features;
[0010] Step 4, extracting lane line candidate regions from the weighted fusion feature map;
[0011] Step 5, extracting multiple straight lines from the lane line candidate regions as initial lane line candidates; grouping the candidate straight lines according to the position relationship of the lane lines in the image to obtain initial left and right lane line models;
[0012] Step 6, matching the lane line model regions of the current frame and the previous frame. If the region matching rate is higher than a preset threshold, the region matching is successful, and an adaptive filtering algorithm is used to update the lane line model parameters;
[0013] Step 7, if the region matching fails, it indicates an abnormal situation in the current frame; the lane line candidate region detected in the current frame is used as a temporary lane line model; the temporary lane line model is matched with the lane line models of historical frames. If the match is successful, the current anomaly is considered a transient interference and the temporary lane line model is updated.
[0014] If anomalies occur continuously for multiple times, the temporary lane line model is used as the new lane line model.
[0015] Furthermore, the Sobel operator is used to calculate the gradient intensity and direction of each pixel point in the image, an edge image is generated based on the gradient intensity and direction, and lane line features are extracted.
[0016] Furthermore, the image is converted to the HSV color space, the pixel regions of the lane line color are extracted, and an adaptive threshold segmentation method is adopted to extract the lane line color features.
[0017] Furthermore, the Gabor filter is used to extract the texture information in the image, the texture direction histogram of the local region of the image is calculated, and the pixels with lane line direction features are screened out.
[0018] Furthermore, a weighted fusion method is adopted to fuse the edge, color, and texture features, denoted as: F(x,y) = w e ·E(x,y) + w C ·C(x,y) + w t ·T(x,y)
[0019] where: F(x,y) is the fused feature value, E(x,y), C(x,y), and T(x,y) are the edge feature value, color feature value, and texture feature value respectively, and w e , w c , w t are the weights of the edge feature, color feature, and texture feature respectively.
[0020] Furthermore, the region matching rate is the ratio of the overlapping region area, denoted as:
[0021]
[0022] where, S overlap is the overlapping region area, and S model is the area of the lane line model region in the previous frame.
[0023] Furthermore, Gaussian filtering is used for image preprocessing.
[0024] Furthermore, for texture direction histogram calculation: for each pixel point in the image, Gabor filters with different directions are applied, the response intensity is calculated, the response intensity in each direction is statistically counted, and a texture direction histogram is generated.
[0025] A method for measuring the distance of the vehicle in front based on a multi - feature fusion adaptive filtering lane line includes the following steps:
[0026] Step 1, parameter definition: Set the vertical height of the reference target relative to the horizontal road surface as h2; Set the vertical height of the visual acquisition system relative to the horizontal road surface as h1; Set the distance between the visual acquisition system and the reference target as L stand ;
[0027] Step 2, obtain the lane line image, and perform lane line detection based on the above - mentioned multi - feature fusion adaptive filtering lane line detection method to obtain a lane line model;
[0028] Step 3, extend the detected lane line model along the vehicle driving direction to the position of the target vehicle;
[0029] Step 4, according to the imaging width n of the lane line in the image road and the focal length f of the visual acquisition system, calculate the vertical distance L' from the visual acquisition system to the lane line stand , the actual distance L of the reference target stand , the straight - line distance L from the visual acquisition system to the lane line of the target vehicle target , the horizontal distance d between the host vehicle and the target vehicle target .
[0030] Furthermore, the horizontal distance d between the host vehicle and the target vehicle target is expressed as:
[0031]
[0032] Advantages of the present invention:
[0033] The present invention realizes multi - feature fusion, uses the combination of edge, color and texture features to improve the accuracy of lane line detection. At the same time, it has adaptive filtering update, can dynamically adjust the filter parameters according to the matching degree, enhances the ability to handle abnormal situations, and does not require external parameter calibration: Utilize the relationship between the reference target and the lane line to achieve accurate measurement of the distance between vehicles without complex parameter calibration of the camera.
[0034] The present invention also relates to a method for obtaining lane line images, extending lane lines, and calculating the distance of a selected reference point and the distance of the target vehicle based on the imaging width of the lane line and the focal length of the visual acquisition system. This method not only improves the accuracy of lane line detection, but also can effectively handle abnormal situations and ensure the reliability of the distance measurement from the vehicle in front. Description of the Drawings
[0035] Figure 1 is a flowchart of a multi - feature fusion adaptive filtering lane line detection method of the present invention.
[0036] Figure 2 It is a schematic diagram of the distance measurement of the vehicle in front. Specific implementation mode
[0037] In order to make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.
[0038] Embodiment 1
[0039] A multi-feature fusion adaptive filtering lane line detection method includes.
[0040] Step 1, image acquisition and preprocessing: Use a vision acquisition system to capture the road ahead to obtain the current frame of lane line image; perform grayscale processing on the obtained current frame of lane line image to reduce the amount of calculation, and use Gaussian filtering to smooth the image and suppress noise.
[0041] Step 2, based on the image preprocessed in Step 1, extract lane line features, color features, and texture features respectively.
[0042] (1) Lane line feature extraction: Perform edge detection on the preprocessed image to obtain an edge image.
[0043] The specific steps are as follows: Use the Sobel operator to calculate the gradient intensity and direction of each pixel point in the image, and generate an edge image according to the gradient intensity and direction to achieve lane line feature extraction; specifically as follows:
[0044] Definition of the Sobel operator:
[0045]
[0046]
[0047] Among them, G x represents the change rate of the image in the horizontal direction, and the matrix represents its corresponding convolution kernel. G y is the change rate in the vertical direction, and the matrix represents its corresponding convolution kernel.
[0048] Gradient calculation:
[0049] For each pixel point I(x, y) in the image, the components of its gradient in the x and y directions are respectively:
[0050] G x (x, y) = (I(x + 1, y - 1) + 2I(x + 1, y) + I(x + 1, y + 1)) - (I(x + 1, y - 1) + 2I(x - 1, y) + I(x + 1, y + 1))
[0051] G y G(x,y) = (I(x - 1,y + 1)+2I(x,y + 1)+I(x + 1,y + 1))-(I(x - 1,y - 1)+2I(x,y - 1)+I(x + 1,y - 1))
[0052] where G x G(x,y) is the component of the gradient in the x - direction, and G y G(x,y) is the component of the gradient in the y - direction.
[0053] Gradient intensity and direction:
[0054] Gradient intensity:
[0055]
[0056] Gradient direction:
[0057]
[0058] (2) Color feature extraction: Convert the image to the HSV color space, extract the pixel region of the lane line color (usually white or yellow), and adopt an adaptive threshold segmentation method to dynamically adjust the threshold according to the overall brightness of the image to improve the adaptability to different lighting conditions.
[0059] Convert the RGB image to the HSV color space to more effectively extract the lane line color features. RGB - to - HSV conversion formula: Let R, G, B be the red, green, and blue components of the pixel, and R, G, B ∈ [0,1];
[0060] C max = max(R, G, B), C min = min(R, G, B), Δ = C max - C min (5)
[0061]
[0062]
[0063] V = C max (8)
[0064] Color threshold segmentation:
[0065] Set the HSV threshold range according to the lane line color (white or yellow), for example:
[0066] White: Hue H ∈ [0°, 360°], Saturation S ∈ [0, 0.3], Value V ∈ [0.5, 1];
[0067] Yellow: Hue H ∈ [20°, 40°], Saturation S ∈ [0.5, 1], Value V ∈ [0.5, 1];
[0068] Mark the pixel as a candidate lane line point if its HSV value is within the above range.
[0069] (3) Texture feature extraction: Use Gabor filters to extract the texture information in the image. Lane lines usually have obvious directional texture features. Calculate the texture direction histogram of the local area of the image and filter out the pixels with lane line direction features.
[0070] Gabor filters are used to extract directional texture features in the image.
[0071] Gabor filter kernel function:
[0072]
[0073] x' = xcosθ + ysinθ (10)
[0074] y' = -xsinθ + ycosθ (11)
[0075] where λ is the wavelength, which determines the frequency of the filter response, θ is the direction, which determines the direction of the filter response, ψ is the phase offset, σ is the standard deviation of the Gaussian envelope, which controls the bandwidth of the filter, and γ is the spatial aspect ratio.
[0076] Calculation of the texture direction histogram: For each pixel point in the image, apply Gabor filters in different directions and calculate its response intensity. Statistically calculate the response intensity in each direction to generate the texture direction histogram. Filter out the pixel points with obvious lane line direction features (for example, perpendicular to the lane line direction).
[0077] Step 3, multi-feature fusion: Fuse the edge features, color features, and texture features to obtain a comprehensive feature map.
[0078] Adopt a weighted fusion method, dynamically adjust the weights according to the importance of different features in different scenarios. For example, in good lighting conditions, the weight of the color feature is higher; in weak lighting or more shadowy conditions, the weights of the edge and texture features are higher.
[0079] Adopt a weighted fusion method to fuse the edge, color, and texture features:
[0080] F(x, y) = w e ·E(x, y) + w C ·C(x, y) + w t ·T(x, y) (12)
[0081] Where: F(x, y) is the fused eigenvalue. E(x, y), C(x, y), T(x, y): are the edge eigenvalue, color eigenvalue, and texture eigenvalue respectively. w e , w c , w t are the weights of the edge feature, color feature, and texture feature respectively, satisfying w e + w c + w t = 1. The weights can be dynamically adjusted according to different scenarios. For example: in good lighting conditions: w e = 0.3, w c = 0.5, w t = 0.2; in weak lighting or more shadowy conditions: w e = 0.5, w c = 0.3, w t = 0.2.
[0082] Step 4, generation of lane line candidate regions: Binarize the weighted fused feature map to obtain lane line candidate regions, and use morphological operations (dilation, erosion) to remove noise and small connected regions, retaining the larger lane line candidate regions.
[0083] Step 5, initialization of the lane line model: Use the lane line candidate regions generated in Step 6 as the initial frame image, perform line detection (such as Hough transform) on the initial frame image, extract multiple lines as initial lane line candidates, and group the initial lane line candidates according to their position relationships in the image (for example, the left and right lane lines have the same longitudinal position and the smallest lateral position difference) to obtain the initial left and right lane line models.
[0084] Step 6, update of the lane line model:
[0085] Region matching: Extract lane line features from the current frame image to obtain the lane line candidate regions of the current frame image, match the lane line candidate regions of the current frame image with those of the previous frame image, and calculate the ratio of the overlapping area to the total area (region matching rate); if the region matching rate is higher than a preset threshold (for example, 50%), the matching is successful, and it is considered that the lane lines detected in the current frame are the same as those in the previous frame.
[0086] Calculate the ratio of the overlapping area of the lane line model regions between the current frame and the previous frame:
[0087]
[0088] Where: S overlap is the overlapping area. S model is the area of the lane line model region of the previous frame. If P matchIf it is ≥ 50%, the matching is successful.
[0089] Adaptive filtering update: If the region matching is successful, the lane line model parameters are updated using an adaptive filtering algorithm (such as Kalman filtering), including the lane line position, direction, curvature, etc. The update coefficient of the filter is dynamically adjusted according to the matching degree between the current frame and the previous frame. The higher the matching degree, the larger the update coefficient, and vice versa, to enhance the robustness to abnormal situations.
[0090] Set the state vector:
[0091]
[0092] Where: x, y are the lane line position coordinates. θ is the lane line direction angle. κ is the lane line curvature.
[0093] So the state transition matrix can be obtained:
[0094]
[0095] Where: v is the vehicle speed, and Δt is the time interval.
[0096] Observation matrix:
[0097]
[0098] Kalman filtering update steps:
[0099] Prediction step:
[0100]
[0101] P k|k-1 = FP k-1|k-1 F T + Q (18)
[0102] Where: is the state estimate at time step k, based on the information at time step k - 1, P k|k-1 is the covariance matrix at time step k, based on the information at time step k - 1, and Q is the process noise.
[0103] Update step:
[0104] K k = P k|k-1 H T (HP k|k-1 H T + R) -1 (19)
[0105]
[0106] P k|k=(I - K k H)P k|k-1 (21)
[0107] Where: K k is the Kalman gain matrix, is the final state estimate at time step k, P k|k is the final covariance matrix at time step k, z k is the current frame observation value (lane line position coordinates), and R is the observation noise covariance matrix.
[0108] Update coefficient adjustment: According to the matching degree P match adjust the update coefficient K of the Kalman filter k , for example:
[0109]
[0110] When the matching degree is high, the update coefficient is small, and the filter trusts the predicted value more; conversely, when the matching degree is low, the update coefficient is large, and the filter trusts the observed value more.
[0111] Step 7, anomaly detection: If the region matching fails, it is considered that an abnormal situation occurs in the current frame. Start the anomaly handling mechanism:
[0112] Step 7.1, establish a temporary lane line model: Use the lane line candidate region detected in the current frame as the temporary lane line model.
[0113] Step 7.2, historical information matching: Match the temporary lane line model with the lane line models in historical frames. If the match is successful, it is considered that the current anomaly is a short-term interference, and update the temporary lane line model.
[0114] Step 7.3, anomaly accumulation judgment: If anomalies occur continuously for multiple times (e.g., exceeding a preset number of frames), then use the temporary lane line model as the new lane line model and reset the anomaly counter.
[0115] Step 7.4, lane line model output: Output the updated lane line model as the lane line detection result of the current frame.
[0116] Embodiment 2
[0117] Based on the above multi-feature fusion adaptive filtering lane line detection method, the present invention also proposes a method for measuring the distance to the vehicle in front of a multi-feature fusion adaptive filtering lane line, and the specific steps are as follows:
[0118] Step 1, parameter definition: Set the vertical height of the reference target (e.g., a fixed point at the front end of the vehicle) relative to the horizontal road surface as h2; set the vertical height of the vision acquisition system (camera) relative to the horizontal road surface as h1; set the distance between the vision acquisition system and the reference target as Lstand ;
[0119] Step 2, Lane line image acquisition: Use a vision acquisition system to capture the road ahead, obtain the lane line image of the current frame, that is, use a camera to capture the road image ahead and perform lane line detection to obtain a lane line model. Specifically refer to a multi-feature fusion adaptive filtering lane line detection method proposed in Embodiment 1.
[0120] Step 3, Lane line extension: Extend the detected lane line model along the vehicle driving direction to the position of the target vehicle, that is, extend the detected lane line model along the vehicle driving direction to the target vehicle position.
[0121] Step 4, Distance calculation:
[0122] Without considering the influence of other factors, perform reference distance calculation:
[0123] According to the imaging width n of the lane line in the image road and the focal length f of the vision acquisition system, calculate the vertical distance L' from the vision acquisition system to the lane line stand :
[0124]
[0125] where d road is the width of the actual lane line, so according to the geometric relationship, calculate the actual distance L from the vision acquisition system to the reference target stand :
[0126]
[0127] Target distance calculation:
[0128] Similarly, according to the lane line imaging width n' of the target vehicle in the image road , calculate the straight-line distance L from the vision acquisition system to the lane line of the target vehicle target :
[0129]
[0130] Then, according to the geometric relationship, calculate the horizontal distance d between the host vehicle and the target vehicle target :
[0131]
[0132] The above embodiments are only used to illustrate the design concept and features of the present invention, and the purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, all equivalent changes or modifications made according to the principles and design concepts disclosed by the present invention are within the protection scope of the present invention.
Claims
1. A multi-feature fusion adaptive filtering lane detection method, characterized in that It includes the following steps: Step 1: Obtain the current frame of lane line image and perform image preprocessing; Step 2: Based on the preprocessed image, extract lane line features, color features, and texture features respectively; Step 3: Perform weighted fusion on the edge features, color features, and texture features; Step 4: Extract the lane line candidate region from the weighted fusion feature map; Step 5: Extract multiple straight lines from the lane line candidate region as the initial lane line candidates; Group the candidate straight lines according to the position relationship of the lane lines in the image to obtain the initial left and right lane line models; Step 6: Match the lane line model region of the current frame with that of the previous frame. If the region matching rate is higher than the preset threshold, the region matching is successful, and the lane line model parameters are updated using an adaptive filtering algorithm; Step 7: If the region matching fails, it means that an abnormal situation occurs in the current frame; use the lane line candidate region detected in the current frame as the temporary lane line model; match the temporary lane line model with the lane line models of historical frames. If the match is successful, it is considered that the current abnormality is a short-term interference, and the temporary lane line model is updated; If abnormalities occur continuously for multiple times, use the temporary lane line model as the new lane line model.
2. The multi-feature fusion adaptive filtering lane detection method according to claim 1, characterized in that Use the Sobel operator to calculate the gradient intensity and direction of each pixel point in the image, generate an edge image based on the gradient intensity and direction, and extract lane line features.
3. A multi-feature fusion adaptive filtering lane detection method according to claim 1, characterized in that Convert the image to the HSV color space, extract the pixel region of the lane line color, and use the adaptive threshold segmentation method to extract the lane line color features.
4. A multi-feature fusion adaptive filtering lane detection method according to claim 1, characterized in that Use the Gabor filter to extract the texture information in the image, calculate the texture direction histogram of the local region of the image, and screen out the pixels with lane line direction features.
5. A multi-feature fusion adaptive filtering lane line detection method according to claim 1, characterized in that, Use the weighted fusion method to fuse the edge, color, and texture features, denoted as: F(x,y) = w e ·E(x,y) + w C ·C(x,y) + w t ·T(x,y) Where: F(x, y) is the fused eigenvalue, and E(x, y), C(x, y), and T(x, y) are the edge eigenvalue, color eigenvalue, and texture eigenvalue respectively, and w e , w c , w t are the weights of the edge feature, color feature, and texture feature respectively.
6. A multi - feature fusion adaptive filtering lane detection method according to claim 1, characterized in that The region matching rate is the proportion of the overlapping region area, denoted as: Among them, S overlap is the area of the overlapping region, and S model is the area of the lane line model region in the previous frame.
7. A multi-feature fusion adaptive filtering lane detection method according to claim 1, wherein The image preprocessing uses Gaussian filtering.
8. A multi-feature fusion adaptive filtering lane detection method according to claim 4, characterized in that Calculation of the texture direction histogram: For each pixel point in the image, apply Gabor filters with different directions, calculate the response intensity, count the response intensity in each direction, and generate the texture direction histogram.
9. A method for measuring the distance to the vehicle in front of a lane line with multi - feature fusion adaptive filtering, characterized in that, It includes the following steps: Step 1, parameter definition: Set the vertical height of the reference target relative to the horizontal road surface as h2; Set the vertical height of the vision acquisition system relative to the horizontal road surface as h1; Set the distance between the vision acquisition system and the reference target as L stand ; Step 2: Obtain the lane line image, perform lane line detection based on the above multi-feature fusion adaptive filtering lane line detection method to obtain the lane line model; Step 3: Extend the detected lane line model along the vehicle driving direction to the position of the target vehicle; Step 4, calculate the vertical distance L' from the vision acquisition system to the lane line according to the imaging width n of the lane line in the image road and the focal length f of the vision acquisition system stand , the actual distance L of the reference target stand , the straight-line distance L from the vision acquisition system to the lane line of the target vehicle target , the horizontal distance d between the host vehicle and the target vehicle target .
10. A method for measuring the distance to the vehicle in front of a lane line with multi-feature fusion adaptive filtering according to claim 9, characterized in that, The horizontal distance d between the host vehicle and the target vehicle target is expressed as: