Lane line detection method and device based on weather adaptive feature enhancement

Abstract

The application discloses a lane line detection method based on weather adaptive feature enhancement, and belongs to the technical field of image processing. The method comprises the following steps: acquiring an original image, pre-enhancing the original image to obtain a pre-enhanced image; performing feature extraction on the pre-enhanced image to obtain a multi-scale feature map; inputting the multi-scale feature map into an adaptive feature enhancement module to obtain an enhanced feature map, wherein the adaptive feature enhancement module comprises a weather condition estimation subnetwork and an image enhancement subnetwork; inputting the enhanced feature map into a lane line segmentation module to obtain a probability map of lane line pixels; performing post-processing on the probability map of lane line pixels to obtain an initial estimation of the lane line; performing geometric parameter estimation on the initial estimation of the lane line to obtain a parameter estimation result of the lane line; and obtaining a lane line detection result based on the parameter estimation result of the lane line. The method improves the safety and reliability of automatic driving.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, and in particular relates to a lane line detection method and apparatus based on weather-adaptive feature enhancement. Background Technology

[0002] Lane detection is a key technology for autonomous driving and advanced driver assistance systems (ADAS), providing vehicles with road structure information. With the rapid development of deep learning technology, deep neural networks have made significant progress in tasks such as environmental perception and autonomous driving decision-making and planning. However, complex weather conditions such as rain, fog, and snow severely interfere with lane detection, target tracking, and road scene understanding. In the field of autonomous driving, rainwater adheres to camera lenses, creating reflective artifacts; fog and haze cause a sharp drop in image contrast; and snow covers road markings. These degradation phenomena distort lane line geometry, reduce the perception accuracy of visual sensors, and threaten the safety of vehicle trajectory control. Therefore, improving the accuracy of lane detection under complex weather conditions has become a core challenge in ensuring the safety of autonomous driving and the effectiveness of monitoring systems.

[0003] Existing lane detection methods primarily rely on Hough transform and Canny edge detection. However, under complex weather conditions (rain, fog, snow, etc.), their robustness is poor due to changes in lighting and image blurring. Methods based on semantic segmentation networks like U-Net achieve pixel-level lane prediction through end-to-end training, but still face key bottlenecks in extreme weather: image degradation leading to local detail loss (e.g., rain erosion obscuring edges) and global context confusion (e.g., fog weakening road boundaries) reduce the network's feature extraction capabilities. Although self-supervised models like DINOv2 improve feature generalization through large-scale pre-training, their attention mechanisms are prone to misfocusing on noise in rain and fog regions, and they lack physical prior embeddings such as lane curvature continuity, resulting in distorted structural reconstruction. Traditional image enhancement methods (e.g., histogram equalization) lack adaptability and struggle to effectively handle different weather conditions. Existing methods suffer from low detection accuracy and poor robustness when detecting lane lines for autonomous driving. Summary of the Invention

[0004] This application aims to address at least one of the technical problems existing in the prior art. To this end, this application proposes a lane detection method and apparatus based on weather-adaptive feature enhancement, which improves the safety and reliability of autonomous driving.

[0005] In a first aspect, this application provides a lane detection method based on weather-adaptive feature enhancement, the method comprising:

[0006] Obtain the original image, perform pre-enhancement on the original image, and obtain the pre-enhanced image;

[0007] The pre-enhanced image is subjected to feature extraction to obtain a multi-scale feature map;

[0008] The multi-scale feature map is input into the adaptive feature enhancement module to obtain the enhanced feature map. The adaptive feature enhancement module includes a weather condition estimation subnetwork and an image enhancement subnetwork.

[0009] The enhanced feature map is input into the lane line segmentation module to obtain the probability map of lane line pixels;

[0010] The probability map of the lane line pixels is post-processed to obtain the initial estimate of the lane line;

[0011] Geometric parameter estimation is performed on the initial estimate of the lane lines to obtain the parameter estimation results of the lane lines;

[0012] Based on the parameter estimation results of the lane lines, the lane line detection results are obtained.

[0013] According to one embodiment of this application, the pre-enhancement of the original image to obtain a pre-enhanced image includes:

[0014] The original image is subjected to illumination correction and detail enhancement using Retinex transform to obtain the transformed image.

[0015] The transformed image is then subjected to contrast enhancement and color balance using Gamma correction to obtain a pre-enhanced image.

[0016] According to one embodiment of this application, the step of inputting the multi-scale feature map into an adaptive feature enhancement module to obtain an enhanced feature map includes:

[0017] The multi-scale feature map is input into the weather condition estimation sub-network to obtain the weather condition of the feature map;

[0018] The multi-scale feature map and weather conditions are input into the image enhancement sub-network to obtain the enhanced feature map.

[0019] According to one embodiment of this application, the lane segmentation module is a U-Net network, which includes an encoder and a decoder. The step of inputting the enhanced feature map into the lane segmentation module to obtain a probability map of lane line pixels includes:

[0020] The enhanced feature map is input into the encoder to obtain the enhanced features;

[0021] The enhanced features are input into the decoder to obtain a probability map of the lane line pixels.

[0022] According to one embodiment of this application, the post-processing of the probability map of the lane line pixels to obtain an initial estimate of the lane line includes:

[0023] The probability map is converted into a binary image by threshold segmentation;

[0024] The binarized image is fitted using the least squares method to obtain an initial estimate of the lane lines.

[0025] According to one embodiment of this application, the step of performing geometric parameter estimation on the initial estimate of the lane line to obtain the parameter estimation result of the lane line includes:

[0026] The initial estimate of the lane line is segmented to obtain multiple line segments;

[0027] Calculate the length of each line segment;

[0028] The degree of deviation of each line segment is determined based on its length and the midpoint of the lane line.

[0029] The length of each line segment and the degree of deviation of each line segment are used as the parameter estimation results of the lane line.

[0030] According to one embodiment of this application, the step of extracting features from the pre-enhanced image to obtain a multi-scale feature map includes:

[0031] The pre-enhanced image is input into the DINOv2 network, and the pre-enhanced image is processed by the FPN algorithm to obtain a multi-scale feature map.

[0032] Secondly, this application provides a lane detection device based on weather-adaptive feature enhancement, the device comprising:

[0033] The acquisition module is used to acquire the original image, perform pre-enhancement on the original image, and obtain the pre-enhanced image;

[0034] The first processing module is used to extract features from the pre-enhanced image to obtain a multi-scale feature map;

[0035] The second processing module is used to input the multi-scale feature map into the adaptive feature enhancement module to obtain the enhanced feature map. The adaptive feature enhancement module includes a weather condition estimation subnetwork and an image enhancement subnetwork.

[0036] The third processing module is used to input the enhanced feature map into the lane line segmentation module to obtain a probability map of lane line pixels.

[0037] The fourth processing module is used to post-process the probability map of the lane line pixels to obtain the initial estimate of the lane line.

[0038] The estimation module is used to perform geometric parameter estimation on the initial estimate of the lane line to obtain the parameter estimation result of the lane line;

[0039] The detection module is used to obtain lane line detection results based on the parameter estimation results of the lane lines.

[0040] Thirdly, this application provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the lane detection method based on weather adaptive feature enhancement as described in the first aspect above.

[0041] Fourthly, this application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the lane detection method based on weather adaptive feature enhancement as described in the first aspect above.

[0042] Fifthly, this application provides a chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the lane detection method based on weather adaptive feature enhancement as described in the first aspect.

[0043] In a sixth aspect, this application provides a computer program product, including a computer program that, when executed by a processor, implements the lane detection method based on weather adaptive feature enhancement as described in the first aspect above.

[0044] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application.

[0045] The lane detection method based on weather-adaptive feature enhancement provided by this invention has the following advantages over existing technologies:

[0046] (1) This invention obtains a pre-enhanced image by processing the original image, which better highlights the lane line features and suppresses noise interference. Multi-scale feature extraction of the pre-enhanced image can more effectively capture the global information and contextual relationship of the lane lines, providing a more powerful feature expression capability and improving adaptability to complex scenes. Further enhancement is achieved through an adaptive feature enhancement module, which improves the accuracy of lane line detection. The probability map of lane line pixels and the initial estimate of the lane lines are obtained through the lane line segmentation module and post-processing. It can stably perform lane line detection under complex weather conditions, such as rain and fog, providing reliable lane line information for autonomous driving systems, enhancing the robustness of autonomous driving, effectively improving the accuracy and real-time performance of lane line detection, enhancing the safety and reliability of autonomous driving systems, and reducing the risk of traffic accidents caused by adverse weather conditions.

[0047] (2) This invention inputs a multi-scale feature map into a weather condition estimation sub-network, processes the feature map in combination with weather condition information to obtain the weather condition of the feature map, and then inputs the multi-scale feature map and the weather condition into an image enhancement sub-network to obtain an enhanced feature map. It can adaptively adjust the feature map weights according to the weather conditions, effectively improving the image quality under complex weather conditions. By dynamically adjusting the feature weights, it can better adapt to complex weather changes, effectively improving the accuracy and robustness of lane line detection and enhancing the adaptability of image enhancement.

[0048] (3) This invention improves the accuracy of lane line segmentation by inputting the enhanced feature map into the encoder of the U-Net network for feature extraction, and then inputting the extracted features into the decoder to obtain the probability map of lane line pixels. It can effectively segment lane lines under complex weather conditions, improving the recognition capability and robustness of the autonomous driving system in adverse weather environments. Through the synergistic effect of the encoder and decoder, it enhances the detail accuracy and stability of lane line detection, providing higher quality output for lane line detection, further improving the safety and reliability of the autonomous driving system, and reducing potential traffic risks. Attached Figure Description

[0049] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0050] Figure 1 This is one of the flowcharts of the lane detection method based on weather adaptive feature enhancement provided in the embodiments of this application;

[0051] Figure 2 This is the second flowchart of the lane detection method based on weather adaptive feature enhancement provided in the embodiments of this application;

[0052] Figure 3 This is a schematic diagram of the structure of the lane detection device based on weather adaptive feature enhancement provided in the embodiments of this application;

[0053] Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0054] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0055] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0056] The lane detection method, device, electronic device, and readable storage medium based on weather adaptive feature enhancement provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0057] Among them, the lane detection method based on weather adaptive feature enhancement can be applied to the terminal, specifically executed by the hardware or software in the terminal.

[0058] The terminal includes, but is not limited to, portable communication devices such as mobile phones or tablets with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads). It should also be understood that, in some embodiments, the terminal may not be a portable communication device, but rather a desktop computer with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads).

[0059] The following embodiments describe a terminal including a display and a touch-sensitive surface. However, it should be understood that the terminal may include one or more other physical user interface devices such as a physical keyboard, mouse, and joystick.

[0060] The lane detection method based on weather adaptive feature enhancement provided in this application embodiment can be executed by an electronic device or a functional module or entity in an electronic device that can implement the lane detection method based on weather adaptive feature enhancement. The electronic devices mentioned in this application embodiment include, but are not limited to, mobile phones, tablets, computers, cameras and wearable devices. The lane detection method based on weather adaptive feature enhancement provided in this application embodiment will be described below using an electronic device as the execution subject.

[0061] Figure 1 This is one of the flowcharts illustrating the lane detection method based on weather-adaptive feature enhancement provided in this application embodiment, such as... Figure 1 As shown, the lane detection method based on weather adaptive feature enhancement includes steps 110, 120, 130, 140, 150, 160 and 170.

[0062] Step 110: Obtain the original image, perform pre-enhancement on the original image, and obtain the pre-enhanced image;

[0063] The process involves acquiring the original image from the acquisition device, which includes lane lines. The original image is then enhanced using a combination of Retinex transform and Gamma correction. Retinex transform is used for illumination correction and detail enhancement, while Gamma correction is used to improve contrast and color balance, resulting in a pre-enhanced image.

[0064] Step 120: Extract features from the pre-enhanced image to obtain a multi-scale feature map;

[0065] In some embodiments, the step of extracting features from the pre-enhanced image to obtain a multi-scale feature map includes:

[0066] The pre-enhanced image is input into the DINOv2 network, and the pre-enhanced image is processed by the FPN algorithm to obtain a multi-scale feature map.

[0067] The pre-enhanced image is input into the DINOv2 network. Using the DINOv2 network as the backbone, the FPN algorithm is combined to extract features and downsample the pre-enhanced image, generating a multi-scale feature map using the following formula:

[0068] F i =FPN(DINOv2(I′))

[0069] Among them, F i I' represents a multi-scale feature map, i' represents the pre-enhanced image, and i represents the level of the feature map. Different values ​​of i correspond to different feature map resolutions.

[0070] In this embodiment, by inputting the pre-enhanced image into the DINOv2 network and processing the image using the FPN algorithm, a multi-scale feature map is obtained, effectively improving the accuracy of lane line detection. Processing the pre-enhanced image under complex weather conditions allows for the full extraction of detailed information and feature representations within the image, thus improving lane line detection performance.

[0071] Step 130: Input the multi-scale feature map into the adaptive feature enhancement module to obtain the enhanced feature map. The adaptive feature enhancement module includes a weather condition estimation subnetwork and an image enhancement subnetwork.

[0072] In some embodiments, inputting the multi-scale feature map into the adaptive feature enhancement module to obtain the enhanced feature map includes:

[0073] The multi-scale feature map is input into the weather condition estimation sub-network to obtain the weather condition of the feature map;

[0074] The multi-scale feature map and weather conditions are input into the image enhancement sub-network to obtain the enhanced feature map.

[0075] It is easy to understand that the multi-scale feature map F i The input is fed into AFEM (Adaptive Feature Enhancement Module) to enhance the expressive power of the features and improve their adaptability to complex weather conditions.

[0076] The AFEM module includes a weather condition estimation subnetwork N. w (F i ) and image enhancement subnetwork N e (F i ,w i Weather condition estimation subnetwork N w (F i ) used to estimate feature map F i Weather conditions w i Such as rain, snow, fog, etc., and the weather conditions w i As an image enhancement subnetwork N e The input, the image enhancement subnetwork N e (F i ,w i ) Received feature map F i and weather conditions w i The weights of the feature map are dynamically adjusted according to weather conditions to adaptively enhance features, suppress noise, and generate an enhanced feature map. The calculation formula is as follows:

[0077] F′ i =Ne (F i ,w i )

[0078] Among them, F′ i F represents the enhanced feature map. i Represents a multi-scale feature map, w i Indicates weather conditions.

[0079] In this embodiment, by inputting a multi-scale feature map into a weather condition estimation subnetwork and processing the feature map in conjunction with weather condition information, the weather condition of the feature map is obtained. Then, the multi-scale feature map and the weather condition are input into an image enhancement subnetwork to obtain an enhanced feature map. The feature map weights can be adaptively adjusted according to the weather conditions, effectively improving the image quality under complex weather conditions. By dynamically adjusting the feature weights, it can better adapt to complex weather changes, effectively improving the accuracy and robustness of lane detection and enhancing the adaptability of image enhancement.

[0080] Step 140: Input the enhanced feature map into the lane line segmentation module to obtain the probability map of lane line pixels;

[0081] In some embodiments, the lane line segmentation module is a U-Net network, which includes an encoder and a decoder. The step of inputting the enhanced feature map into the lane line segmentation module to obtain a probability map of lane line pixels includes:

[0082] The enhanced feature map is input into the encoder to obtain the enhanced features;

[0083] The enhanced features are input into the decoder to obtain a probability map of the lane line pixels.

[0084] The enhanced feature map F′ i The input is fed into the lane segmentation module to predict the probability map of lane line pixels. The lane segmentation module uses a U-Net network, a commonly used image segmentation network with good segmentation performance and robustness. The U-Net network uses an encoder-decoder structure to progressively extract image features and ultimately predict the probability that each pixel belongs to a lane line. The calculation formula is shown below:

[0085] P lane =U-Net(F′) i )

[0086] Among them, P lane This is a probability map of lane line pixels. Each pixel value represents the confidence level that the pixel belongs to the lane line, and the value ranges from 0 to 1.

[0087] In this embodiment, the enhanced feature map is input into the encoder of the U-Net network for feature extraction, and then the extracted features are input into the decoder to obtain a probability map of lane line pixels, thereby improving the accuracy of lane line segmentation. This effectively segments lane lines under complex weather conditions, improving the recognition capability and robustness of the autonomous driving system in adverse weather environments. Through the collaborative action of the encoder and decoder, the detail accuracy and stability of lane line detection are enhanced, providing higher quality output for lane line detection, further improving the safety and reliability of the autonomous driving system, and reducing potential traffic risks.

[0088] Step 150: Post-process the probability map of the lane line pixels to obtain the initial estimate of the lane line;

[0089] The initial estimate of the lane lines is obtained by post-processing the probability map of lane line pixels using threshold segmentation and least squares method.

[0090] Step 160: Perform geometric parameter estimation on the initial estimate of the lane line to obtain the parameter estimation result of the lane line;

[0091] Furthermore, geometric parameter estimation is performed on the initial estimate of the lane lines to obtain key information about the lane lines and obtain the parameter estimation results of the lane lines. The parameters of the lane lines include the curvature and deviation of the lane lines.

[0092] Step 170: Based on the parameter estimation results of the lane lines, obtain the lane line detection results.

[0093] Finally, based on the parameter estimation results of the lane lines, it is determined whether the parameters of the lane lines exceed the preset threshold, and the lane line detection results are obtained, providing a decision-making basis for the vehicle's automatic driving.

[0094] The lane detection method based on weather-adaptive feature enhancement provided in this application's embodiments obtains a pre-enhanced image by processing the original image, which better highlights lane line features and suppresses noise interference. Multi-scale feature extraction of the pre-enhanced image more effectively captures the global information and contextual relationships of lane lines, providing stronger feature representation capabilities and improving adaptability to complex scenes. Further enhancement through an adaptive feature enhancement module improves the accuracy of lane line detection. Through a lane line segmentation module and post-processing, a probability map of lane line pixels and an initial estimate of the lane lines are obtained. This method can stably perform lane line detection under complex weather conditions, such as rain and fog, providing reliable lane line information for autonomous driving systems, enhancing the robustness of autonomous driving, effectively improving the accuracy and real-time performance of lane line detection, enhancing the safety and reliability of autonomous driving systems, and reducing the risk of traffic accidents caused by adverse weather conditions.

[0095] In some embodiments, the pre-enhancement of the original image to obtain a pre-enhanced image includes:

[0096] The original image is subjected to illumination correction and detail enhancement using Retinex transform to obtain the transformed image.

[0097] The transformed image is then subjected to contrast enhancement and color balance using Gamma correction to obtain a pre-enhanced image.

[0098] For example, the Retinex transform performs illumination correction and detail enhancement on the original image using R(I). By simulating the characteristics of human vision, it enhances the dynamic range of the image, reduces the impact of uneven illumination, and highlights image details. Gamma correction G(·,γ) is used to improve contrast and color balance. By adjusting the grayscale distribution of the image, it makes the image clearer and the colors more vibrant, where γ is the Gamma value, which is adjusted according to the actual situation. Combining the Retinex transform and Gamma correction yields the pre-enhanced image, calculated as follows:

[0099] I′=G(R(I),γ)

[0100] Where I′ is the pre-enhanced image and R(I) is the transformed image.

[0101] In this embodiment, by performing Retinex transformation on the original image, illumination correction and detail enhancement are achieved, resulting in a clearer transformed image. Gamma correction is then used to enhance the contrast and balance the color of the transformed image, resulting in a pre-enhanced image. This effectively improves the problems of uneven illumination and insufficient contrast in the image, enhances image details, improves the visibility and resolvability of the image, improves the quality of the original image, and enhances the accuracy and robustness of subsequent lane line detection.

[0102] In some embodiments, the post-processing of the probability map of the lane line pixels to obtain an initial estimate of the lane line includes:

[0103] The probability map is converted into a binary image by threshold segmentation;

[0104] The binarized image is fitted using the least squares method to obtain an initial estimate of the lane lines.

[0105] First, threshold segmentation is used to divide the probability map P lane Convert to binary image B lane Pixels with probability values ​​greater than a threshold are set to 1, and pixels with probability values ​​less than the threshold are set to 0, thereby separating lane line pixels from background pixels. The calculation formula is as follows:

[0106]

[0107] Among them, B lane (x,y) represents the pixel value at coordinates (x,y) in the binarized image, taking values ​​of 0 or 1. lane (x,y) is the probability value of the probability graph at coordinates (x,y), and τ is the set threshold.

[0108] Then, the least squares method is used to fit the quadratic polynomial curve to the binarized image B. lane Lane line pixel set (where N is the number of pixels), the initial estimate of the lane line is obtained, and the calculation formula for the fitted quadratic polynomial is shown below:

[0109] y = ax 2 +bx+c

[0110] Where y is the ordinate of a point on the fitted curve, x is the abscissa of a point on the fitted curve, and a, b, and c are the coefficients of the quadratic polynomial, which are calculated by the least squares method and used to determine the shape of the curve.

[0111] Considering the potential presence of noise in the binarized image, such as small isolated pixels, connected component analysis is used to remove BL. lane This reduces the noise area in the lane detection, thereby improving the robustness of lane detection.

[0112] In this embodiment, the probability map of lane line pixels is thresholded and converted into a binary image, which optimizes the segmentation results and improves the accuracy of lane line detection. The binary image is fitted using the least squares method to obtain an initial estimate of the lane line, which can better fit the curvature of the lane line. This effectively addresses the lane line detection problem under complex road conditions, improves the adaptability of autonomous driving in real road environments, enhances the safety of autonomous driving vehicles, and reduces the risk of traffic accidents that may be caused by lane line recognition errors.

[0113] In some embodiments, the initial estimation of the lane line is subjected to geometric parameter estimation to obtain the parameter estimation result of the lane line, including:

[0114] The initial estimate of the lane line is segmented to obtain multiple line segments;

[0115] Calculate the length of each line segment;

[0116] The degree of deviation of each line segment is determined based on its length and the midpoint of the lane line.

[0117] The length of each line segment and the degree of deviation of each line segment are used as the parameter estimation results of the lane line.

[0118] For example, based on the initial estimate of the lane line segmentation using linear interpolation, multiple line segments are obtained. The calculation formula for linear interpolation is shown below:

[0119]

[0120] Among them, y k Let y be the ordinate of the dividing point between the k-th and (k+1)-th line segments. min The minimum value of the lane line's ordinate, y max is the maximum value of the lane line's ordinate, and k is a positive integer greater than 0.

[0121] The lane line pixel set is segmented and fitted into K line segments, where K is the number of line segments. Each segment starts from the line segment origin (x). k,start ,y k,start ), and the endpoint (x) k,end ,y k,end Define k = 1, 2, ..., K. The length L of the line segment is... k The Euclidean distance calculation formula is shown below:

[0122]

[0123] Degree of deviation Δ k Used to measure the lane lines and the expected vehicle path (image center line x = x) c , where x c The offset (the x-coordinate of the image center line) is first determined by taking the midpoint of the line segment. Then use the formula Δ k =|x k,mid -x c Calculate the degree of deviation.

[0124] These parameters provide a basis for vehicle operation, L k Reflects the local length of a lane segment, Δ k If the deviation is too large, it indicates that the vehicle has deviated from the center and needs to be adjusted.

[0125] For example, by approximating the lane line shape through piecewise linear fitting, the lane line is divided into multiple segments. The geometric characteristics of these segments, such as length, direction angle, and position, are analyzed to estimate key parameters such as the curvature and deviation of the lane line. The curvature of the lane line reflects the degree of bending of the lane line, and the deviation reflects the amount of offset between the vehicle and the center of the lane line. These parameters can provide a basis for vehicle driving decisions, thereby achieving safe and comfortable autonomous driving.

[0126] Figure 2 This is the second flowchart illustrating the lane detection method based on weather-adaptive feature enhancement provided in this application embodiment, as shown below. Figure 2As shown, the original image is first pre-enhanced using a combination of Retinex transform and Gamma correction to improve illumination and contrast. Then, multi-scale features are extracted using the DINOv2 network and FPN algorithm, and input into an adaptive feature enhancement module. The feature weights are dynamically adjusted based on weather conditions to generate an enhanced feature map. Next, the U-Net network is used to segment lane lines on the enhanced feature map, obtaining a probability map of lane line pixels. Finally, post-processing and geometric parameter estimation are performed on the probability map to obtain key parameters such as lane line curvature and deviation, providing a basis for vehicle driving decisions.

[0127] In this embodiment, by segmenting the initial lane line estimation into multiple segments and calculating the length of each segment, the accuracy of lane line parameter estimation is effectively improved. Based on the length of each segment and the midpoint of the lane line, the deviation of each segment is obtained. Using the length and deviation of each segment as the parameter estimation result of the lane line can effectively solve the deviation problem in lane line detection, enhance the adaptability and stability of autonomous driving to lane lines, further improve the safety of vehicle driving, and provide more comprehensive road structure information for the autonomous driving system to assist vehicle decision-making.

[0128] The lane detection method based on weather-adaptive feature enhancement provided in this application can be executed by a lane detection device based on weather-adaptive feature enhancement. This application uses the example of a lane detection device based on weather-adaptive feature enhancement executing the lane detection method to illustrate the lane detection device based on weather-adaptive feature enhancement provided in this application.

[0129] This application also provides a lane detection device based on weather-adaptive feature enhancement, such as... Figure 3 As shown, the lane detection device based on weather adaptive feature enhancement includes: an acquisition module 310, a first processing module 320, a second processing module 330, a third processing module 340, a fourth processing module 350, an estimation module 360, and a detection module 370.

[0130] The acquisition module 310 is used to acquire the original image, perform pre-enhancement on the original image, and obtain a pre-enhanced image;

[0131] The first processing module 320 is used to extract features from the pre-enhanced image to obtain a multi-scale feature map.

[0132] The second processing module 330 is used to input the multi-scale feature map into the adaptive feature enhancement module to obtain the enhanced feature map. The adaptive feature enhancement module includes a weather condition estimation subnetwork and an image enhancement subnetwork.

[0133] The third processing module 340 is used to input the enhanced feature map into the lane line segmentation module to obtain a probability map of lane line pixels.

[0134] The fourth processing module 350 is used to post-process the probability map of the lane line pixels to obtain an initial estimate of the lane line; the estimation module 360 ​​is used to perform geometric parameter estimation on the initial estimate of the lane line to obtain the parameter estimation result of the lane line.

[0135] The detection module 370 is used to obtain lane line detection results based on the parameter estimation results of the lane lines.

[0136] The lane detection method based on weather-adaptive feature enhancement provided in this application's embodiments obtains a pre-enhanced image by processing the original image, which better highlights lane line features and suppresses noise interference. Multi-scale feature extraction of the pre-enhanced image more effectively captures the global information and contextual relationships of lane lines, providing stronger feature representation capabilities and improving adaptability to complex scenes. Further enhancement through an adaptive feature enhancement module improves the accuracy of lane line detection. Through a lane line segmentation module and post-processing, a probability map of lane line pixels and an initial estimate of the lane lines are obtained. This method can stably perform lane line detection under complex weather conditions, such as rain and fog, providing reliable lane line information for autonomous driving systems, enhancing the robustness of autonomous driving, effectively improving the accuracy and real-time performance of lane line detection, enhancing the safety and reliability of autonomous driving systems, and reducing the risk of traffic accidents caused by adverse weather conditions.

[0137] The lane detection device based on weather-adaptive feature enhancement provided in this application embodiment can achieve... Figures 1 to 2 The various processes implemented in the embodiment of the lane line detection method based on weather adaptive feature enhancement will not be described again here to avoid repetition.

[0138] In some embodiments, such as Figure 4 As shown, this application embodiment also provides an electronic device 400, including a processor 401, a memory 402, and a computer program stored in the memory 402 and executable on the processor 401. When the program is executed by the processor 401, it implements the various processes of the above-described lane line detection method embodiment based on weather adaptive feature enhancement and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0139] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0140] This application also provides a non-transitory computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the above-described lane detection method based on weather adaptive feature enhancement and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0141] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0142] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described lane detection method based on weather adaptive feature enhancement.

[0143] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0144] This application also provides a chip, which includes a processor and a communication interface. The communication interface and the processor are coupled. The processor is used to run programs or instructions to implement the various processes of the above-described lane line detection method based on weather adaptive feature enhancement, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0145] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a device-level chip, device chip, chip device, or on-chip device chip, etc.

[0146] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0147] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the lane detection method based on weather adaptive feature enhancement of the various embodiments of this application.

[0148] In the description of this application, "first feature" and "second feature" may include one or more of the features.

[0149] In the description of this application, "multiple" means two or more.

[0150] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

[0151] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0152] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

Claims

1. A lane line detection method based on weather adaptive feature enhancement, characterized in that, The method comprises: obtaining an original image, pre-enhancing the original image to obtain a pre-enhanced image; performing feature extraction on the pre-enhanced image to obtain a multi-scale feature map; input the multi-scale feature map into an adaptive feature enhancement module to obtain an enhanced feature map, the adaptive feature enhancement module comprising a weather condition estimation subnetwork and an image enhancement subnetwork; wherein the weather condition estimation subnetwork is used to estimate the weather condition of the feature map , and take the weather condition as the input of the image enhancement subnetwork ; the image enhancement subnetwork is used to receive the feature map and the weather condition , and dynamically adjust the weight of the feature map according to the weather condition; the calculation formula is as follows: ； wherein, denotes the enhanced feature map, denotes the multi-scale feature map, denotes the weather condition; inputting the enhanced feature map into a lane line segmentation module to obtain a probability map of lane line pixels, specifically comprising: inputting the enhanced feature map into an encoder to obtain an enhanced feature; inputting the enhanced feature into a decoder to obtain the probability map of lane line pixels; the lane line segmentation module is a U-Net network, the U-Net network comprises an encoder and a decoder, the U-Net network gradually extracts features of an image through an encoder-decoder structure, and predicts the probability that each pixel belongs to a lane line, and the calculation formula is as follows: ； wherein, is a probability map of lane line pixels, each pixel value represents the confidence that the pixel belongs to a lane line, and the value range is 0 to 1; Post-processing the probability map of the lane line pixels to obtain an initial estimate of the lane lines specifically includes: segmenting the probability map using a threshold. Convert to binary image Pixels with a probability value greater than the threshold are set to 1, and pixels with a probability value less than the threshold are set to 0, thereby separating the lane line pixels from the background pixels. The calculation formula is as follows: ； wherein, represents a pixel value of the binarized image at coordinates , and takes a value of 0 or 1, is a probability value of the probability map at coordinates , is a set threshold value; based on the least square method, the binarized image is fitted, thereby fitting a quadratic polynomial curve to a set of lane line pixel points in the binarized image , and obtaining an initial estimation of the lane line, and a calculation formula of the fitted quadratic polynomial is as shown below: ； wherein is the ordinate of a point on the fitted curve, is the abscissa of a point on the fitted curve, , , is the coefficient of the quadratic polynomial; performing geometric parameter estimation on the initial estimation of the lane line to obtain a parameter estimation result of the lane line, specifically comprising: segmenting the initial estimation of the lane line based on a linear interpolation method to obtain a plurality of line segments, and the calculation formula of the linear interpolation method is as follows: ； wherein, is the longitudinal coordinate of the split point of the kth segment and the k+1th segment, is the minimum value of the longitudinal coordinate of the lane line, is the maximum value of the longitudinal coordinate of the lane line, k is a positive integer greater than 0; the length of each segment is calculated; the deviation degree of each segment is obtained based on the length of each segment and the midpoint of the lane line; the deviation degree for measuring the lane line and the expected path of the vehicle, and the calculation formula of the deviation degree is as follows: ； wherein, denotes the midpoint of the line segment, denotes the abscissa of the image center line; the length of each line segment and the deviation degree of each line segment are taken as the parameter estimation results of the lane line; obtaining a lane line detection result based on the parameter estimation result of the lane line.

2. The weather-adaptive feature-enhanced lane line detection method according to claim 1, wherein, The pre-enhancing of the original image to obtain the pre-enhanced image comprises: performing illumination correction and detail enhancement on the original image through Retinex transformation to obtain a transformed image; performing contrast enhancement and color balance on the transformed image through Gamma correction to obtain a pre-enhanced image.

3. The weather-adaptive feature-enhanced lane line detection method of claim 1, wherein, The inputting of the multi-scale feature map into an adaptive feature enhancement module to obtain an enhanced feature map comprises: inputting the multi-scale feature map into a weather condition estimation subnetwork to obtain a weather condition of the feature map; inputting the multi-scale feature map and the weather condition into an image enhancement subnetwork to obtain an enhanced feature map.

4. The weather-adaptive feature-enhanced lane line detection method of claim 1, wherein, The feature extraction on the pre-enhanced image to obtain a multi-scale feature map comprises: inputting the pre-enhanced image into a DINOv2 network, processing the pre-enhanced image through an FPN algorithm to obtain a multi-scale feature map.

5. A lane line detection device based on weather adaptive feature enhancement, implemented by the lane line detection method based on weather adaptive feature enhancement according to any one of claims 1 to 4, characterized in that, The device comprises: an acquisition module configured to obtain an original image, pre-enhance the original image to obtain a pre-enhanced image; a first processing module configured to perform feature extraction on the pre-enhanced image to obtain a multi-scale feature map; The second processing module is used to input the multi-scale feature map into the adaptive feature enhancement module to obtain the enhanced feature map. The adaptive feature enhancement module includes a weather condition estimation subnetwork and an image enhancement subnetwork; wherein, the weather condition estimation subnetwork... Used to estimate the feature map Weather conditions and the weather conditions As the image enhancement subnetwork The input; the image enhancement subnetwork Used to receive the feature map and the weather conditions The weights of the feature maps are dynamically adjusted based on the weather conditions; the calculation formula is as follows: ； wherein, denotes the enhanced feature map, denotes the multi-scale feature map, denotes the weather condition; a third processing module configured to input the enhanced feature map into a lane line segmentation module to obtain a probability map of lane line pixels, specifically comprising: inputting the enhanced feature map into an encoder to obtain an enhanced feature; inputting the enhanced feature into a decoder to obtain the probability map of lane line pixels; the lane line segmentation module is a U-Net network, the U-Net network comprises an encoder and a decoder, the U-Net network gradually extracts features of an image through an encoder-decoder structure, and predicts the probability that each pixel belongs to a lane line, and the calculation formula is as follows: ； wherein, is a probability map of lane line pixels, each pixel value represents the confidence that the pixel belongs to a lane line, and the value range is 0 to 1; The fourth processing module is used to post-process the probability map of the lane line pixels to obtain an initial estimate of the lane lines. Specifically, this includes: segmenting the probability map using a threshold. Convert to binary image Pixels with a probability value greater than the threshold are set to 1, and pixels with a probability value less than the threshold are set to 0, thereby separating the lane line pixels from the background pixels. The calculation formula is as follows: ； wherein, represents a pixel value of the binarized image at coordinates , and takes a value of 0 or 1, is a probability value of the probability map at coordinates , is a set threshold value; based on the least square method, the binarized image is fitted, thereby fitting a quadratic polynomial curve to a set of lane line pixel points in the binarized image , and obtaining an initial estimation of the lane line, and a calculation formula of the fitted quadratic polynomial is as shown below: ； wherein is the ordinate of a point on the fitted curve, is the abscissa of a point on the fitted curve, , , is the coefficient of the quadratic polynomial; an estimation module configured to perform geometric parameter estimation on the initial estimation of the lane line to obtain a parameter estimation result of the lane line, specifically comprising: segmenting the initial estimation of the lane line based on a linear interpolation method to obtain a plurality of line segments, and the calculation formula of the linear interpolation method is as follows: ； wherein, is the longitudinal coordinate of the split point of the kth segment and the k+1th segment, is the minimum value of the longitudinal coordinate of the lane line, is the maximum value of the longitudinal coordinate of the lane line, k is a positive integer greater than 0; the length of each segment is calculated; the deviation degree of each segment is obtained based on the length of each segment and the midpoint of the lane line; the deviation degree for measuring the lane line and the expected path of the vehicle, and the calculation formula of the deviation degree is as follows: ； wherein, denotes the midpoint of the line segment, denotes the abscissa of the center line of the image; the length of each line segment and the deviation degree of each line segment are taken as the parameter estimation result of the lane line; A detection module is configured to obtain a lane line detection result based on the parameter estimation result of the lane line.

6. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the lane line detection method based on weather-adaptive feature enhancement according to any one of claims 1 to 4 when executing the program.

7. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program implements the lane line detection method based on weather-adaptive feature enhancement according to any one of claims 1 to 4 when executed by the processor.

Images