Binary lane detection method, device, equipment and storage medium

Abstract

The present invention relates to the field of data processing technology, and discloses a binary classification lane line detection method, device, equipment and storage medium. The method comprises the following steps: performing lane line annotation on a plurality of acquired driving road sample images, performing image preprocessing according to different image adjustment parameters, completing image amplification and enhancement, constructing a plurality of lane line detection samples, respectively training the constructed binary segmentation network, obtaining a plurality of candidate lane line detection models suitable for different environmental scenes, determining a lane line detection model temporary library by using detection association information, then performing lane line detection on the driving road detection image for a plurality of times, and finally updating the lane line detection model temporary library according to the lane line detection result, and considering the influence of the lane line detection result of the previous lane line detection cycle on the driving of the target vehicle, thereby realizing lane line detection with better adaptability and balancing the detection performance and the detection real-time performance, and improving the detection performance and robustness of the lane line detection.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a binary lane line detection method, device, equipment and storage medium. Background Art

[0002] Lane detection technology is an important task in assisted driving systems. Tabelini L et al. used polynomial regression to detect lanes. Although the method is very efficient, it lacks global features and is not accurate enough. LaneATT designed an anchor-based lane detection network and achieved outstanding results on multiple data sets, but the designed anchor shape resulted in poor freedom in describing the line shape. For different driving scenarios, Fabio Pizzati and other researchers designed a lane detection algorithm based on the ROS framework. The algorithm used mean square error uncertainty estimation in the training process of multi-task CNN to achieve better performance. In recent years, with the development of deep learning semantic segmentation networks, many researchers have applied it to lane detection. Xingang Pan et al. proposed a spatial convolution method to detect lanes, but its relatively complex structure made it impossible to keep up with the segmentation speed in practical applications.

[0003] At present, traditional lane line detection methods have poor performance and low robustness in complex scenarios. More importantly, when facing different scenarios and environments (for example, the image parameters of lane lines are different under different environmental factors), using the same lane line detection model for lane line detection has differences in detection accuracy (if the model trained with global sample images or fixed sample images is used for detection, there is a problem of low detection accuracy). However, referring to the detection results of multiple models for comprehensive detection output has certain limitations in detection real-time performance. A good balance between detection performance and detection speed is an important difficulty in lane line detection in assisted driving.

[0004] Therefore, how to improve the detection performance and robustness of lane line detection and balance the real-time detection and detection accuracy is a technical problem that needs to be solved urgently. Summary of the invention

[0005] The present invention provides a binary lane line detection method, device, equipment and storage medium, aiming to solve at least one of the above-mentioned technical problems.

[0006] To achieve the above object, the present invention provides a binary classification lane line detection method, comprising:

[0007] Acquire a number of driving road sample images, perform lane line marking and image preprocessing on the number of driving road sample images, and construct a number of lane line detection samples;

[0008] Constructing a binary segmentation network, training the binary segmentation network respectively using each lane line detection sample that performs different image preprocessing to obtain a plurality of candidate lane line detection models, and constructing a lane line detection model library storing the plurality of candidate lane line detection models;

[0009] In the first lane line detection cycle when the target vehicle performs a driving task, a driving road detection image and detection association information collected by the target vehicle are obtained, several target lane line detection models are extracted from a lane line detection model library using the detection association information, a lane line detection model temporary library is constructed, and several target lane line detection models stored in the lane line detection model temporary library are used to perform lane line detection on the driving road detection image to obtain a lane line detection result;

[0010] According to the lane line detection result of each lane line detection cycle, the lane line detection model temporary library is updated so that the target vehicle performs lane line detection based on the updated lane line detection model temporary library when performing the next lane line detection cycle of the driving task.

[0011] Optionally, a plurality of sample images of driving roads are obtained, lane line marking and image preprocessing are performed on the plurality of sample images of driving roads, and a plurality of lane line detection sample steps are constructed, specifically including:

[0012] Acquire a plurality of sample images of driving roads; wherein the sample images of driving roads include actual images of driving roads collected by historical vehicles under standard environments and generated images of driving roads under standard environments output by using a DreamBooth image generation model based on a PPdiffusers deep learning library;

[0013] The lane lines in the plurality of driving road sample images are subjected to pixel-level point annotation processing to obtain driving road sample images with lane line annotations, and data preprocessing is performed on the driving road sample images with lane line annotations to construct a plurality of lane line detection samples.

[0014] Optionally, data preprocessing is performed on the sample image of the driving road with lane line markings to construct a plurality of lane line detection sample steps, specifically including:

[0015] Step-by-step adjustment of the parameter value of each type of image parameter is performed on the sample image of the driving road with lane line markings to obtain a plurality of adjusted images of each sample image of the driving road, and each adjusted image is associated with a corresponding set of image parameter adjustment values ​​and stored;

[0016] Each adjusted image is flipped and cropped, and a group of adjusted images with exactly the same set of image parameter adjustment values ​​after image flipping and cropping are used as a lane line detection sample to construct and obtain several lane line detection samples.

[0017] Optionally, the steps of constructing a binary segmentation network, training the binary segmentation network using each lane line detection sample that performs different image preprocessing, obtaining a plurality of candidate lane line detection models, and constructing a lane line detection model library storing the plurality of candidate lane line detection models specifically include:

[0018] Constructing a binary segmentation network; wherein the binary segmentation network is configured as a network architecture consisting of an encoder and a decoder, the encoder uses a Resnet18 backbone network to extract multi-scale feature information and encode it, the decoder uses an upsampling and feature fusion module to convert the low-dimensional representation back to the original data space, and the encoder and the decoder are connected using a CBAM model consisting of a CAM and a PAM in parallel to extract deep features for image segmentation;

[0019] Each lane line detection sample that performs different image preprocessing is input into the binary segmentation network for training to obtain a number of candidate lane line detection models, and a lane line detection model library storing the several candidate lane line detection models is constructed.

[0020] Optionally, in the first lane line detection cycle when the target vehicle performs a driving task, a driving road detection image and detection association information collected by the target vehicle are obtained, and a plurality of target lane line detection models are extracted from a lane line detection model library using the detection association information to construct a temporary lane line detection model library step, specifically including:

[0021] In the first lane line detection cycle of the target vehicle performing the driving task, a driving road detection image and detection-related information collected by the target vehicle are obtained; wherein the detection-related information includes the current position and current time of the target vehicle;

[0022] Based on the current position and current time of the target vehicle, the image parameter standard values ​​at the current position and current time of the target vehicle are matched in a comparison table of image parameter standard values ​​corresponding to different position areas and time periods constructed in advance;

[0023] The actual values ​​of the image parameters of the first frame of the driving road detection image in the first lane line detection cycle are extracted. Based on the image parameter interval between the image parameter standard value and the image parameter actual value, all target lane line detection models whose image parameter adjustment values ​​fall within the image parameter interval in the lane line detection model library are extracted to construct a temporary library of lane line detection models.

[0024] Optionally, the step of performing lane line detection on the driving road detection image using a plurality of target lane line detection models stored in the lane line detection model temporary library to obtain a lane line detection result specifically includes:

[0025] Utilizing a plurality of target lane line detection models stored in the lane line detection model temporary library to perform lane line detection on the driving road detection image, and obtaining lane line pixel point detection results of each target lane line detection model on the driving road detection image;

[0026] All lane line pixel points detected by each target lane line detection model are presented in the same driving road detection image, and least squares fitting is performed on all lane line pixel points to generate lane line detection results in the driving road detection image.

[0027] Optionally, according to the lane line detection result of each lane line detection cycle, the lane line detection model temporary library is updated so that when the target vehicle performs the next lane line detection cycle of the driving task, the lane line detection step is performed based on the updated lane line detection model temporary library, which specifically includes:

[0028] In each lane line detection cycle, the driving trajectory coverage area of ​​the target vehicle in each lane line detection cycle is collected, and it is monitored whether the time proportion of the abnormal distance value between the driving trajectory coverage area and the detected lane line detection in each lane line detection cycle exceeds the preset value;

[0029] If so, return to execute the step of obtaining the driving road detection image and detection association information collected by the target vehicle, extracting a number of target lane line detection models from the lane line detection model library using the detection association information, and obtaining an updated lane line detection model temporary library;

[0030] If not, at the end of each lane line detection cycle, directly return to execute the steps of obtaining the driving road detection image and detection association information collected by the target vehicle, extracting several target lane line detection models from the lane line detection model library using the detection association information, and obtaining an updated lane line detection model temporary library;

[0031] The target vehicle is driven to perform lane line detection based on the updated lane line detection model temporary library in the next lane line detection cycle of the driving task.

[0032] In order to achieve the above object, the present invention also provides a binary classification lane line detection device, comprising:

[0033] An acquisition module is used to acquire a number of sample images of driving roads, perform lane line marking and image preprocessing on the sample images of driving roads, and construct a number of lane line detection samples;

[0034] A construction module is used to construct a binary segmentation network, train the binary segmentation network respectively using each lane line detection sample that performs different image preprocessing, obtain a plurality of candidate lane line detection models, and construct a lane line detection model library storing the plurality of candidate lane line detection models;

[0035] A detection module is used to obtain a driving road detection image and detection association information collected by the target vehicle during the first lane line detection cycle when the target vehicle performs a driving task, extract a plurality of target lane line detection models from a lane line detection model library using the detection association information, build a lane line detection model temporary library, perform lane line detection on the driving road detection image using the plurality of target lane line detection models stored in the lane line detection model temporary library, and obtain a lane line detection result;

[0036] The update module is used to update the lane line detection model temporary library according to the lane line detection result of each lane line detection cycle, so that the target vehicle performs lane line detection based on the updated lane line detection model temporary library when performing the next lane line detection cycle of the driving task.

[0037] In addition, in order to achieve the above-mentioned purpose, the present invention also provides a binary classification lane line detection device, which includes: a memory, a processor, and a binary classification lane line detection program stored in the memory and executable on the processor, and when the binary classification lane line detection program is executed by the processor, the binary classification lane line detection method as described above is implemented.

[0038] In addition, in order to achieve the above-mentioned purpose, the present invention also provides a storage medium, on which a binary classification lane line detection program is stored, and when the binary classification lane line detection program is executed by a processor, the above-mentioned binary classification lane line detection method is implemented.

[0039] The beneficial effects of the present invention are as follows: a binary classification lane line detection method, device, equipment and storage medium are proposed, lane lines are annotated on a plurality of acquired driving road sample images, image preprocessing is performed according to different image adjustment parameters, image amplification and enhancement are completed, a plurality of lane line detection samples are constructed, a plurality of lane line detection samples are used to respectively train the constructed binary segmentation network, a plurality of candidate lane line detection models suitable for different environmental scenes are obtained, in the first lane line detection cycle when the target vehicle performs a driving task, the collected driving road detection image and detection association information are obtained, and the lane line detection model is determined by using the detection association information. A lane line detection model temporary library is formed, and then several target lane line detection models contained in the lane line detection model temporary library are used to perform several lane line detections on the driving road detection image. Based on binary classification and least squares method, lane line detection results are generated. Finally, according to the lane line detection results of each lane line detection cycle, the lane line detection model temporary library is updated. By updating several target lane line detection models adapted to the current environment scene in real time and considering the influence of the lane line detection results of the previous lane line detection cycle on the driving of the target vehicle, lane line detection with better adaptability and the ability to balance detection performance and detection real-time is achieved, thereby improving the detection performance and robustness of lane line detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 A schematic diagram of the device structure of the hardware operating environment involved in the embodiment of the present invention;

[0041] Figure 2 It is a flowchart of an embodiment of a binary classification lane line detection method of the present invention;

[0042] Figure 3 It is a structural schematic diagram of the binary segmentation network of the present invention;

[0043] Figure 4 4 is a structural block diagram of a binary classification lane line detection device in an embodiment of the present invention.

[0044] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0045] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0046] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0047] like Figure 1 As shown, Figure 1 It is a schematic diagram of the device structure of the hardware operating environment involved in the embodiment of the present invention.

[0048] like Figure 1 As shown, the device may include: a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or it may be a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0049] Those skilled in the art will understand that Figure 1 The structure of the device shown in the figure does not constitute a limitation of the device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0050] like Figure 1 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a binary classification lane line detection program.

[0051] exist Figure 1 In the terminal shown, the network interface 1004 is mainly used to connect to the backend server and communicate data with the backend server; the user interface 1003 is mainly used to connect to the client (user end) and communicate data with the client; and the processor 1001 can be used to call the binary classification lane line detection program stored in the memory 1005 and perform the following operations:

[0052] Acquire a number of driving road sample images, perform lane line marking and image preprocessing on the number of driving road sample images, and construct a number of lane line detection samples;

[0053] Constructing a binary segmentation network, training the binary segmentation network respectively using each lane line detection sample that performs different image preprocessing to obtain a plurality of candidate lane line detection models, and constructing a lane line detection model library storing the plurality of candidate lane line detection models;

[0054] In the first lane line detection cycle when the target vehicle performs a driving task, a driving road detection image and detection association information collected by the target vehicle are obtained, several target lane line detection models are extracted from a lane line detection model library using the detection association information, a lane line detection model temporary library is constructed, and several target lane line detection models stored in the lane line detection model temporary library are used to perform lane line detection on the driving road detection image to obtain a lane line detection result;

[0055] According to the lane line detection result of each lane line detection cycle, the lane line detection model temporary library is updated so that the target vehicle performs lane line detection based on the updated lane line detection model temporary library when performing the next lane line detection cycle of the driving task.

[0056] The specific embodiments of the present invention applied to the device are basically the same as the embodiments of the following binary classification lane line detection method, and will not be described in detail here.

[0057] The embodiment of the present invention provides a binary classification lane line detection method, referring to Figure 2 , Figure 2 Schematic diagram of the process of an embodiment of a binary classification lane line detection method of the present invention.

[0058] In this embodiment, the binary classification lane line detection method includes the following steps:

[0059] S100: Acquire a plurality of sample images of driving roads, perform lane line marking and image preprocessing on the plurality of sample images of driving roads, and construct a plurality of lane line detection samples;

[0060] S200: constructing a binary segmentation network, training the binary segmentation network respectively using each lane line detection sample that performs different image preprocessing, obtaining a plurality of candidate lane line detection models, and constructing a lane line detection model library storing the plurality of candidate lane line detection models;

[0061] S300: in the first lane line detection cycle of the target vehicle performing the driving task, obtaining the driving road detection image and detection association information collected by the target vehicle, extracting a plurality of target lane line detection models from the lane line detection model library using the detection association information, constructing a lane line detection model temporary library, and performing lane line detection on the driving road detection image using the plurality of target lane line detection models stored in the lane line detection model temporary library to obtain a lane line detection result;

[0062] S400: updating the lane line detection model temporary library according to the lane line detection result of each lane line detection cycle, so that the target vehicle performs lane line detection based on the updated lane line detection model temporary library when performing the next lane line detection cycle of the driving task.

[0063] It should be noted that traditional lane line detection methods have poor performance and low robustness in complex scenarios. More importantly, when facing different scenarios and environments (for example, lane lines have different image parameters under different environmental factors), using the same lane line detection model for lane line detection has differences in detection accuracy (if a model trained with global sample images or fixed sample images is used for detection, there is a problem of low detection accuracy). Referring to the detection results of multiple models for comprehensive detection output has certain limitations in detection real-time. A good balance between detection performance and detection speed is an important difficulty in lane line detection in assisted driving.

[0064] In order to solve the above problems, this embodiment performs lane line annotation on several acquired sample images of driving roads, performs image preprocessing according to different image adjustment parameters, completes image amplification and enhancement, constructs several lane line detection samples, uses several lane line detection samples to train the constructed binary segmentation network respectively, obtains several candidate lane line detection models suitable for different environmental scenarios, obtains the collected driving road detection images and detection association information in the first lane line detection cycle when the target vehicle performs the driving task, determines the lane line detection model temporary library by using the detection association information, and then uses the lane line detection model to train the constructed binary segmentation network. The detection model temporary library contains several target lane line detection models, which perform several lane line detections on the driving road detection image, and generate lane line detection results based on binary classification and least squares method. Finally, according to the lane line detection results of each lane line detection cycle, the lane line detection model temporary library is updated. By updating several target lane line detection models that adapt to the current environment scene in real time and considering the influence of the lane line detection results of the previous lane line detection cycle on the target vehicle driving, a lane line detection with better adaptability and the ability to balance detection performance and detection real-time is achieved, thereby improving the detection performance and robustness of lane line detection.

[0065] In a preferred embodiment, a plurality of sample images of driving roads are obtained, lane line marking and image preprocessing are performed on the plurality of sample images of driving roads, and a plurality of lane line detection sample steps are constructed, specifically including:

[0066] S110: Acquire a plurality of sample images of driving roads; wherein the sample images of driving roads include actual images of driving roads collected by historical vehicles under standard environments and generated images of driving roads under standard environments output by a DreamBooth image generation model based on a PPdiffusers deep learning library;

[0067] S120: Perform pixel-level point annotation processing on the lane lines in the plurality of driving road sample images to obtain driving road sample images with lane line annotations, perform data preprocessing on the driving road sample images with lane line annotations, and construct a plurality of lane line detection samples.

[0068] On this basis, data preprocessing is performed on the sample image of the driving road with lane line markings to construct several lane line detection sample steps, specifically including:

[0069] S121: performing stepwise adjustment of parameter values ​​of each type of image parameter on the sample image of the driving road with lane line markings, obtaining a plurality of adjusted images of each sample image of the driving road, and associating and storing each adjusted image with a corresponding set of image parameter adjustment values;

[0070] S122: performing image flipping and cropping on each adjustment image, and taking a group of adjustment images having the same set of image parameter adjustment values ​​after image flipping and cropping as a lane line detection sample, thereby constructing and obtaining a plurality of lane line detection samples.

[0071] In this embodiment, the actual image of the driving road collected under the standard environment and the generated image of the driving road under the standard environment output by the DreamBooth image generation model based on the PPdiffusers deep learning library are first obtained as the sample images of the driving road, and then the lane lines in the several sample images of the driving road are subjected to pixel-level point annotation processing, parameter values ​​of each type of image parameters are adjusted stepwise, and data amplification and data enhancement processing such as image flipping and cropping are performed, and finally several adjusted images corresponding to each image parameter adjustment value set are obtained, and several lane line detection samples are constructed respectively. It should be noted that the standard environment refers to an environment in which each type of image parameter of the sample image of the driving road collected or generated is a corresponding preset value, and by generating several lane line detection samples in which the image parameters in the image parameter adjustment value set are adjusted stepwise, model training samples under different environmental scenes are provided, so that different lane line detection models obtained by training can adapt to different environmental scenes. Among them, the image parameters are configured as parameters for adjusting the environmental scene of the driving road image, such as brightness, contrast, exposure, color balance, etc.

[0072] In a preferred embodiment, a binary segmentation network is constructed, each lane line detection sample that performs different image preprocessing is used to train the binary segmentation network respectively, a plurality of candidate lane line detection models are obtained, and a lane line detection model library storing the plurality of candidate lane line detection models is constructed, specifically comprising:

[0073] S210: constructing a binary segmentation network; wherein the binary segmentation network is configured as a network architecture consisting of an encoder and a decoder, the encoder uses a Resnet18 backbone network to extract multi-scale feature information and encode it, the decoder uses an upsampling and feature fusion module to convert the low-dimensional representation back to the original data space, and the encoder and the decoder are connected using a CBAM model consisting of a CAM and a PAM in parallel to extract deep features for image segmentation;

[0074] S220: Input each lane line detection sample that performs different image preprocessing into the binary segmentation network for training, obtain a number of candidate lane line detection models, and construct a lane line detection model library that stores the several candidate lane line detection models.

[0075] In this embodiment, Figure 3 As shown in the figure, a binary segmentation CNN network is used to extract the lane line position, and a codec structure is adopted. Specifically, the codec architecture is used to first extract the binary semantic information of the lane line, and the Resnet18 backbone network is used to extract multi-scale feature information for encoding. A dual attention module is added to extract semantic information from the deepest high-level feature map. In the decoding stage, the low-dimensional representation is converted back to the original data space through upsampling and feature fusion operations, and the binary segmentation feature information map can be obtained.

[0076] First, in the model encoder structure, ResNet18 is used as the backbone network of the binary segmentation model. Its residual connection structure effectively overcomes the gradient vanishing problem and can capture subtle differences between input and output. In the design, high and low dimensional feature maps of layer2, layer3, and layer4 in ResNet18 are extracted respectively as the output of the backbone network. There are more complex situations in the lane line detection task. Road intersections, bifurcations, and lane changes will make the shape and direction of the lane line complicated. At the same time, vehicles or obstacles may block the lane line, which will affect the extraction of feature position information. Therefore, in order to accurately locate the lane line, this design introduces the position attention module PAM (PositionAttention Module) mentioned in DANet, so that the model can better understand the importance of different positions in the image, help the model find lanes at different positions in the image, and ensure the accuracy of detection.

[0077] At the same time, the lane line is not clear or some features are not obvious in the original feature map, which often cannot be accurately extracted due to uneven lighting. In order to solve the above problems, this design introduces the channel attention module CAM (Channel Attention Module) to weight different channel features so that the model pays more attention to the channel information related to the lane line. It helps to suppress channel information unrelated to the lane line, thereby reducing noise and improving the anti-interference ability of the model. Most scholars use CAM and PAM in series, which form the CBAM model and are widely used in the feature extraction stage. However, in this application, the low-resolution image layer4 is compressed by 1x1 convolution channel, and then the feature image is input into the parallel CAM and PAM attention modules, which are simultaneously focused and perceived at the feature level and spatial level, thereby obtaining rich deep feature information and detail information. This method greatly reduces the computational overhead in the encoding stage, improves computational efficiency, and helps to better understand the global semantic information of the image, thereby enhancing the understanding and representation of the input data.

[0078] The decoding stage mainly converts the high-level feature maps from the encoding stage into the segmentation results of the lane lines. The decoder usually uses upsampling and feature fusion techniques to restore the resolution and retain richer semantic information. In Seg-Head, these low-resolution feature maps are upsampled to the resolution of the original input image to more accurately locate the lane lines. For feature maps of different scales, a high- and low-dimensional mixed fusion method is used to make up for the edge contour information lost in the encoder.

[0079] Therefore, in order to better utilize semantic information at different levels, improve the model's sensitivity to lane lines, and reduce the impact of irrelevant information, this module combines the attention mechanism proposed in SENet and proposes a high-low dimension guidance module (HLGM), which uses the spatial position information in the low-level feature map to guide the displacement of high-level semantic information during upsampling, mainly based on different resolutions.

[0080] The input of HLGM is features from different encoder levels, which are skipped and use the spatial attention mechanism to determine the contribution of each position to the fusion. The features at each position are normalized between channels so that the features at each position compete with each other in the channel dimension. The normalized features are mapped using a convolutional layer to generate position attention weights. These attention weights are applied to the original features to weightedly fuse the information at different positions. The weighted channel features and position features are fused by simple element-by-element addition. The fused features are finally output and can be passed to Seg-Head for the final prediction of the task.

[0081] In a preferred embodiment, in the first lane line detection cycle of the target vehicle performing the driving task, the driving road detection image and detection association information collected by the target vehicle are obtained, and several target lane line detection models are extracted from the lane line detection model library using the detection association information. The step of constructing a temporary lane line detection model library specifically includes:

[0082] S310: In the first lane line detection cycle of the target vehicle performing the driving task, a driving road detection image and detection-related information collected by the target vehicle are obtained; wherein the detection-related information includes the current position and current time of the target vehicle;

[0083] S320: Based on the current position and current time of the target vehicle, match the image parameter standard value of the current position and current time of the target vehicle in a pre-tested and constructed image parameter standard value comparison table corresponding to different position areas and time periods;

[0084] S330: Extract the actual values ​​of the image parameters of the first frame of the driving road detection image in the first lane line detection cycle, and based on the image parameter interval between the image parameter standard value and the image parameter actual value, extract all the target lane line detection models whose image parameter adjustment values ​​fall within the image parameter interval in the lane line detection model library, and build a temporary library of lane line detection models.

[0085] Furthermore, the step of performing lane line detection on the driving road detection image using a plurality of target lane line detection models stored in the lane line detection model temporary library to obtain a lane line detection result specifically includes:

[0086] S340: Utilizing a plurality of target lane line detection models stored in the lane line detection model temporary library to perform lane line detection on the driving road detection image, and obtaining lane line pixel point detection results of each target lane line detection model on the driving road detection image;

[0087] S350: presenting all lane line pixel points detected by each target lane line detection model in the same driving road detection image, and performing least squares fitting on all lane line pixel points to generate lane line detection results in the driving road detection image.

[0088] In this embodiment, lane lines are annotated on several acquired sample images of driving roads, and image preprocessing is performed according to different image adjustment parameters to complete image amplification and enhancement, and several lane line detection samples are constructed. The constructed binary segmentation network is trained using the several lane line detection samples to obtain several candidate lane line detection models suitable for different environmental scenarios. In the first lane line detection cycle when the target vehicle performs a driving task, the collected driving road detection images and detection association information are obtained, and a lane line detection model temporary library is determined using the detection association information. Then, several lane line detections are performed on the driving road detection images using several target lane line detection models contained in the lane line detection model temporary library, and lane line detection results are generated based on binary classification and least squares method.

[0089] Specifically, when training the binary segmentation network, the data is first divided into training set, validation set and test set according to 7:1:2; during the training process, the training set is divided into samples according to batch=16, and iterative training is performed; each training cycle is verified, and the model is saved according to the verification results, and finally the model with the best segmentation effect can be obtained; the standard for judging the quality of the model is based on the MIoU value. The MIoU calculation formula is as follows:

[0090]

[0091] Among them, K+1 represents the total number of semantic categories in the dataset, TP is the true number of predictions, FN is the number of predicted false negatives, and FP is the number of predicted false positives.

[0092] In a preferred embodiment, according to the lane line detection result of each lane line detection cycle, the lane line detection model temporary library is updated so that when the target vehicle performs the next lane line detection cycle of the driving task, the lane line detection step is performed based on the updated lane line detection model temporary library, which specifically includes:

[0093] S410: In each lane line detection cycle, the driving trajectory coverage area of ​​the target vehicle in each lane line detection cycle is collected, and it is monitored whether the time proportion of the abnormal distance value between the driving trajectory coverage area and the detected lane line detection in each lane line detection cycle exceeds a preset value;

[0094] S420: If yes, return to execute the step of acquiring the driving road detection image and detection association information collected by the target vehicle, extracting a plurality of target lane line detection models from the lane line detection model library using the detection association information, and obtaining an updated lane line detection model temporary library;

[0095] S430: If not, at the end of each lane line detection cycle, directly return to execute the step of obtaining the driving road detection image and detection association information collected by the target vehicle, and extracting a plurality of target lane line detection models from the lane line detection model library using the detection association information to obtain an updated lane line detection model temporary library;

[0096] S440: Driving the target vehicle to perform lane line detection based on the updated lane line detection model temporary library in the next lane line detection cycle of the driving task.

[0097] In this embodiment, according to the lane line detection results of each lane line detection cycle, the lane line detection model temporary library is updated. By updating several target lane line detection models that adapt to the current environment scene in real time and considering the impact of the lane line detection results of the previous lane line detection cycle on the driving of the target vehicle, a lane line detection with better adaptability and the ability to balance detection performance and detection real-time is achieved, thereby improving the detection performance and robustness of the lane line detection.

[0098] Reference Figure 4 , Figure 4 It is a structural block diagram of an embodiment of a binary classification lane line detection device of the present invention.

[0099] like Figure 4 As shown, the binary classification lane line detection device proposed in the embodiment of the present invention includes:

[0100] An acquisition module 10 is used to acquire a plurality of sample images of driving roads, perform lane line marking and image preprocessing on the plurality of sample images of driving roads, and construct a plurality of lane line detection samples;

[0101] A construction module 20 is used to construct a binary segmentation network, train the binary segmentation network using each lane line detection sample that performs different image preprocessing, obtain a plurality of candidate lane line detection models, and construct a lane line detection model library storing the plurality of candidate lane line detection models;

[0102] The detection module 30 is used to obtain the driving road detection image and detection association information collected by the target vehicle during the first lane line detection cycle of the target vehicle performing the driving task, extract a plurality of target lane line detection models from the lane line detection model library using the detection association information, build a lane line detection model temporary library, perform lane line detection on the driving road detection image using the plurality of target lane line detection models stored in the lane line detection model temporary library, and obtain a lane line detection result;

[0103] The updating module 40 is used to update the lane line detection model temporary library according to the lane line detection result of each lane line detection cycle, so that the target vehicle performs lane line detection based on the updated lane line detection model temporary library when performing the next lane line detection cycle of the driving task.

[0104] Other embodiments or specific implementations of the binary lane line detection device of the present invention can refer to the above-mentioned method embodiments and will not be described in detail here.

[0105] In addition, the present invention also proposes a binary classification lane line detection device, which includes: a memory, a processor, and a binary classification lane line detection program stored in the memory and executable on the processor, and when the binary classification lane line detection program is executed by the processor, the binary classification lane line detection method as described above is implemented.

[0106] The specific implementation of the binary lane line detection device of the present application is basically the same as the embodiments of the binary lane line detection method described above, and will not be repeated here.

[0107] In addition, the present invention also proposes a readable storage medium, the readable storage medium includes a computer readable storage medium, on which a binary classification lane line detection program is stored. The readable storage medium may be Figure 1 The memory 1005 in the terminal may also be at least one of a ROM (Read-Only Memory) / RAM (Random Access Memory), a magnetic disk, and an optical disk. The readable storage medium includes a number of instructions for enabling a binary lane line detection device with a processor to execute the binary lane line detection method described in each embodiment of the present invention.

[0108] The specific implementation in the readable storage medium of the present application is basically the same as the embodiments of the above-mentioned two-classification lane line detection method, and will not be repeated here.

[0109] It is understood that, in the description of this specification, the description with reference to the terms "one embodiment", "another embodiment", "other embodiments", or "first to Nth embodiments" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0110] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0111] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0112] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0113] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A binary lane detection method, characterized in that: The following steps are involved: Acquire a number of sample images of driving roads, perform lane line annotation and image preprocessing on the sample images of driving roads, and construct a number of lane line detection samples; specifically comprising: acquire a number of sample images of driving roads; wherein the sample images of driving roads include actual images of driving roads collected by historical vehicles under standard environments and generated images of driving roads under standard environments output by using the DreamBooth image generation model based on the PPdiffusers deep learning library; perform pixel-level point annotation processing on the lane lines in the sample images of driving roads to obtain sample images of driving roads with lane line annotations, perform data preprocessing on the sample images of driving roads with lane line annotations, and construct a number of lane line detection samples; Among them, the data preprocessing is performed on the sample image of the driving road with lane line markings to construct a number of lane line detection sample steps, specifically including: step-by-step adjustment of the parameter value of each type of image parameter of the sample image of the driving road with lane line markings to obtain a number of adjustment images of each sample image of the driving road, and each adjustment image is associated and stored with a corresponding set of image parameter adjustment values; image flipping and cropping are performed on each adjustment image, and a group of adjustment images with the same set of image parameter adjustment values ​​after image flipping and cropping are used as a lane line detection sample to construct and obtain a number of lane line detection samples; Wherein, the image parameters are configured as parameters for adjusting the environmental scene of the driving road image; Constructing a binary segmentation network, training the binary segmentation network respectively using each lane line detection sample that performs different image preprocessing to obtain a plurality of candidate lane line detection models, and constructing a lane line detection model library storing the plurality of candidate lane line detection models; In the first lane line detection cycle when the target vehicle performs a driving task, a driving road detection image and detection association information collected by the target vehicle are obtained, several target lane line detection models are extracted from a lane line detection model library using the detection association information, a lane line detection model temporary library is constructed, and several target lane line detection models stored in the lane line detection model temporary library are used to perform lane line detection on the driving road detection image to obtain a lane line detection result; Among them, in the first lane line detection cycle when the target vehicle performs the driving task, the driving road detection image and detection association information collected by the target vehicle are obtained, and several target lane line detection models are extracted from the lane line detection model library using the detection association information to construct a temporary lane line detection model library step, specifically including: in the first lane line detection cycle when the target vehicle performs the driving task, the driving road detection image and detection association information collected by the target vehicle are obtained; wherein the detection association information includes the current position and current time of the target vehicle; based on the current position and current time of the target vehicle, the image parameter standard value corresponding to different position areas and time periods constructed in advance is matched in the image parameter standard value comparison table; the image parameter actual value of the first frame of the driving road detection image in the first lane line detection cycle is extracted, and based on the image parameter interval between the image parameter standard value and the image parameter actual value, all the target lane line detection models whose image parameter adjustment values ​​fall into the image parameter interval in the lane line detection model library are extracted to construct a temporary lane line detection model library; According to the lane line detection result of each lane line detection cycle, the lane line detection model temporary library is updated so that the target vehicle performs lane line detection based on the updated lane line detection model temporary library when performing the next lane line detection cycle of the driving task.

2. The binary lane detection method according to claim 1, characterized in that: The steps of constructing a binary segmentation network, training the binary segmentation network respectively using each lane detection sample that performs different image preprocessing to obtain a plurality of candidate lane detection models, and constructing a lane detection model library storing the plurality of candidate lane detection models specifically include: Constructing a binary segmentation network; wherein the binary segmentation network is configured as a network architecture consisting of an encoder and a decoder, the encoder uses a Resnet18 backbone network to extract multi-scale feature information and encode it, the decoder uses an upsampling and feature fusion module to convert the low-dimensional representation back to the original data space, and the encoder and the decoder are connected using a CBAM model consisting of a CAM and a PAM in parallel to extract deep features for image segmentation; Each lane line detection sample that performs different image preprocessing is input into the binary segmentation network for training to obtain a number of candidate lane line detection models, and a lane line detection model library storing the several candidate lane line detection models is constructed.

3. The binary lane detection method according to claim 1, characterized in that: The step of performing lane line detection on the driving road detection image using a plurality of target lane line detection models stored in the lane line detection model temporary library to obtain a lane line detection result specifically includes: Utilizing a plurality of target lane line detection models stored in the lane line detection model temporary library to perform lane line detection on the driving road detection image, and obtaining lane line pixel point detection results of each target lane line detection model on the driving road detection image; All lane line pixel points detected by each target lane line detection model are presented in the same driving road detection image, and least squares fitting is performed on all lane line pixel points to generate lane line detection results in the driving road detection image.

4. The binary lane line detection method according to claim 3, characterized in that: According to the lane line detection result of each lane line detection cycle, the lane line detection model temporary library is updated so that when the target vehicle performs the next lane line detection cycle of the driving task, the lane line detection step is performed based on the updated lane line detection model temporary library, specifically including: In each lane line detection cycle, the driving trajectory coverage area of ​​the target vehicle in each lane line detection cycle is collected, and it is monitored whether the time proportion of the abnormal distance value between the driving trajectory coverage area and the detected lane line detection in each lane line detection cycle exceeds the preset value; If so, return to execute the step of obtaining the driving road detection image and detection association information collected by the target vehicle, extracting a number of target lane line detection models from the lane line detection model library using the detection association information, and obtaining an updated lane line detection model temporary library; If not, at the end of each lane line detection cycle, directly return to execute the steps of obtaining the driving road detection image and detection association information collected by the target vehicle, extracting several target lane line detection models from the lane line detection model library using the detection association information, and obtaining an updated lane line detection model temporary library; The target vehicle is driven to perform lane line detection based on the updated lane line detection model temporary library in the next lane line detection cycle of the driving task.

5. A binary lane detection device, characterized in that: include: An acquisition module is used to acquire a number of sample images of driving roads, perform lane line marking and image preprocessing on the sample images of driving roads, and construct a number of lane line detection samples; A construction module is used to construct a binary segmentation network, train the binary segmentation network respectively using each lane line detection sample that performs different image preprocessing, obtain several candidate lane line detection models, and construct a lane line detection model library storing several candidate lane line detection models; specifically comprising: obtaining several sample images of driving roads; wherein the sample images of driving roads include actual images of driving roads collected by historical vehicles under standard environments and generated images of driving roads under standard environments output by a DreamBooth image generation model based on a PPdiffusers deep learning library; pixel-level point annotation processing is performed on lane lines in the several sample images of driving roads to obtain sample images of driving roads with lane line annotations, data preprocessing is performed on the sample images of driving roads with lane line annotations, and several lane line detection samples are constructed; Among them, the data preprocessing is performed on the sample image of the driving road with lane line markings to construct a number of lane line detection sample steps, specifically including: step-by-step adjustment of the parameter value of each type of image parameter of the sample image of the driving road with lane line markings to obtain a number of adjustment images of each sample image of the driving road, and each adjustment image is associated and stored with a corresponding set of image parameter adjustment values; image flipping and cropping are performed on each adjustment image, and a group of adjustment images with the same set of image parameter adjustment values ​​after image flipping and cropping are used as a lane line detection sample to construct and obtain a number of lane line detection samples; Wherein, the image parameters are configured as parameters for adjusting the environmental scene of the driving road image; A detection module is used to obtain a driving road detection image and detection association information collected by the target vehicle during the first lane line detection cycle when the target vehicle performs a driving task, extract a plurality of target lane line detection models from a lane line detection model library using the detection association information, build a lane line detection model temporary library, perform lane line detection on the driving road detection image using the plurality of target lane line detection models stored in the lane line detection model temporary library, and obtain a lane line detection result; Among them, in the first lane line detection cycle when the target vehicle performs the driving task, the driving road detection image and detection association information collected by the target vehicle are obtained, and several target lane line detection models are extracted from the lane line detection model library using the detection association information to construct a temporary lane line detection model library step, specifically including: in the first lane line detection cycle when the target vehicle performs the driving task, the driving road detection image and detection association information collected by the target vehicle are obtained; wherein the detection association information includes the current position and current time of the target vehicle; based on the current position and current time of the target vehicle, the image parameter standard value corresponding to different position areas and time periods constructed in advance is matched in the image parameter standard value comparison table; the image parameter actual value of the first frame of the driving road detection image in the first lane line detection cycle is extracted, and based on the image parameter interval between the image parameter standard value and the image parameter actual value, all the target lane line detection models whose image parameter adjustment values ​​fall into the image parameter interval in the lane line detection model library are extracted to construct a temporary lane line detection model library; The update module is used to update the lane line detection model temporary library according to the lane line detection result of each lane line detection cycle, so that the target vehicle performs lane line detection based on the updated lane line detection model temporary library when performing the next lane line detection cycle of the driving task.

6. A binary lane detection device, characterized in that: The binary lane line detection device includes: a memory, a processor, and a binary lane line detection program stored in the memory and executable on the processor. When the binary lane line detection program is executed by the processor, the binary lane line detection method according to any one of claims 1 to 4 is implemented.

7. A storage medium, characterized in that: The storage medium stores a binary lane line detection program, and when the binary lane line detection program is executed by the processor, the binary lane line detection method according to any one of claims 1 to 4 is implemented.

Images