Lane line detection method and device based on structure search, equipment and storage medium

Abstract

The invention discloses a lane line detection method and device based on structure search, equipment and a storage medium, and relates to the technical field of intelligent driving, and the method comprises the steps: obtaining a road condition image, carrying out the preprocessing of the road condition image, and obtaining a training set; based on a preset search block structure and a preset function, obtaining a target preset search block structure through the training set; establishing a lane line detection model through the target preset search block structure; and completing lane line detection through the lane line detection model. The lane line detection model is established through the preset search block structure, and the design efficiency and the detection precision of the lane line detection model are improved.

Description

Technical Field

[0001] This application relates to the field of intelligent driving technology, and particularly to a lane line detection method, device, equipment and storage medium based on structure search. Background Art

[0002] Lane line detection is a key link for environmental perception in an autonomous driving system, aiming to identify and detect information such as the position, color, solid or dashed state of lane lines in a road image, so as to help the driving system achieve functions such as lane keeping and lane changing.

[0003] Currently, lane line detection methods are mainly divided into two categories: traditional lane line detection methods and deep learning-based lane line detection methods. Traditional methods detect lane lines in an image by performing operations such as gray-scale transformation, edge extraction, and lane line fitting on the image, and the recognition effect is poor in scenarios with complex road conditions. Currently, deep learning-based lane line detection algorithms are widely used. These methods have a strong feature extraction ability by stacking a large number of convolutional layers. However, as the model scale becomes larger and larger, the difficulty of manually designing the network becomes higher and higher, and the improvement of model accuracy has reached a bottleneck. Summary of the Invention

[0004] The main purpose of this application is to provide a lane line detection method, device, equipment and storage medium based on structure search, aiming to solve the technical problem that the existing lane line detection solutions cannot guarantee both high accuracy and detection speed of the detection results.

[0005] To achieve the above purpose, this application proposes a lane line detection method based on structure search, and the lane line detection method based on structure search includes:

[0006] Obtain a road condition image, preprocess the road condition image to obtain a training set;

[0007] Based on a preset search block structure and a preset function, obtain a target preset search block structure through the training set;

[0008] Establish a lane line detection model through the target preset search block structure;

[0009] Complete lane line detection through the lane line detection model.

[0010] In one embodiment, the step of alternately optimizing the structure parameters and model parameters in the preset function through the training set to obtain optimized structure parameters and optimized model parameters includes:

[0011] Divide the training set into a first subset and a second subset according to a preset ratio;

[0012] Keep the structural parameters in the preset function unchanged, and use the first subset to train and optimize the model parameters;

[0013] Keep the model parameters in the preset function unchanged, and use the second subset to train and optimize the structural parameters;

[0014] Obtain the number of rounds of training optimization. When the number of rounds is less than or equal to the preset number of rounds, return to execute the step of keeping the structural parameters in the preset function unchanged and using the first subset to train and optimize the model parameters until the number of rounds is greater than the preset number of rounds, so as to obtain the optimized structural parameters and the optimized model parameters.

[0015] In one embodiment, the step of alternately optimizing the structural parameters and the model parameters in the preset function through the training set to obtain the optimized structural parameters and the optimized model parameters includes:

[0016] Divide the training set into a first subset and a second subset according to a preset ratio;

[0017] Keep the structural parameters in the preset function unchanged, and use the first subset to train and optimize the model parameters;

[0018] Keep the model parameters in the preset function unchanged, and use the second subset to train and optimize the structural parameters;

[0019] Obtain the number of rounds of training optimization. When the number of rounds is less than or equal to the preset number of rounds, return to execute the step of keeping the structural parameters in the preset function unchanged and using the first subset to train and optimize the model parameters until the number of rounds is greater than the preset number of rounds, so as to obtain the optimized structural parameters and the optimized model parameters.

[0020] In one embodiment, the step of obtaining the weights corresponding to the operators between the input features and the intermediate features in the preset search block structure and retaining the operator with the largest weight includes:

[0021] In the preset search block structure, perform operator processing on the input features to obtain a number of intermediate results;

[0022] Perform weighted summation on the number of intermediate results to obtain intermediate features;

[0023] Obtain the weights corresponding to the operators between the input features and the intermediate features, and retain the operator with the largest weight.

[0024] In one embodiment, the preset search block structure at least includes: a downsampling search block and an upsampling search block;

[0025] The step of performing operator processing on the input features in the preset search block structure to obtain a number of intermediate results includes:

[0026] In the downsampling search block, the input features are subjected to convolution, depth convolution, dilated convolution, average pooling, max pooling, and identity mapping to obtain a first intermediate result;

[0027] In the upsampling search block, the input features are subjected to transposed convolution and bilinear interpolation upsampling to obtain a second intermediate result;

[0028] A number of intermediate results are obtained based on the first intermediate result and the second intermediate result.

[0029] In one embodiment, the steps of obtaining a road condition image and preprocessing the road condition image to obtain a training set include:

[0030] Obtain a road condition image and perform anomaly detection on the road condition image;

[0031] When an abnormal image is detected in the road condition image, delete the abnormal image;

[0032] Use the road condition image after the deletion operation as the original data set;

[0033] Perform image processing on the original data set to obtain a training set.

[0034] In one embodiment, the steps of performing image processing on the original data set to obtain a training set include:

[0035] Perform image processing such as lane line annotation, scaling, cropping, random flipping, adding Gaussian noise, and normalization on the image data in the original data set to obtain an updated data set;

[0036] Use a preset ratio of the image data in the updated data set as the training set.

[0037] In addition, to achieve the above object, the present application also proposes a lane line detection device based on structure search, and the lane line detection device based on structure search includes:

[0038] An acquisition module for acquiring a road condition image and preprocessing the road condition image to obtain a training set;

[0039] A determination module for obtaining a target preset search block structure through the training set based on a preset search block structure and a preset function;

[0040] The determination module is further configured to establish a lane line detection model through the target preset search block structure;

[0041] A detection module for completing lane line detection through the lane line detection model.

[0042] In addition, to achieve the above object, the present application further provides a lane line detection device based on structural search, the device comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, the computer program being configured to implement the steps of the lane line detection method based on structural search as described above.

[0043] In addition, to achieve the above object, the present application further provides a storage medium, the storage medium being a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the lane line detection method based on structural search as described above are implemented.

[0044] In addition, to achieve the above object, the present application further provides a computer program product, the computer program product comprising a computer program, and when the computer program is executed by a processor, the steps of the lane line detection method based on structural search as described above are implemented.

[0045] One or more technical solutions proposed by the present application include: acquiring a road condition image, preprocessing the road condition image to obtain a training set; obtaining a target preset search block structure through the training set based on a preset search block structure and a preset function; establishing a lane line detection model through the target preset search block structure; and completing lane line detection through the lane line detection model. By establishing a lane line detection model through a preset search block structure, the design efficiency and detection accuracy of the lane line detection model are improved. Description of the Drawings

[0046] The drawings here are incorporated into the specification and form a part of this specification, showing embodiments consistent with the present application and used together with the specification to explain the principles of the present application.

[0047] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0048] Figure 1 It is a schematic flowchart provided for the first embodiment of the lane line detection method based on structural search of the present application;

[0049] Figure 2 It is a schematic diagram of the principle of the lane line detection model composed of repeatedly stacking downsampling search blocks and upsampling search blocks in an embodiment of the lane line detection method based on structural search of the present application;

[0050] Figure 3 It is a schematic flowchart provided for the second embodiment of the lane line detection method based on structural search of the present application;

[0051] Figure 4 This is a schematic flowchart of a lane line detection method based on structure search provided by an embodiment of the present application;

[0052] Figure 5 This is a schematic module structure diagram of a lane line detection device based on structure search according to an embodiment of the present application;

[0053] Figure 6 This is a schematic device structure diagram of the hardware operating environment involved in the lane line detection method based on structure search in an embodiment of the present application.

[0054] The realization of the purpose, functional features and advantages of the present application will be further described with reference to the embodiments and the accompanying drawings. Specific embodiments

[0055] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0056] For a better understanding of the technical solutions of the present application, the following will be described in detail in conjunction with the accompanying drawings of the specification and specific embodiments.

[0057] The main solution of the embodiment of the present application is: obtaining a road condition image, preprocessing the road condition image to obtain a training set; obtaining a target preset search block structure through the training set based on a preset search block structure and a preset function; establishing a lane line detection model through the target preset search block structure; and completing lane line detection through the lane line detection model.

[0058] Since lane line detection methods are mainly divided into two categories, traditional lane line detection methods and lane line detection methods based on deep learning. Traditional methods detect lane lines in an image by performing operations such as gray-scale transformation, edge extraction, and lane line fitting on the image, and the recognition effect is poor in scenarios with complex road conditions. At present, lane line detection algorithms based on deep learning are widely used. These methods have a strong feature extraction ability by stacking a large number of convolutional layers. However, as the model scale becomes larger and larger, the difficulty of manually designing the network becomes higher and higher, and the improvement of model accuracy has fallen into a bottleneck.

[0059] The present application provides a solution to establish a lane line detection model through a preset search block structure, improving the design efficiency and detection accuracy of the lane line detection model.

[0060] It should be noted that the execution subject of this embodiment can be a computing service device with data processing, network communication, and program running functions, such as a tablet computer, a personal computer, a mobile phone, etc., or an electronic device capable of implementing the above functions, a lane line detection device based on structure search, etc. Hereinafter, taking the lane line detection device based on structure search as an example, this embodiment and the following embodiments will be described.

[0061] Based on this, an embodiment of the present application provides a lane line detection method based on structure search. Referring to Figure 1 , Figure 1 is a schematic flowchart of the first embodiment of the lane line detection method based on structure search of the present application.

[0062] In this embodiment, the lane line detection method based on structure search includes steps S10 to S40:

[0063] Step S10: Obtain a road condition image, preprocess the road condition image, and obtain a training set.

[0064] It should be noted that a road condition image usually refers to image data of the road traffic state obtained in real time through devices such as sensors and cameras. These images can reflect information such as road surface conditions, traffic flow, obstacles, and traffic lights, helping the driving system to judge the environment and improve driving safety and efficiency. Among them, road conditions can cover various different scenarios, such as different lighting conditions such as sunny, cloudy, rainy, day, and night, as well as different types of roads such as urban roads, expressways, ramps, straight roads, and curved roads.

[0065] In addition, a training set usually refers to a set of data containing data and its corresponding labels or results, which is used to train a machine learning model so that it can learn the patterns and relationships in the data. In this embodiment, the training set is used for the parameters in the preset function.

[0066] In a specific embodiment, the preprocessing includes anomaly detection and image processing. A road condition image can be obtained, and anomaly detection is performed on the road condition image; when an abnormal image is detected in the road condition image, the abnormal image is deleted; the road condition image after the deletion operation is used as the original data set; and image processing is performed on the original data set to obtain a training set.

[0067] Among them, performing image processing on the original data set to obtain a training set specifically means: performing image processing such as lane line annotation, scaling, cropping, random flipping, adding Gaussian noise, and normalization on the image data in the original data set to obtain an updated data set; and using a preset ratio of the image data in the updated data set as the training set.

[0068] Specifically, obtain a road condition image and perform anomaly detection on the road condition image; when an abnormal image (such as a blurred or occluded image caused by foreign matter on the camera lens) is detected in the road condition image, delete the abnormal image; use the road condition image after the deletion operation as the original data set; the road condition images in the original data set can be labeled with annotation software, tags can be added to each lane line, and image processing such as scaling, cropping (such as setting the image size to 256*576), random flipping, adding Gaussian noise, and normalization can be performed on the road condition image to enhance the effect of the road condition image, and an updated data set is obtained; use a preset ratio (such as taking 5 / 6 of the updated data set) of the image data in the updated data set as the training set, and use the training set for subsequent training optimization.

[0069] It can be understood that scaling usually refers to changing the size of an image, which can be enlarging or reducing the image. The enlargement operation usually uses interpolation algorithms (such as nearest neighbor interpolation, bilinear interpolation, bicubic interpolation, etc.) to increase the number of pixels and make the image visually larger. However, excessive enlargement may cause the image to become blurred. Reducing the image is to reduce the number of pixels and sample the original image according to certain rules, such as taking one pixel point every few pixels, etc., so that the image size becomes smaller. Scaling can be based on a fixed scale factor, such as uniformly reducing the length and width of the image to 50% of the original, or a specific target size can be specified, such as scaling the image to a width of 300 pixels and a height of 200 pixels. By processing the image at different scaling ratios, the appearance of the image at different viewing distances or different display resolutions can be simulated, enriching the data representation form, helping the model learn more comprehensive features, and enhancing its generalization ability.

[0070] Cropping usually refers to selecting a specific local area from the original image and removing the redundant content at the edge of the image. There are various cropping methods, which can be cropped according to a fixed shape (such as a rectangle) and size. For example, in a road condition image, only the part containing key traffic elements such as traffic signs or vehicles is cropped, and the area to be retained is accurately selected by setting the coordinate range. Cropping can reduce the interference of irrelevant information, highlight the main object and key features in the image, enable the algorithm to focus more on the important part during subsequent processing, and increase the data diversity to a certain extent.

[0071] Random flipping usually refers to randomly flipping the image horizontally or vertically. By random flipping, the data volume can be expanded and the image states at different perspectives can be simulated, enabling the model to learn the features of the object when presented at different angles and enhancing the generalization ability of the model.

[0072] Gaussian noise is a type of random noise that follows a normal distribution. At each pixel point of an image, corresponding noise values are added according to the set mean and standard deviation. For example, if the mean is set to 0 and the standard deviation is a certain value (such as 10, 20, etc., depending on the actual situation and the expected noise intensity), then the original pixel value at each pixel point will be superimposed with this noise value that conforms to the normal distribution law, making the image become "noisy" and showing some changes like graininess. Adding Gaussian noise enables the model to learn how to accurately extract features and make judgments in the presence of noise interference, that is, when the image quality is not ideal, and avoids a significant drop in the model's performance when facing various interference noises that may occur in practical applications, enhancing the robustness of the model.

[0073] There are various common methods for normalizing image pixel values. For example, mapping the pixel values from the original range of 0 - 255 (for 8-bit images) to the interval of 0 - 1, that is, dividing the original value of each pixel point by 255; or through a more complex standardization operation, first calculating the mean and standard deviation of all pixel values of the image, and then subtracting the mean from each pixel value and dividing by the standard deviation, so that the processed image pixel values conform to the characteristics of a standard normal distribution with a specific mean of 0 and a standard deviation of 1, etc. Normalization helps to accelerate the training convergence speed of the model and improve the training efficiency. For different images, the pixel value ranges and so on may vary greatly. After normalization, the data can be in a more unified and standardized numerical interval. When input into deep learning and other models, the model can better learn features and adjust parameters based on these regularized data, and to a certain extent, it can also improve the stability and accuracy of the model.

[0074] Step S20: Based on a preset search block structure and a preset function, obtain a target preset search block structure through the training set.

[0075] It should be noted that the preset search block structure at least includes: a downsampling search block and an upsampling search block. And by repeatedly stacking the downsampling search block and the upsampling search block, a lane line detection model can be constructed, as Figure 2 shown, Figure 2 is a schematic diagram of the principle of a lane line detection model constructed by repeatedly stacking the downsampling search block and the upsampling search block. In this embodiment, it is necessary to obtain a target preset search block structure through the training set based on the preset search block structure and the preset function, and establish a lane line detection model through the target preset search block structure, where the target preset search block structure represents the optimal preset search block structure.

[0076] In addition, a suitable search algorithm can be used to find the optimal preset search block structure. The search algorithm at least includes: a differentiable search method, a method based on reinforcement learning or an evolutionary algorithm.

[0077] It can be understood that a preset function can be obtained according to the structural parameters, model parameters, delay parameters, balance parameters, and amplitude parameters. The preset function is used to limit the scale of the lane line detection model.

[0078] Step S30: Establish a lane line detection model through the target preset search block structure.

[0079] Specifically, the appropriate model architecture can be selected and constructed according to the search results obtained from the optimal preset search block structure. For example, if the search results indicate that a specific Convolutional Neural Networks (CNN) architecture and parameter combination perform optimally, the lane line detection model will be constructed according to this result. In addition to the traditional CNN model, some models specifically for lane line detection can also be considered, such as LaneNet (a deep learning model for lane line detection), Spatial Pyramid Convolutional Neural Networks (SCNN), Research, Evaluation, and Systems Analysis (RESA), etc.

[0080] Step S40: Complete lane line detection through the lane line detection model.

[0081] In a specific embodiment, the image or video frame to be detected can be input into the lane line detection model. These images or video frames can come from the camera installed on the vehicle or a pre-collected and stored data set; through the network structure inside the lane line detection model, such as the convolutional layer and pooling layer of CNN, etc., the input image data is feature-extracted to obtain a feature map containing lane line features; based on the extracted features, the model predicts the lane line through a specific algorithm or network layer. For example, a model based on semantic segmentation will classify each pixel point in the image to determine whether it belongs to the lane line category, thereby generating a semantic segmentation map, where the pixel points in the lane line area are marked with specific categories, such as the category labels of white or yellow lane lines, while a model based on key point detection will predict the key position points on the lane line, such as the starting point, ending point, turning point, etc. of the lane line, and then fit the shape of the lane line through these key points; then post-process the prediction results to improve the accuracy and stability of lane line detection, which can include: removing noise and small-area interference, lane line fitting and optimization, and multi-frame fusion and tracking; output the post-processed lane line detection results, usually by visually drawing the detected lane lines on the original image or video, or outputting information such as the position, type, and curvature of the lane line in the form of data for subsequent use by the autonomous driving decision-making system or other applications.

[0082] Among them, removing noise and small-area interference specifically means that morphological operations, such as erosion and dilation operations, can be used to remove isolated noise points and small-area non-lane-line regions in the segmentation map, making the shape of the lane lines smoother and more continuous; lane line fitting and optimization specifically means that for the detected lane line key points or segmentation regions, a curve fitting algorithm, such as polynomial fitting, is used to generate a curve that is smoother and more in line with the actual shape of the lane lines. The prior knowledge of lane lines, such as the width and curvature range of lane lines, can also be combined to constrain and optimize the detection results; multi-frame fusion and tracking specifically means that if lane line detection is performed on video data, the information association between adjacent frames can be utilized, and through multi-frame fusion and tracking algorithms, the accuracy and stability of lane line detection can be further improved, and the situations of false detection and missed detection can be reduced.

[0083] This embodiment provides a lane line detection method based on structural search, which acquires a road condition image, preprocesses the road condition image to obtain a training set; based on a preset search block structure and a preset function, obtains a target preset search block structure through the training set; establishes a lane line detection model through the target preset search block structure; and completes lane line detection through the lane line detection model. By establishing a lane line detection model through a preset search block structure, the design efficiency and detection accuracy of the lane line detection model are improved.

[0084] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar content as that in the above-mentioned first embodiment can be referred to the above introduction and will not be repeated hereinafter. On this basis, please refer to Figure 3 , step S20 includes steps S201 to S203:

[0085] Step S201: Alternately optimize the structural parameters and model parameters in the preset function through the training set to obtain optimized structural parameters and optimized model parameters.

[0086] In a specific embodiment, the training set can be divided into a first subset and a second subset according to a preset ratio; keep the structural parameters in the preset function unchanged, and use the first subset to train and optimize the model parameters; keep the model parameters in the preset function unchanged, and use the second subset to train and optimize the structural parameters; obtain the number of training optimization rounds, and when the number of rounds is less than or equal to the preset number of rounds, return to execute the step of keeping the structural parameters in the preset function unchanged and using the first subset to train and optimize the model parameters until the number of rounds is greater than the preset number of rounds to obtain optimized structural parameters and optimized model parameters.

[0087] It should be noted that a preset function can be obtained based on the structural parameters, model parameters, latency parameters, balance parameters, and amplitude parameters. The preset function is used to limit the scale of the lane line detection model. Refer to Equation 1 below, which is the expression of the preset function:

[0088] L(a, w) = SegLoss(a, w)·αlog(LAT(a)) β (Equation 1)

[0089] Among them, the structural parameter a represents the structural parameter corresponding to the operator between features, the model parameter w represents the learnable model parameter inside the operator, the latency parameter LAT() represents the latency of the overall network structure on the hardware device, the balance parameter α is used to balance the model effect and the model latency, and the amplitude parameter β is used to control the amplitude of the latency loss term.

[0090] Specifically, the structural parameter a and the model parameter w can be optimized. Therefore, the training set can be divided into a first subset and a second subset according to a preset ratio (such as 1:1) to obtain the target preset search block structure. In the first stage, the structural parameter a is fixed, and the first subset of data is used to train the model parameter w; in the second stage, the model parameter w is fixed, and the second subset of data is used to train the structural parameter a. The two stages are alternated until the number of rounds (after going through the first stage and the second stage successively is counted as one round) is greater than the preset number of rounds (the size of the number of rounds pre-set manually), and the optimized structural parameter and the optimized model parameter are obtained. In addition, when the above number of rounds is greater than the preset number of rounds, the structural parameter a and the model parameter w usually converge.

[0091] Step S202: Obtain the weights corresponding to the operators between the input features and the intermediate features in the preset search block structure, and retain the operator with the largest weight.

[0092] It should be noted that the input features refer to the original data received by the model, and these data are usually preprocessed image, audio, or text data. The input features usually have specific dimensions, such as height, width, and number of channels. For example, the input features of an image may be a tensor with a shape of (height, width, 3), where 3 represents the color channels.

[0093] In addition, the intermediate features usually refer to a series of intermediate representations generated by the model during the process of processing the input data. In a CNN, these feature maps are extracted from the input features through operations such as convolutional layers and pooling layers. The intermediate feature maps capture different levels of information of the input data, from low-level edge and texture information to higher-level abstract concepts.

[0094] In this embodiment, the operator can refer to a convolution operator, which is used to scan the input data (such as an image) in the convolutional layer and calculate the dot product of the convolution operator and the input data at each position, thereby generating a feature map. This process helps to extract local features in the input data, such as edges, textures, etc.

[0095] In a specific implementation, in the preset search block structure, the input feature is processed by an operator to obtain a number of intermediate results; the number of intermediate results are weighted and summed to obtain an intermediate feature; the weight corresponding to the operator between the input feature and the intermediate feature is obtained, and the operator with the largest weight is retained.

[0096] The formula for retaining the operator with the largest weight is shown in Equation 2 below:

[0097]

[0098] Among them, in the preset search block structure, the step of processing the input feature by an operator to obtain a number of intermediate results is specifically to perform convolution, depth convolution, dilated convolution, average pooling, max pooling, and identity mapping processing on the input feature in the downsampling search block to obtain a first intermediate result; in the upsampling search block, perform transposed convolution and bilinear interpolation upsampling processing on the input feature to obtain a second intermediate result; a number of intermediate results are obtained according to the first intermediate result and the second intermediate result.

[0099] Specifically, after obtaining a number of intermediate results, these intermediate results can be weighted and summed to obtain an intermediate feature. After multiple processes, multiple intermediate features are obtained, and then these intermediate features are summed to obtain an output feature. For ease of understanding, the operator between the input feature and the intermediate feature is as shown in Equation 3 below:

[0100]

[0101] Among them, o represents the operator, o(x) represents the intermediate result after the operator processing, represents the weight corresponding to the operator between the input feature and the intermediate feature.

[0102] Convolution is a mathematical operation that is used in deep learning to extract features of input data. In a convolutional neural network (CNN), the convolutional layer slides a window (convolution kernel or filter) over the input data (such as an image) and calculates the dot product between the window and the elements of the input data to generate a feature map. This process can capture local features, such as edges, textures, etc.

[0103] Depth convolution is part of depthwise separable convolution, which independently applies a convolutional kernel to each channel of the input data. This means that if the input data has multiple channels, each channel will be convolved with a convolutional kernel separately. This operation reduces the number of model parameters and computational volume while maintaining the ability to extract features.

[0104] Atrous convolution is a special convolutional operation that inserts spaces (holes) between the elements of the convolutional kernel, thereby expanding the receptive field of the convolutional kernel without increasing the number of parameters. This operation helps to capture more extensive context information and is commonly used in tasks such as image segmentation.

[0105] Average pooling is a pooling operation that reduces the spatial size of the feature map by calculating the average value within the region of the input feature map. This operation can reduce the spatial size of the data, extract local features, and provide a certain degree of invariance.

[0106] Max pooling is another pooling operation that reduces the spatial size of the feature map by selecting the maximum value within the region of the input feature map. Max pooling helps to highlight the most important features and increase the invariance of the model to small transformations.

[0107] Identity mapping is a special operation that directly passes the input data to the output without any change. In some network structures, such as residual networks, identity mapping is used to connect the outputs of different layers, allowing the network to learn features other than the identity transformation while maintaining the direct flow of information, which helps to solve the vanishing gradient problem in deep networks.

[0108] Transposed convolution is used to enlarge the size of the feature map. It enlarges the size of the input image by padding with zeros, then rotates the convolutional kernel, and then performs the forward convolution process. Transposed convolution is commonly used in tasks such as image segmentation and object detection to upsample low-resolution feature maps to high resolution.

[0109] Bilinear interpolation upsampling is based on the principle of bilinear interpolation, that is, linear interpolation is performed in two directions (usually horizontal and vertical directions). Its core idea is to estimate the values of the newly added pixels in the high-resolution image based on the pixel values in the known low-resolution image.

[0110] In summary, these operator processes are for feature extraction, dimensionality reduction, increasing invariance, and constructing complex feature representations.

[0111] It can be understood that by retaining the operator with the largest weight, it helps to quickly find the optimal preset search block structure suitable for the lane detection model.

[0112] Step S203: Obtain the target preset search block structure according to the optimized structural parameters, optimized model parameters, and the operator with the largest weight.

[0113] It can be understood that, according to the optimized structural parameters, the optimized model parameters, and the operator with the largest weight, the target preset search block structure can be obtained, which enables the subsequent established lane line detection model to have better performance and efficiency.

[0114] In this embodiment, the training set is used to alternately optimize the structural parameters and model parameters in the preset function to obtain the optimized structural parameters and the optimized model parameters; the weights corresponding to the operators between the input features and the intermediate features in the preset search block structure are obtained, and the operator with the largest weight is retained; the target preset search block structure is obtained according to the optimized structural parameters, the optimized model parameters, and the operator with the largest weight. By using the training set to optimize the structural parameters and model parameters, and by retaining the operator with the largest weight to obtain the target preset search block structure, the efficiency of finding the target preset search block structure is improved.

[0115] Exemplarily, to facilitate understanding of the implementation process of the lane line detection method based on structural search obtained by combining the above-mentioned Embodiment 1, please refer to Figure 4 , Figure 4 A schematic flowchart of a brief process of a lane line detection method based on structural search is provided. Specifically: obtain a road condition image, preprocess the road condition image to obtain a training set; divide the training set into a first subset and a second subset according to a preset ratio; keep the structural parameters in the preset function unchanged, and use the first subset to train and optimize the model parameters; keep the model parameters in the preset function unchanged, and use the second subset to train and optimize the structural parameters; obtain the number of training optimization rounds, and when the number of rounds is less than or equal to the preset number of rounds, return to execute the step of keeping the structural parameters in the preset function unchanged and using the first subset to train and optimize the model parameters until the number of rounds is greater than the preset number of rounds to obtain the optimized structural parameters and the optimized model parameters; obtain the weights corresponding to the operators between the input features and the intermediate features in the preset search block structure, and retain the operator with the largest weight; obtain the target preset search block structure according to the optimized structural parameters, the optimized model parameters, and the operator with the largest weight; establish a lane line detection model through the target preset search block structure; complete lane line detection through the lane line detection model. By using the preset function and automatically optimizing the parameters, the technical effect of reducing the complexity and computational amount of the model while maintaining the high accuracy of the model is achieved, thereby improving the detection speed and reducing the demand for hardware resources.

[0116] It should be noted that the above examples are only for understanding the present application and do not constitute a limitation to the lane line detection method based on structural search of the present application. Any simple transformation in more forms based on this technical concept is within the protection scope of the present application.

[0117] The present application also provides a lane line detection device based on structure search. Please refer to Figure 5 , the lane line detection device based on structure search includes:

[0118] An acquisition module 10, configured to acquire a road condition image, preprocess the road condition image, and obtain a training set.

[0119] A determination module 20, configured to obtain a target preset search block structure through the training set based on a preset search block structure and a preset function.

[0120] The determination module 20 is further configured to establish a lane line detection model through the target preset search block structure.

[0121] A detection module 30, configured to complete the lane line detection degree through the lane line detection model.

[0122] The lane line detection device based on structure search provided by the present application adopts the lane line detection method based on structure search in the above embodiment, and can solve the technical problem that the existing lane line detection scheme cannot guarantee both high accuracy and detection speed of the detection result at the same time. Compared with the prior art, the beneficial effects of the lane line detection device based on structure search provided by the present application are the same as those of the lane line detection method based on structure search provided by the above embodiment, and other technical features in the lane line detection device based on structure search are the same as the features disclosed in the method of the above embodiment, and will not be elaborated here.

[0123] In one embodiment, the determination module 20 is further configured to alternately optimize the structure parameters and model parameters in the preset function through the training set to obtain optimized structure parameters and optimized model parameters; obtain the weights corresponding to the operators between the input features and intermediate features in the preset search block structure, and retain the operator with the largest weight; obtain the target preset search block structure according to the optimized structure parameters, optimized model parameters, and the operator with the largest weight.

[0124] In one embodiment, the determination module 20 is further configured to divide the training set into a first subset and a second subset according to a preset ratio; keep the structure parameters in the preset function unchanged, and use the first subset to train and optimize the model parameters; keep the model parameters in the preset function unchanged, and use the second subset to train and optimize the structure parameters; obtain the number of rounds of training optimization, and when the number of rounds is less than or equal to a preset number of rounds, return to execute the step of keeping the structure parameters in the preset function unchanged and using the first subset to train and optimize the model parameters until the number of rounds is greater than the preset number of rounds, to obtain optimized structure parameters and optimized model parameters.

[0125] In one embodiment, the determining module 20 is further configured to perform operator processing on the input features in a preset search block structure to obtain a plurality of intermediate results; perform weighted summation on the plurality of intermediate results to obtain intermediate features; obtain the weights corresponding to the operators between the input features and the intermediate features, and retain the operator with the largest weight.

[0126] In one embodiment, the determining module 20 is further configured to perform convolution, depth convolution, dilated convolution, average pooling, max pooling, and identity mapping processing on the input features in the downsampling search block to obtain a first intermediate result; perform transposed convolution and bilinear interpolation upsampling processing on the input features in the upsampling search block to obtain a second intermediate result; and obtain a plurality of intermediate results according to the first intermediate result and the second intermediate result.

[0127] In one embodiment, the determining module 20 is further configured to obtain a road condition image, and perform anomaly detection on the road condition image; delete the abnormal image when an abnormal image is detected in the road condition image; use the road condition image after the deletion operation as the original data set; and perform image processing on the original data set to obtain a training set.

[0128] In one embodiment, the determining module 20 is further configured to perform image processing such as lane line annotation, scaling, cropping, random flipping, adding Gaussian noise, and normalization on the image data in the original data set to obtain an updated data set; and use the image data of a preset ratio in the updated data set as the training set.

[0129] The present application provides a lane line detection device based on structure search. The lane line detection device based on structure search includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the lane line detection method based on structure search in the first embodiment above.

[0130] Next, refer to Figure 6, which shows a schematic structural diagram of a lane line detection device suitable for implementing the embodiment of the present application based on structural search. The lane line detection device based on structural search in the embodiment of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistant), PADs (Portable Application Description: tablet computers), PMPs (Portable Media Players), in-vehicle terminals (such as in-vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 6 The shown lane line detection device based on structural search is only an example and should not impose any limitations on the functions and usage scope of the embodiments of the present application.

[0131] As Figure 6 shown, the lane line detection device based on structural search may include a processing device 1001 (such as a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM: Read Only Memory) 1002 or the program loaded from the storage device 1003 into the random access memory (RAM: Random Access Memory) 1004. In the RAM 1004, various programs and data required for the operation of the lane line detection device based on structural search are also stored. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other through a bus 1005. The input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems may be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the lane line detection device based on structural search to communicate with other devices wirelessly or wiredly to exchange data. Although the figure shows a lane line detection device with various systems, it should be understood that it is not required to implement or have all the shown systems. Instead, more or fewer systems may be implemented or had.

[0132] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program codes for executing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by a processing device 1001, the above functions defined in the methods of the embodiments disclosed in the present application are executed.

[0133] The lane line detection device based on structural search provided by the present application adopts the lane line detection method based on structural search in the above embodiments, and can solve the technical problem that the existing lane line detection solutions cannot guarantee both high accuracy and detection speed of the detection results at the same time. Compared with the prior art, the beneficial effects of the lane line detection device based on structural search provided by the present application are the same as those of the lane line detection method based on structural search provided in the above embodiments, and other technical features in the lane line detection device based on structural search are the same as the features disclosed in the method of the previous embodiment, and will not be elaborated here.

[0134] It should be understood that each part disclosed in the present application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in a suitable manner in any one or more embodiments or examples.

[0135] As described above, the above are only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present application, and all of them should be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0136] The present application provides a computer-readable storage medium having computer-readable program instructions (i.e., computer programs) stored thereon, and the computer-readable program instructions are used to execute the lane line detection method based on structural search in the above embodiments.

[0137] The computer-readable storage medium provided by the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or components, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) or flash memory, optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0138] The above computer-readable storage medium may be included in a lane line detection device based on structural search; or may exist separately without being assembled into a lane line detection device based on structural search.

[0139] The above computer-readable storage medium carries one or more programs, which, when executed by a lane line detection device based on structural search, cause the lane line detection device based on structural search to: obtain a road condition image, preprocess the road condition image to obtain a training set; based on a preset search block structure and a preset function, obtain a target preset search block structure through the training set; establish a lane line detection model through the target preset search block structure; and complete lane line detection through the lane line detection model.

[0140] Computer program code for performing the operations of this application can be written in one or more programming languages or combinations thereof. The above-mentioned programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any kind of network, including a local area network (LAN: Local Area Network) or a wide area network (WAN: Wide Area Network), or it can be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).

[0141] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in the flowchart or block diagram can represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0142] The modules described in the embodiments of this application can be implemented in software or in hardware. Among them, the name of the module does not constitute a limitation to the unit itself in some cases.

[0143] The readable storage medium provided by this application is a computer-readable storage medium. The computer-readable storage medium stores computer-readable program instructions (i.e., computer programs) for performing the above-mentioned lane line detection method based on structural search, and can solve the technical problem that the existing lane line detection solutions cannot ensure both high accuracy and detection speed of the detection results at the same time. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided by this application are the same as those of the lane line detection method based on structural search provided by the above embodiments, and will not be elaborated here.

[0144] The present application also provides a computer program product, including a computer program, which implements the steps of the lane line detection method based on structural search as described above when executed by a processor.

[0145] The computer program product provided by the present application can solve the technical problem that the existing lane line detection solutions cannot guarantee both high accuracy and detection speed of the detection results at the same time. Compared with the prior art, the beneficial effects of the computer program product provided by the present application are the same as those of the lane line detection method based on structural search provided by the above embodiments, and will not be elaborated here.

[0146] The above are only partial embodiments of the present application, and thus do not limit the patent scope of the present application. Any equivalent structural transformation made under the technical concept of the present application by using the content of the specification and drawings of the present application, or direct / indirect application in other related technical fields, is included in the patent protection scope of the present application.

Claims

1. A lane line detection method based on structure search, characterized in that: The method comprises: Acquire a road condition image, and preprocess the road condition image to obtain a training set; Based on the preset search block structure and the preset function, obtaining a target preset search block structure through the training set; A lane detection model is established through a target preset search block structure; Lane line detection is completed through the lane line detection model.

2. The method according to claim 1, characterized in that The step of obtaining a target preset search block structure through the training set based on the preset search block structure and the preset function comprises: Alternately optimizing the structural parameters and the model parameters in the preset function through the training set to obtain optimized structural parameters and optimized model parameters; Obtain the weights corresponding to the operators between the input features and the intermediate features in the preset search block structure, and retain the operator with the largest weight; The target preset search block structure is obtained according to the optimized structural parameters, the optimized model parameters and the operator with the largest weight.

3. The method according to claim 2, characterized in that The step of alternately optimizing the structural parameters and the model parameters in the preset function through the training set to obtain the optimized structural parameters and the optimized model parameters comprises: Dividing the training set into a first subset and a second subset according to a preset ratio; Keep the structural parameters in the preset function unchanged, and use the first subset to train and optimize the model parameters; Keep the model parameters in the preset function unchanged, and use the second subset to train and optimize the structural parameters; Obtain a round of training optimization. When the round is less than or equal to a preset round, return to execute the step of keeping the structural parameters in the preset function unchanged and using the first subset to perform training optimization on the model parameters until the round is greater than the preset round, and obtain optimized structural parameters and optimized model parameters.

4. The method according to claim 2, characterized in that The step of obtaining the weights corresponding to the operators between the input features and the intermediate features in the preset search block structure and retaining the operator with the largest weight comprises: In the preset search block structure, the input features are processed by operators to obtain several intermediate results; Performing weighted summation on the intermediate results to obtain intermediate features; Obtain the weights corresponding to the operators between the input feature and the intermediate feature, and retain the operator with the largest weight.

5. The method according to claim 4, characterized in that The preset search block structure includes at least: a down-sampling search block and an up-sampling search block; In the preset search block structure, the step of performing operator processing on the input features to obtain a number of intermediate results includes: In the downsampling search block, convolution, depth convolution, dilated convolution, average pooling, maximum pooling, and identity mapping are performed on the input features to obtain a first intermediate result; In the upsampling search block, deconvolution and bilinear difference upsampling are performed on the input features to obtain a second intermediate result; Several intermediate results are obtained according to the first intermediate result and the second intermediate result.

6. The method according to claim 1, characterized in that The step of acquiring a road condition image and preprocessing the road condition image to obtain a training set comprises: Acquire a road condition image, and perform anomaly detection on the road condition image; When detecting that an abnormal image exists in the road condition image, deleting the abnormal image; The road condition image after the deletion operation is performed is used as the original data set; Perform image processing on the original data set to obtain a training set.

7. The method according to claim 6, characterized in that The step of performing image processing on the original data set to obtain a training set comprises: Performing lane line marking, scaling, cropping, random flipping, adding Gaussian noise, and normalizing image processing on the image data in the original data set to obtain an updated data set; The image data with a preset ratio in the updated data set is used as a training set.

8. A lane line detection device based on structure search, characterized in that: The lane line detection device based on structure search includes: An acquisition module, used for acquiring road condition images, preprocessing the road condition images, and obtaining a training set; A determination module, configured to obtain a target preset search block structure through the training set based on a preset search block structure and a preset function; The determination module is further used to establish a lane line detection model through a target preset search block structure; The detection module is used to complete the lane line detection through the lane line detection model.

9. A lane line detection device based on structure search, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the lane line detection method based on structure search as described in any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the lane line detection method based on structure search as described in any one of claims 1 to 7 are implemented.

Images