Method for constructing lane detection model and lane line detection method in complex scenes

Abstract

The present disclosure relates to a method for constructing a lane detection model in a complex scene, including: collecting a video or image containing a lane in a real scene, randomly dividing the lane image into a training set and a verification set, and converting the normal view of the lane image into a top view; obtaining lane scene information from the top view of the lane image, calculating the illumination intensity and image blur of the lane image and comparing them with corresponding predetermined thresholds respectively, and using an image enhancement algorithm and an image deblurring algorithm to perform image preprocessing on the lane image according to the comparison results; annotating the top view and the normal view of the lane image to establish a corresponding label file; and training the dual feature representation of the normal view and the top view of the lane image as the input of the YOLOv3 network model to obtain a lane detection model in a complex scene. The present disclosure also relates to a method for detecting a lane image using the constructed lane detection model in a complex scene to locate the specific position of the lane line.

Description

Technical Field

[0001] The present disclosure generally relates to the fields of artificial intelligence and computer vision, and more specifically to lane line detection in smart transportation using artificial intelligence and computer vision technologies. Background Art

[0002] As a hot topic pursued by the technology industry, smart transportation is a wonderful product of the "Internet +" and artificial intelligence era. At present, smart transportation is mainly restricted by the urban traffic pattern and the construction of laws and regulations. At the same time, there are still huge challenges in terms of algorithm accuracy and real-time performance, multi-sensor costs, etc. The rise of 5G technology has brought new opportunities to the field of smart transportation. Thanks to the robust, fast, high-bandwidth and low-latency characteristics of 5G technology, people can wirelessly transmit traffic data collected by on-board sensors to the cloud and build massive traffic data centers to promote the development of intelligent driving technology. Moreover, algorithm models can be used in the cloud for data processing and feedback to improve the degree of automation and safety of driving.

[0003] Lane line detection is a key part of the smart transportation field, and is crucial for lane departure warning, lane keeping, path planning, etc. Due to the low investment cost, large amount of information obtained, and wide range of action of visual sensors, the existing technology mainly uses traditional computer vision algorithms and deep learning technologies to extract lane lines or lane areas. However, these technologies are not robust in bad weather, complex lighting scenes, etc. Although deep learning technology can improve the robustness of the algorithm by increasing training data, due to the complex environmental background and unstable lighting conditions in real scenes, the training data required for deep learning technology is of an order of magnitude that is basically impossible to obtain. Especially for the case of detecting the main lane of the vehicle's driving direction, if deep learning technology is directly used to identify the main lane in a real environment, it is difficult to achieve a high accuracy rate.

[0004] In order to solve the problem of weak robustness of lane line detection in complex scenarios, the present disclosure provides an innovative method for constructing a lane detection model in complex scenarios, and a method for detecting lane lines in complex scenarios using the constructed lane detection model, so as to improve the inference capability of the lane detection network in complex lane scenarios. Summary of the invention

[0005] A brief overview of the disclosure is given below in order to provide a basic understanding of some aspects of the disclosure. However, it should be understood that this overview is not an exhaustive overview of the disclosure. It is not intended to identify the key or important parts of the disclosure, nor is it intended to limit the scope of the disclosure. Its purpose is simply to give some concepts of the disclosure in a simplified form as a prelude to a more detailed description given later.

[0006] According to one aspect of the present disclosure, a method for constructing a lane detection model in a complex scene is provided, comprising the following steps: collecting a video or image containing a lane in a real scene, randomly dividing the lane image into a training set and a validation set, and converting a normal view of the lane image into a top view; acquiring lane scene information from the top view of the lane image, calculating the illumination intensity of the lane image and comparing it with a predetermined illumination intensity upper threshold t1 and lower threshold t2, and calculating the lane image blur and comparing it with a predetermined blur threshold t; if the lane image illumination intensity is greater than the upper threshold t1 or less than the lower threshold t2, using an image enhancement algorithm to perform image preprocessing on the lane image; if the lane image blur is greater than the predetermined blur threshold t, using an image deblurring algorithm to perform image preprocessing on the lane image; annotating the top view and the normal view of the lane image to establish a corresponding label file; and constructing a YOLOv3 network model, using the dual feature representation of the normal view and the top view of the lane image as input to train the YOLOv3 network model, so as to obtain a lane detection model in a complex scene.

[0007] According to another aspect of the present disclosure, a method for detecting lane lines using a lane detection model is provided, comprising the following steps: acquiring a video or image containing a lane in a real scene, and converting a normal view of the lane image into a top view; acquiring lane scene information from the top view of the lane image, calculating the illumination intensity of the lane image and comparing it with a predetermined illumination intensity upper threshold t1 and lower threshold t2, and calculating the lane image blur and comparing it with a predetermined blur threshold t; if the lane image illumination intensity is greater than the upper threshold t1 or less than the lower threshold t2, using an image enhancement algorithm to perform image preprocessing on the lane image; if the lane image blur is greater than the predetermined blur threshold t, using an image deblurring algorithm to perform image preprocessing on the lane image; and inputting the top view and the normal view of the lane image into the lane detection model constructed by the method described above for detection to locate the specific position of the lane line.

[0008] According to another aspect of the present disclosure, a computing device is provided, including a memory on which instructions are stored; and a processor configured to execute the instructions stored on the memory to perform the method according to the above aspects of the present disclosure.

[0009] According to another aspect of the present disclosure, a computer-readable storage medium is provided, which includes computer-executable instructions. When the computer-executable instructions are executed by one or more processors, the one or more processors execute the method according to the above aspects of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The accompanying drawings, which constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0011] The present disclosure may be more clearly understood from the following detailed description with reference to the accompanying drawings, in which:

[0012] Figure 1 A flow chart of a method for constructing a lane detection model in a complex scene according to an embodiment of the present disclosure is shown;

[0013] Figure 2 A YOLOv3 network model structure diagram according to an embodiment of the present disclosure is shown;

[0014] Figure 3 A flowchart of creating and training a YOLOv3 network model according to an embodiment of the present disclosure is shown;

[0015] Figure 4 A method for detecting lane lines using a lane detection model in a complex scene according to an embodiment of the present disclosure is shown;

[0016] Figure 5 shows a top view of a lane image according to an embodiment of the present disclosure;

[0017] Figure 6 A schematic diagram showing a lane image after enhancement according to an embodiment of the present disclosure is shown;

[0018] Figure 7 A schematic diagram showing a lane image after image deblurring processing according to an embodiment of the present disclosure is shown;

[0019] Figure 8 A schematic diagram showing a lane image after being detected by a lane detection model according to an embodiment of the present disclosure; and

[0020] Fig. 9 An exemplary configuration of a computing device that can implement an embodiment according to the present disclosure is shown. DETAILED DESCRIPTION

[0021] The following detailed description is made with reference to the accompanying drawings, and the following detailed description is provided to assist in a comprehensive understanding of the various exemplary embodiments of the present disclosure. The following description includes various details to assist in understanding, but these details are considered to be examples only and not to limit the present disclosure, which is defined by the appended claims and their equivalents. The words and phrases used in the following description are only used to enable a clear and consistent understanding of the present disclosure. In addition, for the sake of clarity and brevity, descriptions of well-known structures, functions, and configurations may be omitted. Those of ordinary skill in the art will recognize that various changes and modifications may be made to the examples described herein without departing from the spirit and scope of the present disclosure.

[0022] Figure 1 A flowchart of a method for constructing a lane detection model in a complex scene according to an embodiment of the present disclosure is shown.

[0023] like Figure 1 As shown in step 101 in , first, collect videos or images containing lanes in real scenes, expand the data set by horizontal flipping, scale transformation, adding noise, etc., and randomly divide the images into training sets and validation sets according to a certain ratio to serve as inputs of the lane detection model. The training set of images is used as input to create and train the lane detection model, and the validation set of images is used as input to verify the performance of the trained lane detection model.

[0024] Since the lane lines in the top view are almost parallel and equal in width, interference information is greatly reduced, making it easier to accurately extract the lane lines. The present invention converts the normal view of the lane image into a top view based on the perspective principle, and then uses the dual feature representation of the normal view and the top view of the lane image as the input of the deep learning network, i.e., the lane detection model.

[0025] In step 102, lane scene information, such as the illumination intensity and blur of the lane image, is obtained from the top view of the lane image as a basis for determining the lane scene. In the construction, training and detection stages of the lane detection model, the lane scene information can be directly used.

[0026] More specifically, in order to obtain the illumination intensity attribute of the lane image, the illumination intensity of the lane image is calculated. First, the lane image is converted into a grayscale image, where g(x, y) represents the grayscale value of the pixel point (x, y), and the mean M and variance V of the image are calculated according to the following formula:

[0027]

[0028]

[0029] Among them, m0 is the mean of the initialized image, w and h represent the width and height of the image respectively, and Hist(i) is the histogram of the grayscale image.

[0030] Define the upper and lower thresholds t1 and t2 of the light intensity, and compare the lane image light intensity with them respectively to extract two light intensities, 1-strong light and 0-dim light, from the lane image light intensity:

[0031]

[0032] In order to obtain the blur attribute of the lane image, the blur of the lane image is calculated. First, the image is converted into a grayscale image. In order to adapt to the angle of the lane line in the image, the sobel edge detection operator in the direction of 45° and 135° is used to perform convolution operation with the grayscale image to obtain the edge image edge(x, y):

[0033]

[0034] Next, calculate the variance of the edge image. If the variance is small, it means that the frequency response range of the image is narrow, which means that there are fewer edges in the image, that is, the lane image is more blurred. Define a predetermined blur threshold t and compare the lane image blur with it. If the image blur is lower than the threshold t, the image blur is set to 1, otherwise it is set to 0.

[0035]

[0036] In step 103, it is determined whether the acquired lane image is a complex scene that requires image preprocessing. If the lane image illumination intensity is greater than the upper threshold t1 or less than the lower threshold t2, that is, the image illumination intensity is 1 or 0, an image enhancement algorithm is used for image preprocessing; if the image blur is lower than the threshold t, that is, the blur is 1, an image deblurring algorithm is used for image preprocessing.

[0037] If it is determined that the acquired lane image is a complex scene, then in step 104 , image preprocessing is performed on the lane image according to the acquired lane scene information.

[0038] In one embodiment, according to the image illumination intensity, an image enhancement algorithm is required to perform image preprocessing on the lane image. First, each pixel in the image is normalized to the interval [0-1], and then a product operation is performed. In order to enhance the lane image with strong light and dim illumination intensity, the present disclosure defines the following formula based on the mean value within the pixel field to stretch or compress the area with too high or too low grayscale:

[0039] O(x, y) = I(x, y) γ·m

[0040] Among them, I(x, y) represents the input image, O(x, y) is the corrected image, m is the mean in the field of I(x, y)n, and γ is the correction parameter.

[0041] In one embodiment, according to the image light blur, an image deblurring algorithm needs to be used to preprocess the lane image. Image blur is generally regarded as a process of obtaining a blurred image by convolution of a clear image with a blur kernel, and its degradation model is shown in the following formula:

[0042]

[0043] Among them, B(x, y) is the blurred image, I(x, y) is the original image, K is the blur kernel, and N is the additional noise.

[0044] The image deblurring algorithm is an inverse filtering process. The present invention first initializes the Laplacian operator as the blur kernel and restores the image, and then re-estimates the blur kernel to restore the image until it finally converges.

[0045] Next, in step 105, the lane image is labeled. The preprocessed top view and normal view are labeled using the labelme image labeling tool to create a corresponding label file. The labeling information included in the label file includes the center point coordinates (x, y) of the bounding box where each target is located, the width w and height h of the bounding box, and the category information class. i , i=0 represents the background, i=1 represents the target lane line.

[0046] It is worth noting that if it is determined in the aforementioned step S103 that the acquired lane scene does not require image preprocessing, then the process directly proceeds to step 105 to annotate the lane image.

[0047] Then, proceed to step S106 to train the lane detection model. The present invention uses the YOLOv3 network as the lane detection model for lane detection. First, the YOLOv3 network is constructed, the weight strategy is initialized, and then the training data set is input for model training. In this embodiment, the normal view and the top view of the lane image after annotation are used as the input of the YOLOv3 network. This dual feature representation method can improve the inference ability of the network in complex lane scenes.

[0048] Figure 2 The figure shows the structure of the YOLOv3 network model. The normal view and top view of the lane image are input into the darknet-53 network. After deep convolution, the dimensions are reduced to feature maps of three different scales: 16×16, 32×32, and 64×64. Then, after three full convolution feature extractors, upsampling, and concatenation operations, feature maps of three different scales are output. The YOLOv3 detection layer predicts the input image on the feature maps of three different scales and outputs the prediction results.

[0049] Figure 3 A flowchart for creating and training a YOLOv3 network model is shown, including the following steps:

[0050] Feature extraction step S301: The YOLOv3 network model uses the Darknet-53 network to implement feature extraction. Darknet-53 mainly has 53 layers of convolution and makes extensive use of skip connections similar to those in residual networks. At the same time, in order to reduce the negative gradient effects caused by pooling, a convolution with a step size of 2 is used for downsampling.

[0051] In one embodiment, according to actual needs, after deep convolution is performed on the input image, the dimensions are reduced to feature maps of three different scales: 16×16, 32×32, and 64×64; then, three full convolution feature extractors, upsampling, and splicing operations are performed respectively to output feature maps of three different scales.

[0052] Network prediction step S302: The YOLOv3 detection layer predicts the input image on feature maps of three different scales. At each scale, the image is divided into 8×8 grids, and each grid predicts a conditional probability value for each category (i.e., the probability that the object in the bounding box belongs to the target lane line or the background) and three bounding boxes. Each bounding box predicts five regression values, including the coordinates of the center point of the bounding box, the width and height of the bounding box, the probability that the bounding box contains the target object, and the accuracy of the bounding box position (i.e., IoU).

[0053] Non-maximum suppression processing step S303: Since there are many output bounding boxes, it is necessary to define the confidence and set the confidence threshold, and use non-maximum suppression (Non-Maximum Suppression) to filter and obtain the final prediction box.

[0054] The confidence consists of the probability that the bounding box contains the target object and the accuracy of the bounding box position, which is defined as:

[0055]

[0056] Among them, P(object)∈(0,1) is the probability that the bounding box contains the lane line; It is the ratio of the area of ​​the overlap between the predicted bounding box and the annotated bounding box to the area of ​​their union.

[0057] Adjust the network configuration parameters step S304: Use the mean square and error of the predicted bounding box information and the manually labeled information as the loss function, back-propagate the error between the true value and the predicted value to the network, repeat steps S303-S304, and adjust the network configuration parameters to determine the optimal network, that is, the required lane detection network.

[0058] Optionally, after the lane detection model is trained, you can execute Figure 1The trained lane detection model is verified in step S107. The lane images of the real scene collected in step S101 and assigned to the verification set are used as inputs of the lane detection model, and the performance of the trained YOLO v3 network is verified using precision and recall.

[0059] The precision and recall are calculated according to the following formula:

[0060]

[0061]

[0062] If the precision and recall are lower than the predetermined threshold, then new training images are added to the training set, and steps S102 to S106 of the above-mentioned training lane detection model are re-executed until the recall and precision are higher than the predetermined threshold.

[0063] In another embodiment, a successfully verified lane detection model is used to detect the lane image to locate the specific position of the lane line.

[0064] like Figure 4 As shown in step 401 in the example, first, a video or image containing a lane in a real scene is collected. The normal view of the lane image is converted into a top view according to the perspective principle, so that the dual feature representation of the normal view and the top view of the lane image is used as the input of the lane detection model.

[0065] In step 402, the same operation as in the aforementioned step 102 is followed to obtain properties of the lane image such as illumination intensity and blur from the overhead view of the lane image.

[0066] In step 403, the same operation as in the aforementioned step 103 is followed to determine whether the acquired lane scene is a complex scene that requires image preprocessing. If the lane image illumination intensity is greater than the upper threshold t1 or less than the lower threshold t2, that is, the image illumination intensity is 1 or 0, an image enhancement algorithm is used for image preprocessing; if the image blur is lower than the threshold t, that is, the blur is 1, an image deblurring algorithm is used for image preprocessing.

[0067] If it is determined that the acquired lane scene is a complex scene, then in step 404, the lane image is preprocessed according to the acquired lane scene information in the same manner as in the aforementioned step 104.

[0068] Finally, in step S405, the top view and the normal view of the lane image are input into the successfully verified lane detection model for detection to locate the specific position of the lane line.

[0069] If it is determined in the aforementioned step S403 that the acquired lane scene does not require image preprocessing, then the process directly proceeds to step 405 to detect the lane image.

[0070] Figure 5 A schematic diagram showing a night lane image after perspective transformation processing according to an embodiment of the present disclosure is shown. Figure 6 A schematic diagram showing a lane image after image enhancement processing according to an embodiment of the present disclosure is shown. Figure 7 A schematic diagram showing a lane image after image deblurring processing according to an embodiment of the present disclosure is shown. Figure 8 It shows that the night lane image is input into the finally trained lane detection model to obtain the lane detection result, so that the specific position of the lane line can be located.

[0071] The present disclosure identifies complex scenes by analyzing the attribute information of the collected lane images, performs image preprocessing for different complex scenes, and then uses the normal view and top view dual feature expressions of the lane images to train the lane detection model, which can build an improved lane detection model for complex scenes. The present disclosure is more feasible than the deep learning method that directly identifies lane lines in complex environments, and requires fewer training sets. In addition, the constructed improved lane detection model can locate the specific position of the lane line in complex scenes, is more robust, and improves the inference ability of the lane detection network in harsh and complex scenes.

[0072] The present disclosure can be applied to road status detection, high-precision map drawing, unmanned driving and other fields, providing strong technical support for the development of intelligent assisted driving and the construction of intelligent transportation.

[0073] Fig. 9 An exemplary configuration of a computing device 900 capable of implementing embodiments according to the present disclosure is shown.

[0074] The computing device 900 is an example of a hardware device to which the above aspects of the present disclosure can be applied. The computing device 900 can be any machine configured to perform processing and / or calculations. The computing device 900 can be, but is not limited to, a workstation, a server, a desktop computer, a laptop computer, a tablet computer, a personal data assistant (PDA), a smart phone, a vehicle-mounted computer, or a combination thereof.

[0075] like Fig. 9As shown, computing device 900 may include one or more elements that can be connected or communicated with bus 902 via one or more interfaces. Bus 902 may include, but is not limited to, Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, Video Electronics Standards Association (VESA) local bus, and Peripheral Component Interconnect (PCI) bus, etc. Computing device 900 may include, for example, one or more processors 904, one or more input devices 906, and one or more output devices 908. One or more processors 904 may be any kind of processor, and may include, but is not limited to, one or more general-purpose processors or special-purpose processors (such as special-purpose processing chips). Input device 906 may be any type of input device that can input information to computing device, and may include, but is not limited to, mouse, keyboard, touch screen, microphone and / or remote controller. Output device 908 may be any type of device that can present information, and may include, but is not limited to, display, speaker, video / audio output terminal, vibrator and / or printer.

[0076] The computing device 900 may also include or be connected to a non-transitory storage device 914, which may be any non-transitory storage device that can implement data storage, and may include but is not limited to a disk drive, an optical storage device, a solid-state memory, a floppy disk, a flexible disk, a hard disk, a tape or any other magnetic medium, a compressed disk or any other optical medium, a cache memory and / or any other storage chip or module, and / or any other medium from which a computer can read data, instructions and / or code. The computing device 900 may also include a random access memory (RAM) 910 and a read-only memory (ROM) 912. The ROM 912 may store programs, utilities or processes to be executed in a non-volatile manner. The RAM 910 may provide volatile data storage and store instructions related to the operation of the computing device 900. The computing device 900 may also include a network / bus interface 916 coupled to a data link 918. The network / bus interface 916 may be any type of device or system capable of enabling communication with an external device and / or a network, and may include but is not limited to a modem, a network card, an infrared communication device, a wireless communication device, and / or a chipset (such as a Bluetooth TM equipment, 802.11 equipment, WiFi equipment, WiMax equipment, cellular communication facilities, etc.).

[0077] The present disclosure may be implemented as any combination of an apparatus, a system, an integrated circuit, and a computer program on a non-transitory computer-readable medium. One or more processors may be implemented as an integrated circuit (IC), an application specific integrated circuit (ASIC), or a large scale integrated circuit (LSI), a system LSI, a super LSI, or a super LSI component that performs some or all of the functions described in the present disclosure.

[0078] The present disclosure includes the use of software, applications, computer programs, or algorithms. The software, applications, computer programs, or algorithms may be stored on a non-transitory computer-readable medium to enable a computer such as one or more processors to perform the above steps and the steps described in the accompanying drawings. For example, one or more memories store the software or algorithm in executable instructions, and one or more processors may associate a set of instructions to execute the software or algorithm to provide various functions according to the embodiments described in the present disclosure.

[0079] Software and computer programs (which may also be referred to as programs, software applications, applications, components or code) include machine instructions for a programmable processor and may be implemented in a high-level procedural language, an object-oriented programming language, a functional programming language, a logic programming language, or an assembly language or a machine language. The term "computer-readable medium" refers to any computer program product, apparatus or device for providing machine instructions or data to a programmable data processor, such as magnetic disks, optical disks, solid-state storage devices, memories and programmable logic devices (PLDs), including computer-readable media that receive machine instructions as computer-readable signals.

[0080] For example, computer readable media may include dynamic random access memory (DRAM), random access memory (RAM), read-only memory (ROM), electrically erasable read-only memory (EEPROM), compact disk read-only memory (CD-ROM) or other optical disk storage devices, magnetic disk storage devices or other magnetic storage devices, or any other medium that can be used to carry or store the desired computer readable program code in the form of instructions or data structures and can be accessed by a general or special purpose computer or a general or special purpose processor. As used herein, disk or disc includes compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc, where disks usually reproduce data magnetically, while discs reproduce data optically by laser. Combinations of the above are also included in the scope of computer readable media.

[0081] The subject matter of the present disclosure is provided as an example of devices, systems, methods and programs for performing the features described in the present disclosure. However, in addition to the above-mentioned features, other features or variations may also be expected. It is expected that the implementation of the components and functions of the present disclosure may be completed with any emerging technology that may replace any of the above-mentioned implementation technologies.

[0082] In addition, the above description provides examples without limiting the scope, applicability or configuration set forth in the claims. Without departing from the spirit and scope of the present disclosure, the functions and arrangements of the elements discussed may be changed. Various embodiments may appropriately omit, replace or add various processes or components. For example, features described with respect to certain embodiments may be combined in other embodiments.

[0083] In addition, in the description of the present disclosure, the terms “first”, “second”, “third”, etc. are used for descriptive purposes only and cannot be understood as indicating or implying relative importance and order.

[0084] Similarly, while operations are depicted in a particular order in the drawings, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed to achieve desired results. In certain circumstances, multitasking and parallel processing can be advantageous.

Claims

1. A method for constructing a lane detection model in a complex scene, comprising the following steps: Collect videos or images containing lanes in real scenes, randomly divide the lane images into training sets and validation sets, and convert the normal view of the lane images into a bird's-eye view; Acquire lane scene information from a top view of the lane image, calculate the illumination intensity of the lane image and compare it with a predetermined illumination intensity upper threshold t1 and a predetermined illumination intensity lower threshold t2, and calculate the blurriness of the lane image and compare it with a predetermined blurriness threshold t; If the lane image illumination intensity is greater than the upper threshold t1 or less than the lower threshold t2, the image enhancement algorithm is used to preprocess the lane image; If the lane image blur is greater than a predetermined blur threshold t, an image deblurring algorithm is used to perform image preprocessing on the lane image; Annotate the top view and normal view of the lane image to create a corresponding label file; as well as Build a YOLOv3 network model, and use the dual feature representations of the normal view and top view of the lane image as input for training to obtain a lane detection model in complex scenarios. The calculation of the illumination intensity of the lane image and the comparison with the predetermined illumination intensity upper threshold t1 and lower threshold t2 include the following steps: The lane image is converted into a grayscale image, where g(x,y) represents the grayscale value of the pixel (x,y). The mean M and variance V of the grayscale image are calculated according to the following formula: Where m0 is the mean of the initialized grayscale image, w and h represent the width and height of the grayscale image respectively, and Hist(i) is the histogram of the grayscale image; as well as Define the upper and lower thresholds t1 and t2 of the light intensity, and compare the lane image light intensity with them respectively to extract two light intensities, 1-strong light and 0-dim light, from the lane image light intensity:

2. The method according to claim 1, wherein if the lane image illumination intensity is greater than an upper threshold value t1 or less than a lower threshold value t2, then using an image enhancement algorithm to perform image preprocessing on the lane image comprises the following steps: Normalize each pixel in the image to the [0-1] interval and perform a product operation; The lane image with enhanced illumination intensity is determined as 1- bright light and 0- dim light. Based on the mean value in the pixel area, the following formula is defined to stretch or compress the areas with too high or too low gray levels: O(x,y)=I(x,y) γ·m in, I(x,y) represents the input image, O(x,y) is the corrected image, m is the mean in the domain of I(x,y)n, and γ is the correction parameter.

3. The method according to claim 1, wherein calculating the lane image blurriness and comparing it with a predetermined blurriness threshold t comprises the following steps: The lane image is converted into a grayscale image. In order to adapt to the angle of the lane line in the image, the sobel edge detection operator in the 45° and 135° directions is used to perform convolution operations with the grayscale image to obtain the edge image edge(x,y): Calculate the variance of the edge image. The smaller the variance, the more blurred the lane image is; and Define a predetermined blur threshold t and compare the lane image blur with it. If the image blur is lower than the threshold t, the image blur is set to 1, otherwise it is set to 0.

4. The method according to claim 3, wherein if the blurriness of the lane image is greater than a predetermined blurriness threshold t, using an image deblurring algorithm to perform image preprocessing on the lane image comprises the following steps: The inverse process of the image blur algorithm is performed on the image, where image blur is the process of convolving a clear image with a blur kernel to obtain a blurred image. Its degradation model is shown in the following formula: in, B(x, y) is the blurred image, I(x, y) is the original image, K is the blur kernel, and N is the additional noise; as well as The image deblurring algorithm is an inverse filtering process, which initializes the Laplacian operator as the blur kernel and restores the image, and then re-estimates the blur kernel to restore the image until it finally converges.

5. The method according to any one of claims 1 to 4, wherein training the dual feature representation of the normal view and the top view of the lane image as the input of the YOLOv3 network model comprises the following steps: The feature extraction step S301 includes: The YOLOv3 network model uses the Darknet-53 network to achieve feature extraction. Darknet-53 includes 53 layers of convolution and uses convolution with a step size of 2 for downsampling. After deep convolution of the input image, the dimensions are reduced to feature maps of three different scales: 16×16, 32×32, and 64×64; as well as The feature maps of three different scales, 16×16, 32×32, and 64×64, are respectively subjected to three fully convolutional feature extractors, up-sampled, and concatenated to output three feature maps of different scales; The network prediction step S302 includes: The YOLOv3 detection layer predicts the input image on feature maps of three different scales. At each scale, the image is divided into 8×8 grids. Each grid predicts a conditional probability value and three bounding boxes for each category. Each bounding box predicts five regression values, including the coordinates of the center point of the bounding box, the width and height of the bounding box, the probability that the bounding box contains the target object, and the accuracy of the bounding box position. The non-maximum suppression processing step S303 includes: Define the confidence level and set the confidence threshold, and use non-maximum suppression filtering to get the final prediction box; The confidence consists of the probability that the bounding box contains the target object and the accuracy of the bounding box position, which is defined as: Among them, P(object)∈(0,1) is the probability that the bounding box contains the lane line; is the ratio of the area of ​​the overlap between the predicted bounding box and the annotated bounding box to the area of ​​the union of the two; and The step S304 of adjusting network configuration parameters includes: Using the mean square sum error of the predicted bounding box information and the manually labeled information as the loss function, the error between the true value and the predicted value is back-propagated into the network, and steps S303-S304 are repeated to adjust the network configuration parameters to determine the optimal lane detection network.

6. The method according to any one of claims 1 to 4, further comprising the steps of: Input the lane images of the validation set into the trained lane detection model to validate it; The precision and recall are calculated according to the following formula: If the precision and recall are lower than the predetermined threshold, a new lane image is added to the training set, and the above steps are repeated until the recall and precision are higher than the predetermined threshold.

7. The method according to any one of claims 1 to 4, further comprising expanding the lane image data set by horizontal flipping, scale transformation, and noise addition methods. 8 . The method according to claim 1 , wherein the lane images are randomly divided into a training set and a validation set according to a predetermined ratio.

9. A method for detecting lane lines using a lane detection model, comprising the following steps: Collect videos or images containing lanes in real scenes, and convert the normal view of the lane image into a bird's-eye view; Acquire lane scene information from a top view of the lane image, calculate the illumination intensity of the lane image and compare it with a predetermined illumination intensity upper threshold t1 and a predetermined illumination intensity lower threshold t2, and calculate the blurriness of the lane image and compare it with a predetermined blurriness threshold t; If the lane image illumination intensity is greater than the upper threshold t1 or less than the lower threshold t2, the image enhancement algorithm is used to preprocess the lane image; If the lane image blur is greater than a predetermined blur threshold t, an image deblurring algorithm is used to perform image preprocessing on the lane image; as well as The top view and the normal view of the lane image are input into the lane detection model constructed by the method according to any one of claims 1 to 8 for detection, so as to locate the specific position of the lane line.

10. A computing device comprising: a memory having instructions stored therein; as well as A processor configured to execute instructions stored on the memory to perform the method according to any one of claims 1-9.

11. A computer-readable storage medium comprising computer-executable instructions, which, when executed by one or more processors, cause the one or more processors to perform the method according to any one of claims 1-9.

Images