Model training method, lane line detection method and system

Abstract

The invention discloses a model training method and a lane line detection method and system, and the method comprises the steps: collecting a road scene image, carrying out the marking of the road scene image, obtaining a marked image data set, obtaining label data from the marked image data set, and carrying out the marking of the label data; the label data is a data file formed by marking a lane line area in a scene image of vehicle driving, image shallow detail information is extracted from the label data through detail branches, the image shallow detail information is information of a lane line position, the semantic branches of the semantic segmentation network are optimized, and the image shallow detail information is obtained. And extracting deep semantic information from a scene image of vehicle driving through the optimized semantic branch, supervising image shallow detail information output by the detail branch and the deep semantic information output by the semantic branch through a preset function, completing training of a lane line detection model, and detecting a to-be-detected image through the lane line detection model. And obtaining a segmentation image, and realizing unmanned driving through the segmentation image.

Description

Technical Field

[0001] The present application relates to the technical field of driverless, and more specifically, to a model training method, a lane line detection method and a system. Background Art

[0002] In the existing lane line detection methods in the field of autonomous driving, most use a lane line network model to detect whether each pixel in an image is on a lane line.

[0003] In the lane line network model, in order to reduce the number of model parameters and complexity, a lightweight network is used as the backbone network backbone. The network layers of the backbone network backbone mostly use small-sized convolutions to meet the real-time perception needs of autonomous driving.

[0004] Although the backbone network backbone has few parameters and fast detection speed, it sacrifices the detection effect and has poor detection effect under occlusion and extreme lighting conditions; secondly, the small-sized convolutions of such networks have a small receptive field, which is not conducive to lane line detection.

[0005] Therefore, how to improve the accuracy of lane line detection to achieve driverless is an urgent problem to be solved in the present application. Summary of the Invention

[0006] In view of this, the present application discloses a model training method, a lane line detection method and a system, aiming to improve the lane line detection effect and accuracy to achieve driverless.

[0007] To achieve the above object, the disclosed technical solutions are as follows:

[0008] The first aspect of the present application discloses a model training method, and the model training method includes:

[0009] Collect road scene images;

[0010] Annotate the road scene images to obtain an annotated image data set, and obtain label data from the annotated image data set; the label data is a data file formed by annotating the lane line area in the scene image of vehicle driving;

[0011] Extract image shallow layer detail information from the label data through a detail branch; the image shallow layer detail information is the information of the lane line position;

[0012] Optimize the semantic branch of the semantic segmentation network, and extract deep semantic information from the scene image of vehicle driving through the optimized semantic branch;

[0013] Supervise the shallow image detail information output by the detail branch and the deep semantic information output by the semantic branch through a preset function to complete the training of the lane line detection model.

[0014] Preferably, obtaining label data from the labeled image dataset includes:

[0015] Read the scene image of vehicle driving from the labeled image dataset, and obtain the label data corresponding to the scene image of vehicle driving.

[0016] Preferably, the process of optimizing the semantic branch of the semantic segmentation network includes:

[0017] Extract features from the feature maps of the network layers input to the semantic segmentation network to obtain feature maps of various preset types;

[0018] Fuse the feature maps of various preset types to enhance the extraction of lane line information.

[0019] Preferably, the extracting features from the feature maps of the network layers input to the semantic segmentation network to obtain feature maps of various preset types includes:

[0020] Through multi-scale convolution optimization with strip-shaped convolution, perform convolution operations on the feature maps of the network layers input to the semantic segmentation network for a preset number of times to obtain feature maps with various preset convolution kernel sizes.

[0021] Preferably, the process of optimizing the semantic branch of the semantic segmentation network includes:

[0022] Obtain global context through global average pooling;

[0023] Calculate the attention vector according to the global context;

[0024] Refine the output features of each stage in the context path of the global context through the attention vector to complete the process of optimizing the semantic branch of the semantic segmentation network through attention optimization.

[0025] Preferably, the preset function includes a learning rate function and a loss function, and the process of supervising the shallow image detail information output by the detail branch and the deep semantic information output by the semantic branch through the preset function includes:

[0026] Supervise the shallow image detail information output by the detail branch through the learning rate function;

[0027] Supervise the deep semantic information output by the semantic branch through the loss function.

[0028] The second aspect of the present application discloses a lane line detection method, which is applicable to the lane line detection model obtained by the model training method of any item in the first aspect. The lane line detection method includes:

[0029] Obtain the image to be detected;

[0030] Detect the image to be detected through the lane line detection model to obtain a segmentation map, and realize driverless driving through the segmentation map;

[0031] The step of detecting the image to be detected through the lane line detection model to obtain a segmentation map, and realizing driverless driving through the segmentation map includes:

[0032] Perform a detail branch on the image to be detected through the lane line detection model to obtain the image shallow layer detail information of the image to be detected;

[0033] Optimize the image to be detected through the lane line detection model to obtain the deep semantic information of the image to be detected;

[0034] Detect the image shallow layer detail information and the deep semantic information through the lane line detection model to obtain a segmentation map, and realize driverless driving through the segmentation map.

[0035] Preferably, it further includes:

[0036] Assist in the detection of the missing lane lines in the image to be detected through the optimized semantic branch and the prior information of the lane lines; the prior information at least includes the sampling information obtained by uniformly sampling the ordinate of the lane lines in the image coordinate system of the image to be detected and the information related to the abscissa and ordinate of the lane lines in the image coordinate system.

[0037] The third aspect of the present application discloses a model training system, which includes:

[0038] An acquisition unit for acquiring road scene images;

[0039] A labeling unit for labeling the road scene images to obtain a labeled image data set, and obtaining label data from the labeled image data set; the label data is a data file formed by labeling the lane line areas in the scene images of vehicle driving;

[0040] An extraction unit for extracting the image shallow layer detail information from the label data through a detail branch; the image shallow layer detail information is the information of the lane line positions;

[0041] An optimization unit for optimizing the semantic branch of a semantic segmentation network and extracting deep semantic information from the scene image in which the vehicle travels through the optimized semantic branch;

[0042] A supervision unit for supervising the shallow image detail information output by the detail branch and the deep semantic information output by the semantic branch through a preset function to complete the training of the lane line detection model.

[0043] A fourth aspect of the present application discloses a lane line detection system applicable to the lane line detection model trained by the model training system described in the third aspect. The lane line detection system includes:

[0044] An acquisition unit for acquiring an image to be detected;

[0045] A detection unit for detecting the image to be detected through the lane line detection model to obtain a segmentation map and realizing driverless driving through the segmentation map;

[0046] The detection unit includes:

[0047] A detail branch module for performing a detail branch on the image to be detected through the lane line detection model to obtain shallow image detail information of the image to be detected;

[0048] An optimization module for optimizing the image to be detected through the lane line detection model to obtain deep semantic information of the image to be detected;

[0049] A detection module for detecting the shallow image detail information and the deep semantic information through the lane line detection model to obtain a segmentation map and realizing driverless driving through the segmentation map.

[0050] As can be seen from the above technical solutions, road scene images are collected, the road scene images are labeled to obtain a labeled image dataset, and label data is obtained from the labeled image dataset. The label data is a data file formed by labeling the lane line area in the scene image of vehicle driving. Through the detail branch, the shallow image detail information is extracted from the label data. The shallow image detail information is the information of the lane line position. The semantic branch of the semantic segmentation network is optimized, and the deep semantic information is extracted from the scene image of vehicle driving through the optimized semantic branch. The shallow image detail information output by the detail branch and the deep semantic information output by the semantic branch are supervised by a preset function to complete the training of the lane line detection model. Through the lane line detection model, the image to be detected is detected to obtain a segmentation map, and autonomous driving is realized through the segmentation map. Through the above solution, the image to be detected is optimized by the lane line detection model, so as to improve the convolutional receptive field to enhance the extraction of lane line information, assist in detecting the lane lines lost in the image field caused by occlusion or strong light, etc., and shield the interference of non-lane lines, so as to improve the lane line detection effect and accuracy to realize autonomous driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0052] Figure 1 It is a schematic flow chart of a model training method disclosed in an embodiment of the present application;

[0053] Figure 2 It is a schematic flow chart of a lane line detection method disclosed in an embodiment of the present application;

[0054] Figure 3 It is the segmentation map required for autonomous driving disclosed in an embodiment of the present application;

[0055] Figure 4 It is a schematic structural diagram of a model training system disclosed in an embodiment of the present application;

[0056] Figure 5 It is a schematic structural diagram of a lane line detection system disclosed in an embodiment of the present application;

[0057] Figure 6 It is a schematic structural diagram of an electronic device disclosed in an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0058] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the protection scope of the present application.

[0059] In the present application, the term "including", "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article or device including a series of elements not only includes those elements but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including a..." does not exclude the presence of additional identical elements in the process, method, article or device including the said element.

[0060] As can be seen from the background art, although the backbone network has few parameters and fast detection speed, it sacrifices the detection effect and has poor detection effect under occlusion and extreme lighting conditions; secondly, the convolutional receptive field of such small-sized networks is small, which is not conducive to lane line detection. Therefore, how to improve the accuracy of lane line detection to achieve driverless driving is an urgent problem to be solved in the present application.

[0061] To solve the above problems, the present application discloses a model training method, a lane line detection method and a system. The lane line detection model is used to optimize the image to be detected, so as to increase the convolutional receptive field to enhance the extraction of lane line information, assist in detecting lane lines lost in the image field caused by occlusion or strong light, and shield the interference of non-lane lines, thereby improving the lane line detection effect and accuracy to achieve driverless driving. The specific implementation manners will be described through the following embodiments.

[0062] Refer to Figure 1 As shown, it is a schematic flowchart of a model training method disclosed in an embodiment of the present application. The model training method mainly includes the following steps:

[0063] S101: Collect road scene images.

[0064] In S101, a road scene image required for the lane line detection algorithm is collected by a collection device.

[0065] Among them, the collection device may be a camera, a webcam, etc. The specific determination of the collection device is not specifically limited in the present application.

[0066] S102: Annotate the road scene images to obtain an annotated image dataset, and obtain label data from the annotated image dataset; the label data is a data file formed by annotating the lane line area in the scene images of vehicle driving.

[0067] It should be noted that the road scene images are annotated to obtain an annotated image dataset to create a training set required for training a lane line detection model.

[0068] Read the scene images of vehicle driving from the annotated image dataset, and obtain the label data corresponding to the scene images of vehicle driving.

[0069] The scene images of vehicle driving are obtained by reading from the annotated image dataset.

[0070] After executing S102, S103 and S104 are executed respectively, and S103 and S104 are in a parallel relationship. The scene images of vehicle driving enter two parallel branches of the lane line detection model, and the two parallel branches include a detail branch and a semantic branch of the semantic segmentation network.

[0071] S103: Extract the shallow image detail information from the label data through the detail branch; the shallow image detail information is the information of the lane line position.

[0072] After executing S103, S105 is executed.

[0073] It should be noted that the detail branch has a shallow network branch with a wider channel width (the number of channels in each layer of the network branch is more, but the number of network layers is less), which is used to capture low-level details and generate a high-resolution feature representation.

[0074] Among them, capturing low-level details and generating a high-resolution feature representation means that a low-resolution feature map can achieve a stronger high-resolution feature representation, that is, the shallow image detail information of the lane line position.

[0075] In S103, after continuously convolving and pooling the label data, the shallow image detail information is extracted.

[0076] S104: Optimize the semantic branch of the semantic segmentation network, and extract deep semantic information from the scene images of vehicle driving through the optimized semantic branch.

[0077] In S104, the semantic branch of the semantic segmentation network can be optimized by optimization methods such as multi-scale convolution optimization, preset auxiliary optimization, and attention optimization.

[0078] The determination of the specific optimization method is not specifically limited in this application. The optimization methods of this application preferably include multi-scale convolution optimization, preset auxiliary optimization, and attention optimization.

[0079] Multi-scale convolution optimization is used to enhance the extraction of lane line information in the scene images of vehicle driving.

[0080] Preset auxiliary optimization is used to assist in detecting the lane lines lost in the image caused by reasons such as occlusion or strong light in the image to be detected.

[0081] Attention optimization is used to shield the interference of non-lane lines and improve the lane line detection effect.

[0082] The semantic branch, that is, the scene image of vehicle driving is continuously downsampled multiple times using convolution to obtain the downsampling result, and the downsampling result is input into the pooling layer for average pooling to obtain deep semantic information, that is, high-level semantic context information.

[0083] Among them, the number of times of continuously downsampling the scene image of vehicle driving using convolution is not specifically limited in this application. The number of times of continuous downsampling in this application is preferably 4 times.

[0084] The deep semantic information at least includes the category information of the texture, color, and / or shape of the scene image of vehicle driving.

[0085] Specifically, the process of optimizing the semantic branch of the semantic segmentation network through multi-scale convolution optimization is shown in A1 - A2.

[0086] A1: Feature extraction is performed on the feature map of the network layer input to the semantic segmentation network to obtain feature maps of various preset types.

[0087] Among them, through multi-scale convolution optimization with strip-shaped convolution, the feature map of the network layer input to the semantic segmentation network is subjected to a preset number of convolution operations to obtain feature maps with various preset convolution kernel sizes.

[0088] Among them, the preset number of times can be 3 times or 4 times. The specific determination of the preset number of times is set according to the actual situation and is not limited in this application. The preset number of times in this application is preferably 3 times.

[0089] For example, in the semantic branch, multi-scale convolution optimization with strip-shaped convolution is used to perform 3 convolution operations on the feature map of the network layer input to the semantic segmentation network to obtain three strip-shaped convolutions with convolution kernel sizes of 3, 5, and 7. The feature maps corresponding to the convolution kernel sizes of 3, 5, and 7 are used to perform feature extraction on the feature map respectively to obtain the feature map corresponding to the convolution kernel size of 3, the feature map corresponding to the convolution kernel size of 5, and the feature map corresponding to the convolution kernel size of 7.

[0090] A2: The feature maps of various preset types are fused to enhance the extraction of lane line information.

[0091] For example, in the semantic branch, multi-scale convolution optimization with strip-shaped convolution is used. Three convolution operations are performed on the feature map of the network layer input to the semantic segmentation network to obtain three strip-shaped convolutions with convolution kernel sizes of 3, 5, and 7. These three strip-shaped convolutions with convolution kernel sizes of 3, 5, and 7 are used to extract features from the feature map respectively, obtaining the feature map corresponding to the convolution kernel size of 3, the feature map corresponding to the convolution kernel size of 5, and the feature map corresponding to the convolution kernel size of 7. Then, the three extracted feature maps are fused to enhance the extraction of lane line information.

[0092] In the process of optimizing the semantic branch of the semantic segmentation network through preset auxiliary optimization, an optimization module based on longitudinal anchor (column-anchor) is added to the semantic branch to assist in detecting lane lines lost in the image field of view caused by situations such as occlusion or extreme strong light. The optimization module of column-anchor utilizes the prior information of the lane lines in the scene image of vehicle driving, that is, the lane lines are evenly sampled on the ordinate in the image coordinate system, and the abscissa is associated with and corresponds one-to-one with their respective ordinates, representing the lane lines with sparse coordinates on a series of mixed (row and column) anchors to assist in detecting the lane lines with lost field of view in the image.

[0093] The specific process of optimizing the semantic branch of the semantic segmentation network is shown in B1 - B3.

[0094] B1: Obtain global context through global average pooling.

[0095] Among them, an attention optimization module (Attention Refinement Module, ARM) is used in the semantic branch. This attention optimization module obtains global context through global average pooling.

[0096] Global average pooling performs average operations on each channel respectively, that is, calculates an average value for all pixels of the feature map of each output channel. After global average pooling, a feature vector of one dimension is obtained.

[0097] Global average pooling can reduce the number of parameters. The global average pooling layer has no parameters, which can prevent overfitting in this layer.

[0098] B2: Calculate the attention vector according to the global context.

[0099] B3: Refine the output features of each stage in the context path of the global context through the attention vector, and complete the process of optimizing the semantic branch of the semantic segmentation network through attention optimization.

[0100] Among them, an attention optimization module is used in the semantic branch. This module obtains global context through global average pooling and calculates attention vectors to guide feature learning, refining the output features at each stage of the context path.

[0101] Attention optimization can enhance the detection effect of the lane line detection model on lane lines and shield the interference of other non-lane line landmarks with similar shapes on the detection.

[0102] S105: Supervise the shallow image detail information output by the detail branch and the deep semantic information output by the semantic branch through a preset function to complete the training of the lane line detection model.

[0103] Among them, the shallow image detail information output by the detail branch and the deep semantic information output by the semantic branch are supervised through a preset function to complete the training of the lane line detection model.

[0104] The preset function at least includes a learning rate function and a loss function, etc.

[0105] The loss function is an operation function used to measure the difference between the predicted value and the true value of the lane line detection model. The smaller the loss function, the better the robustness of the lane line detection model.

[0106] Supervise the shallow image detail information output by the detail branch through the learning rate function, and supervise the deep semantic information output by the semantic branch through the loss function.

[0107] This solution effectively utilizes the column-anchor-based detection method to optimize the semantic segmentation network through multi-scale convolution optimization and channel attention optimization, meets the operation requirements of in-vehicle computer motherboards, and realizes enhanced real-time detection effects on lane lines.

[0108] In the embodiment of this application, the semantic branch of the semantic segmentation network is optimized through multi-scale convolution optimization, preset auxiliary optimization, and attention optimization, and deep semantic information is extracted from the scene image of the vehicle's driving through the optimized semantic branch. The shallow image detail information output by the detail branch and the deep semantic information output by the semantic branch are supervised through a preset function to complete the training of the lane line detection model, thereby improving the accuracy of the lane line detection model in detecting image data.

[0109] Reference Figure 2 , shows a schematic flowchart of a lane line detection method provided by an embodiment of this application. This lane line detection method is applicable to a lane line detection model trained by the Figure 1 disclosed model training method. This lane line detection method specifically includes the following steps:

[0110] S201: Obtain the image to be detected.

[0111] Among them, the image to be detected is the image that has not been detected by the lane line detection model.

[0112] S202: Detect the image to be detected through the lane line detection model to obtain the segmentation map required for driverless driving, and realize driverless driving through the segmentation map.

[0113] Specifically, detect the image to be detected through the lane line detection model to obtain the segmentation map required for driverless driving, and realize driverless driving through the segmentation map, as shown in C1 - C3.

[0114] C1: Through the lane line detection model, perform a detail branch on the image to be detected to obtain the shallow - layer detail information of the image to be detected.

[0115] In C1, the shallow - layer detail information of the image to be detected represents the information of the lane line position in the image to be detected.

[0116] C2: Through the lane line detection model, optimize the image to be detected to obtain the deep semantic information of the image to be detected.

[0117] In C2, through the lane line detection model, optimization methods such as multi - scale convolution optimization, preset auxiliary optimization, and attention optimization can be respectively performed on the image to be detected to obtain the deep semantic information of the image to be detected.

[0118] The determination of the specific optimization method is not specifically limited in this application. The optimization methods of this application preferably include multi - scale convolution optimization, preset auxiliary optimization, and attention optimization.

[0119] The deep semantic information of the image to be detected at least includes the category information of the texture, color, and / or shape of the scene image in which the vehicle travels in the image to be detected.

[0120] With the optimized semantic branch and the prior information of the lane line, assist in the detection of the missing lane line in the image to be detected; the prior information at least includes the sampling information obtained by uniformly sampling the ordinate of the lane line in the image coordinate system of the image to be detected and the information associated with the abscissa and ordinate of the lane line in the image coordinate system.

[0121] Among them, through multi - scale convolution optimization, preset auxiliary optimization, and attention optimization, the semantic branch is optimized, and with the optimized semantic branch and the prior information of the lane line, assist in the detection of the missing lane line in the image to be detected.

[0122] C3: Through the lane line detection model, detect the shallow - layer detail information and the deep semantic information of the image to obtain the segmentation map, and realize driverless driving through the segmentation map.

[0123] Specific segmentation map as Figure 3 shown.

[0124] In the embodiments of the present application, the lane line detection model is used to perform multi-scale convolution optimization, preset auxiliary optimization, and attention optimization on the image to be detected respectively. The multi-scale convolution optimization is used to increase the convolution receptive field to enhance the extraction of lane line information. The preset auxiliary optimization is used to assist in detecting lane lines that are lost in the image field of view due to occlusion or strong light, etc. The attention optimization is used to shield the interference of non-lane lines, so as to improve the lane line detection effect and accuracy to achieve driverless driving.

[0125] Based on the above embodiments Figure 1 The flow schematic diagram of a model training method disclosed, and the embodiments of the present application also correspondingly disclose the structural schematic diagram of the model training system, as Figure 4 shown, the model training system includes an acquisition unit 401, a labeling unit 402, an extraction unit 403, an optimization unit 404, and a supervision unit 405.

[0126] The acquisition unit 401 is used to acquire road scene images.

[0127] The labeling unit 402 is used to label the road scene images to obtain a labeled image data set, and obtain label data from the labeled image data set; the label data is a data file formed by labeling the lane line area in the scene image of vehicle driving.

[0128] The extraction unit 403 is used to extract the shallow image detail information from the label data through the detail branch; the shallow image detail information is the information of the lane line position.

[0129] The optimization unit 404 is used to optimize the semantic branch of the semantic segmentation network, and extract deep semantic information from the scene image of vehicle driving through the optimized semantic branch.

[0130] The supervision unit 405 is used to supervise the shallow image detail information output by the detail branch and the deep semantic information output by the semantic branch through a preset function to complete the training of the lane line detection model.

[0131] Further, the labeling unit 402 that obtains label data from the labeled image data set is specifically used to read the scene image of vehicle driving from the labeled image data set and obtain the label data corresponding to the scene image of vehicle driving.

[0132] Further, the optimization unit 404 that optimizes the semantic branch of the semantic segmentation network includes an extraction module and a fusion module.

[0133] The extraction module is used to perform feature extraction on the feature map of the network layer input to the semantic segmentation network to obtain feature maps of various preset types.

[0134] A fusion module, which is used to fuse the feature maps of various preset types to enhance the extraction of lane line information.

[0135] Further, an extraction module, which is specifically used to perform a preset number of convolution operations on the feature maps of the network layer input to the semantic segmentation network through multi-scale convolution optimization with strip-shaped convolution, to obtain feature maps of various preset convolution kernel sizes.

[0136] Further, an optimization unit 404 for optimizing the semantic branch of the semantic segmentation network includes an acquisition module, a calculation module, and a refinement module.

[0137] The acquisition module is used to obtain global context through global average pooling.

[0138] The calculation module is used to calculate an attention vector according to the global context.

[0139] The refinement module is used to refine the output features of each stage in the context path of the global context through the attention vector, and complete the process of optimizing the semantic branch of the semantic segmentation network through attention optimization.

[0140] Further, a supervision unit 405 includes a first supervision module and a second supervision module.

[0141] The first supervision module is used to supervise the shallow-layer detail information of the image output by the detail branch through a learning rate function.

[0142] The second supervision module is used to supervise the deep semantic information output by the semantic branch through a loss function.

[0143] In the embodiment of the present application, the semantic branch of the semantic segmentation network is optimized, and deep semantic information is extracted from the scene image of the vehicle driving through the optimized semantic branch. The shallow-layer detail information of the image output by the detail branch and the deep semantic information output by the semantic branch are supervised through a preset function, and the training of the lane line detection model is completed, so as to improve the accuracy of the lane line detection model for detecting image data.

[0144] Based on the above embodiment Figure 2 A schematic flowchart of a lane line detection method disclosed, the embodiment of the present application also correspondingly discloses a schematic structural diagram of a lane line detection system, as Figure 5 shown, the lane line detection system includes an acquisition unit 501 and a detection unit 502.

[0145] The acquisition unit 501 is used to acquire an image to be detected.

[0146] The detection unit 502 is used to detect the image to be detected through a lane line detection model, obtain a segmentation map, and realize driverless driving through the segmentation map.

[0147] The further detection unit 502 includes a detail branch module, an optimization module, and a detection module.

[0148] The detail branch module is used to perform detail branching on the image to be detected through a lane line detection model, so as to obtain the shallow-layer detail information of the image to be detected.

[0149] The optimization module is used to optimize the image to be detected through a lane line detection model, so as to obtain the deep semantic information of the image to be detected.

[0150] The detection module is used to detect the shallow-layer detail information and the deep semantic information of the image through a lane line detection model, so as to obtain a segmentation map, and realize driverless driving through the segmentation map.

[0151] In the embodiment of the present application, the image to be detected is respectively subjected to multi-scale convolution optimization, preset auxiliary optimization, and attention optimization through a lane line detection model. The multi-scale convolution optimization is used to improve the convolution receptive field to enhance the extraction of lane line information. The preset auxiliary optimization is used to assist in detecting lane lines lost in the image field caused by occlusion or strong light, etc. The attention optimization is used to shield the interference of non-lane lines, so as to improve the lane line detection effect and accuracy to realize driverless driving.

[0152] The embodiment of the present application also provides a storage medium, which includes stored instructions. When the instructions run, the device where the storage medium is located is controlled to execute the above model training method or the above lane line detection method.

[0153] The embodiment of the present application also provides an electronic device, and its structural schematic diagram is as Figure 6 shown, specifically including a memory 601 and one or more instructions 602. One or more instructions 602 are stored in the memory 601 and are configured to be executed by one or more processors 603 to execute the above model training method or the above lane line detection method.

[0154] The specific implementation processes and their derivative methods of the above various embodiments are all within the protection scope of the present application.

[0155] Each embodiment in this specification is described in a progressive manner. For the same or similar parts among the embodiments, reference can be made to each other, and the key point of each embodiment is to illustrate the differences from other embodiments. In particular, for a system or system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and reference can be made to the corresponding part of the method embodiment for relevant content. The systems and system embodiments described above are merely illustrative. The units described as clustering components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. A person of ordinary skill in the art can understand and implement it without creative work.

[0156] Those skilled in the art can further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0157] The above description of the disclosed embodiments enables those skilled in the art to implement or use this application. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the scope of this application. Therefore, this application will not be limited to the embodiments shown herein, but will be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0158] The above is only the preferred embodiment of this application. It should be noted that for those of ordinary skill in the art, without departing from the principle of this application, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of this application.

Claims

1. A model training method, characterized in that, the model training method includes: Collect road scene images; Annotate the road scene images to obtain an annotated image dataset, and obtain label data from the annotated image dataset; the label data is a data file formed by annotating the lane line area in the scene image of the vehicle driving; Extract image shallow - layer detail information from the label data through a detail branch; the image shallow - layer detail information is information about the position of the lane line; Optimize the semantic branch of the semantic segmentation network, and extract deep semantic information from the scene image of the vehicle driving through the optimized semantic branch; Supervise the image shallow - layer detail information output by the detail branch and the deep semantic information output by the semantic branch through a preset function to complete the training of the lane line detection model.

2. The model training method according to claim 1, characterized in that, the obtaining of the label data from the annotated image dataset includes: Read the scene image of the vehicle driving from the annotated image dataset, and obtain the label data corresponding to the scene image of the vehicle driving.

3. The model training method according to claim 1, characterized in that, The process of optimizing the semantic branch of the semantic segmentation network includes: Extract features from the feature maps of the network layers input to the semantic segmentation network to obtain feature maps of various preset types; Fuse the feature maps of various preset types to enhance the extraction of lane line information.

4. The model training method according to claim 3, characterized in that, the extracting of features from the feature maps of the network layers input to the semantic segmentation network to obtain feature maps of various preset types includes: Through multi - scale convolution optimization with strip - shaped convolution, perform convolution operations on the feature maps of the network layers input to the semantic segmentation network for a preset number of times to obtain feature maps with various preset convolution kernel sizes.

5. The model training method according to claim 1, characterized in that, The process of optimizing the semantic branch of the semantic segmentation network includes: Obtain global context through global average pooling; Calculate an attention vector according to the global context; Refine the output features of each stage in the context path of the global context through the attention vector to complete the process of optimizing the semantic branch of the semantic segmentation network through attention optimization.

6. The model training method according to claim 1, characterized in that, the preset function includes a learning rate function and a loss function, and the process of supervising the image shallow - layer detail information output by the detail branch and the deep semantic information output by the semantic branch through the preset function includes: Supervise the image shallow - layer detail information output by the detail branch through the learning rate function; Supervise the deep semantic information output by the semantic branch through the loss function.

7. A lane line detection method, characterized in that, It is applicable to the lane line detection model obtained by the model training method according to any one of claims 1 - 5, and the lane line detection method includes: Obtain an image to be detected; Using the lane line detection model, detect the to-be-detected image to obtain a segmentation map, and achieve driverless driving through the segmentation map; The method of using the lane line detection model to detect the to-be-detected image to obtain a segmentation map and achieve driverless driving through the segmentation map includes: Using the lane line detection model, perform a detail branch on the to-be-detected image to obtain the image shallow-layer detail information of the to-be-detected image; Using the lane line detection model, optimize the to-be-detected image to obtain the deep semantic information of the to-be-detected image; Using the lane line detection model, detect the image shallow-layer detail information and the deep semantic information to obtain a segmentation map, and achieve driverless driving through the segmentation map.

8. The lane line detection method according to claim 7, wherein, it further includes: Using the optimized semantic branch and the prior information of the lane line to assist in the detection of the missing lane lines in the to-be-detected image; the prior information at least includes the sampling information obtained by uniformly sampling the ordinate of the lane lines in the to-be-detected image in the image coordinate system and the information that the abscissa of the lane lines in the image coordinate system is associated with the ordinate.

9. A model training system, wherein, the model training system includes: An acquisition unit for acquiring road scene images; A labeling unit for labeling the road scene images to obtain a labeled image data set, and obtaining label data from the labeled image data set; the label data is a data file formed by labeling the lane line regions in the scene images of vehicle driving; An extraction unit for extracting the image shallow-layer detail information from the label data through a detail branch; the image shallow-layer detail information is the information of the lane line positions; An optimization unit for optimizing the semantic branch of the semantic segmentation network and extracting the deep semantic information from the scene images of vehicle driving through the optimized semantic branch; A supervision unit for supervising the image shallow-layer detail information output by the detail branch and the deep semantic information output by the semantic branch through a preset function to complete the training of the lane line detection model.

10. A lane line detection system, wherein, being applicable to the lane line detection model trained by the model training system according to claim 9, the lane line detection system includes: An acquisition unit for acquiring a to-be-detected image; A detection unit for detecting the to-be-detected image through the lane line detection model to obtain a segmentation map, and achieving driverless driving through the segmentation map; The detection unit includes: A detail branch module for performing a detail branch on the to-be-detected image through the lane line detection model to obtain the image shallow-layer detail information of the to-be-detected image; An optimization module for optimizing the to-be-detected image through the lane line detection model to obtain the deep semantic information of the to-be-detected image; A detection module for detecting the image shallow-layer detail information and the deep semantic information through the lane line detection model to obtain a segmentation map, and achieving driverless driving through the segmentation map.

Images