Lane detection method, device, equipment and storage medium

Abstract

The present invention relates to the field of assisted driving and automatic driving, and discloses a lane line detection method, device, equipment and storage medium. The method comprises: acquiring image information in front of a target vehicle, and performing feature extraction on the image information to obtain corresponding image feature information; predicting a starting point and an end point of the lane line according to the image feature information to obtain the starting point and the end point of the lane line; generating an initial lane line according to the starting point and the end point; predicting an offset between the initial lane line and a target lane line, adjusting the coordinates of the initial lane line according to the predicted offset, and obtaining the coordinates of the predicted lane line, so as to realize the detection of the lane line in front of the target vehicle. Compared with the traditional lane line detection method based on artificially set image features, the present application can obtain a point on the lane line by predicting the deviation between the initial lane line and the target lane line, thereby effectively improving the efficiency and accuracy of lane line detection.

Description

Technical Field

[0001] The present invention relates to the field of assisted driving and automatic driving, and in particular to a lane line detection method, device, equipment and storage medium. Background Art

[0002] In the past decade, with the popularization of in-vehicle smart devices and the continuous improvement of vehicle-road cooperative systems, autonomous driving has attracted more and more attention. Lane detection is a basic step in many intelligent advanced driving assistance systems (ADAS), such as lane departure warning systems, which warn drivers when the vehicle deviates from its lane. Current machine learning-based lane detection methods usually use two-dimensional convolutional neural networks for instance segmentation of lane lines. The segmentation methods based on square kernel functions of two-dimensional convolutional neural networks have difficulty in capturing the slender curve characteristics of lanes. Due to the neglect of high-level global semantic features (or contextual information), these methods often suffer from discontinuous and noisy detection results. Secondly, the desired output of autonomous driving is control-related parameters, namely vehicle lateral offset, turning angle, and curvature. However, the output of most lane detection methods based on convolutional neural networks (CNNs) is the pixel points belonging to the lane line. To fill this gap, some post-processing procedures are needed, such as inverse perspective mapping (IPM) and lane model fitting.

[0003] The above contents are only used to assist in understanding the technical solution of the present invention and do not constitute an admission that the above contents are prior art. Summary of the invention

[0004] The main purpose of the present invention is to provide a lane line detection method, device, equipment and storage medium, aiming to improve the efficiency and accuracy of lane line detection.

[0005] To achieve the above object, the present invention provides a lane line detection method, the lane line detection method comprising the following steps:

[0006] Acquire image information in front of the target vehicle, and perform feature extraction on the image information to obtain corresponding image feature information;

[0007] Predicting the starting point and the end point of the lane line according to the image feature information to obtain the starting point and the end point of the lane line;

[0008] Generate an initial lane line according to the starting point and the end point;

[0009] The offset between the initial lane line and the target lane line is predicted, and the coordinates of the initial lane line are adjusted according to the predicted offset to obtain the coordinates of the predicted lane line, so as to realize the detection of the lane line in front of the target vehicle.

[0010] Optionally, the acquiring image information in front of the target vehicle and performing feature extraction on the image information to obtain corresponding image feature information includes:

[0011] Acquire a preset acquisition device arranged on the target vehicle, and acquire image information in front of the target vehicle according to the preset acquisition device;

[0012] A preset backbone network model is obtained, and features are extracted from the collected image information according to the preset backbone network model to obtain corresponding image feature information.

[0013] Optionally, predicting the starting point and the end point of the lane line according to the image feature information to obtain the starting point and the end point of the lane line includes:

[0014] Obtaining a convolutional layer head network, performing channel conversion on the image feature information according to the head network, and obtaining a heat map for key point detection;

[0015] Analyze the heat map according to the key point preset strategy to obtain the peak point of the heat map, and use the coordinates of the peak point as the starting point of the lane line;

[0016] The end point of the lane line is predicted according to the image feature information to obtain the end point of the lane line.

[0017] Optionally, before taking the coordinates of the peak point as the starting point of the lane line, the method further includes:

[0018] Selecting the coordinates of the peak points of the heat map to obtain the coordinates of the corresponding peak points;

[0019] Obtaining a preset threshold value, and determining whether the score of the peak coordinate is greater than the preset threshold value;

[0020] If the score of the peak coordinate is greater than a preset threshold, executing the step of using the coordinate of the peak point as the starting point of the lane line;

[0021] Based on the number of detected lane line starting points, the number of predicted lane lines L is determined.

[0022] Optionally, predicting the starting point and the end point of the lane line according to the image feature information to obtain the starting point and the end point of the lane line includes:

[0023] Predicting the starting point of the lane line according to the image feature information to obtain the starting point of the lane line;

[0024] Obtaining a convolutional layer head network, and performing mapping conversion on the image feature information according to the head network to obtain a probability map for estimating the position of the lane line vanishing point;

[0025] An end point of the lane line is predicted according to the probability map and the vanishing point.

[0026] Optionally, predicting the end point of the lane line according to the probability map and the vanishing point includes:

[0027] Scan the probability map to obtain the width occupied by the current lane line point;

[0028] The width of the next lane line point corresponding to the scanned probability map is obtained. If the width of the current lane line point is greater than half of the width of the next lane line point, the row where the lane line vanishing point is located and its width are obtained according to the width of the current row and the vanishing point.

[0029] The row of the lane line vanishing point is divided into L-1 equal parts, and the end point of the lane line is predicted.

[0030] Optionally, the predicted offset between the initial lane line and the target lane line, adjusting the coordinates of the initial lane line according to the predicted offset, and obtaining the coordinates of the predicted lane line to detect the lane line in front of the target vehicle, comprises:

[0031] Obtaining image feature information and an LDNet algorithm, and based on the image feature information, LDNet predicts an offset between an initial lane line and a corresponding target lane line;

[0032] The coordinates of the initial lane line are adjusted according to the predicted offset to obtain the coordinates of the predicted lane line, so as to detect the lane line in front of the target vehicle.

[0033] In addition, to achieve the above-mentioned purpose, the present invention further provides a lane line detection device, the lane line detection device comprising:

[0034] A feature extraction module is used to obtain image information in front of the target vehicle in real time in response to the lane marking instruction, and perform feature extraction on the image information to obtain corresponding image feature information;

[0035] A prediction module, used to predict the starting point and the end point of the lane line according to the image feature information to obtain the starting point and the end point of the lane line;

[0036] A generating module, used for generating an initial lane line according to the starting point and the end point;

[0037] The adjustment module is used to predict the offset between the initial lane line and the target lane line, adjust the coordinates of the initial lane line according to the predicted offset, and obtain the coordinates of the predicted lane line to realize the detection of the lane line in front of the target vehicle.

[0038] In addition, to achieve the above-mentioned purpose, the present invention also proposes a lane line detection device, which includes: a memory, a processor, and a lane line detection program stored in the memory and executable on the processor, and the lane line detection program is configured to implement the steps of the lane line detection method described above.

[0039] In addition, to achieve the above-mentioned purpose, the present invention also proposes a storage medium, on which a lane line detection program is stored. When the lane line detection program is executed by a processor, the steps of the lane line detection method described above are implemented.

[0040] The lane line detection method proposed by the present invention obtains the image information in front of the target vehicle, and extracts the features of the image information to obtain the corresponding image feature information; predicts the starting point and the end point of the lane line according to the image feature information to obtain the starting point and the end point of the lane line; generates the initial lane line according to the starting point and the end point; predicts the offset between the initial lane line and the target lane line, and adjusts the coordinates of the initial lane line according to the predicted offset to obtain the coordinates of the predicted lane line, so as to realize the detection of the lane line in front of the target vehicle. The present invention obtains the image information in front of the target vehicle during the driving process, extracts the features of the image information to obtain the corresponding feature image information, and obtains the starting point and the end point of the lane line according to the feature image information, connects the starting point and the end point to generate the corresponding initial lane line, predicts the offset between the initial lane line and the target lane line, and adjusts the coordinates of the initial lane line according to the predicted offset to obtain the coordinates of the predicted lane line, so as to realize the detection of the lane line in front of the target vehicle, so as to obtain the point on the lane line by predicting the deviation between the initial lane line and the target lane line, thereby effectively improving the efficiency and accuracy of lane line detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a structural schematic diagram of a lane line detection device in a hardware operating environment involved in an embodiment of the present invention;

[0042] Figure 2 A schematic diagram of a flow chart of a first embodiment of a lane line detection method of the present invention;

[0043] Figure 3 A schematic diagram of a flow chart of a second embodiment of a lane line detection method of the present invention;

[0044] Figure 4 A lane line detection flow chart of an embodiment of a lane line detection method of the present invention;

[0045] Figure 5 A diagram showing a lane line detection implementation scheme of an embodiment of a lane line detection method of the present invention;

[0046] Figure 6 A schematic diagram of a flow chart of a third embodiment of a lane line detection method of the present invention;

[0047] Figure 7 Schematic diagram of functional modules of the first embodiment of the lane line detection device of the present invention.

[0048] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0049] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0050] Reference Figure 1 , Figure 1 A schematic diagram of the structure of a lane line detection device in a hardware operating environment involved in an embodiment of the present invention.

[0051] like Figure 1 As shown, the lane line detection device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (WIreless-FIdelity, WI-FI) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM) memory, or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0052] Those skilled in the art will understand that Figure 1The structure shown in the figure does not constitute a limitation on the lane detection device, and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0053] like Figure 1 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and a lane line detection program.

[0054] exist Figure 1 In the lane line detection device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the lane line detection device of the present invention can be set in the lane line detection device, and the lane line detection device calls the lane line detection program stored in the memory 1005 through the processor 1001, and executes the lane line detection method provided by the embodiment of the present invention.

[0055] Based on the above hardware structure, an embodiment of a lane line detection method of the present invention is proposed.

[0056] Reference Figure 2 , Figure 2 FIG. 1 is a flow chart of a first embodiment of a lane line detection method according to the present invention.

[0057] In a first embodiment, the lane line detection method comprises the following steps:

[0058] Step S10, acquiring image information in front of the target vehicle, and performing feature extraction on the image information to obtain corresponding image feature information.

[0059] It should be noted that the executor of this embodiment may be a lane line detection device, or other devices that can achieve the same or similar functions. This embodiment does not limit this. In this embodiment, the lane line detection device is taken as an example for description.

[0060] It should be understood that the image information obtained in the forward direction of the target vehicle refers to the image information collected by a preset acquisition device during the driving process of the target vehicle. The image information can be the ground information passed by the target vehicle, or the vegetation information within the line of sight of the target vehicle, or other information. All image information collected by the preset acquisition device is the image information in the forward direction of the target vehicle. The preset acquisition device can be a driving recorder, an image acquisition recorder, and other devices with a shooting function, and this embodiment does not limit this.

[0061] It can be understood that the image feature information refers to the information obtained by extracting features from the image information. When extracting features from the image information, a preset backbone network model is required. The preset backbone network model is a fully convolutional neural network model, which can be a deep residual network model (Residual Network, ResNet), or an hourglass network model (Hourglass Network, Hourglass), or a deep fusion network model (DeepLayer Aggregation, DLA). This embodiment does not limit this. In this embodiment, the deep residual network model is taken as an example for explanation. The function of the deep residual network model is to take image information as input, process it through the deep residual network model, and output image feature information for use in predicting the initial point and end point of the subsequent lane line.

[0062] In a specific implementation, a lane line detection device obtains image information in front of a target vehicle, and performs feature extraction on the image information to obtain corresponding image feature information.

[0063] Step S20, predicting the starting point and the end point of the lane line according to the image feature information to obtain the starting point and the end point of the lane line.

[0064] It should be understood that the starting point of the lane line is obtained by analyzing the heat map according to the preset key point prediction strategy, and the preset key point prediction strategy refers to a key point prediction strategy based on a heat map. Specifically, the preset key point prediction strategy is used to convert the image feature information into a heat map for key point detection, and then the starting point coordinates of the lane line are obtained according to the peak value of the heat map.

[0065] In a specific implementation, a lane line detection device predicts the starting point and the end point of the lane line according to the image feature information to obtain the starting point and the end point of the lane line.

[0066] Step S30, generating an initial lane line according to the starting point and the end point.

[0067] It should be understood that the initial lane line refers to the lane line obtained by connecting the initial point and the end point, and in order to prevent the initial lane line from intersecting during the process of generating the lane line, when obtaining the starting point of the lane line, it is necessary to sort the obtained starting points in counterclockwise order so that the generation of the initial lane line can be carried out in an orderly manner without the occurrence of lane line intersection.

[0068] In a specific implementation, the lane line detection device generates an initial lane line according to the starting point and the end point.

[0069] Step S40, predicting the offset between the initial lane line and the target lane line, adjusting the coordinates of the initial lane line according to the predicted offset, and obtaining the coordinates of the predicted lane line to achieve detection of the lane line in front of the target vehicle.

[0070] It should be understood that the offset between the predicted initial lane line and the target lane line is calculated by the LDNet algorithm based on the image feature information. After obtaining the offset between the initial lane line and the target lane line, the coordinates of the initial lane line are adjusted according to the predicted offset to obtain the coordinates of the predicted lane line to achieve detection of the lane line. For example, the offset calculated by the LDNet algorithm based on the image feature information is A, the value corresponding to the offset A is (Δx, Δy), the initial lane line is B, and the coordinates corresponding to the initial lane line point are (x, y), then the coordinates corresponding to the detected lane line point are (x+Δx, y+Δy).

[0071] It can be understood that the LDNet algorithm is divided into three parts, namely, feature learning block, fusion block and prediction block. The input is the position of the initial lane line and the image feature information extracted by the backbone network, and the output is the offset of each point pointing to the boundary of the target lane line. Specifically, after obtaining the initial lane line, a lane line point is selected on the initial lane line, and a corresponding feature vector is constructed according to the selected lane line point. The input feature is the concatenation of the image feature corresponding to the point and the coordinates of the point. The offset between the initial lane line and the target lane line is predicted based on the input feature vector. The coordinates of the initial lane line are adjusted according to the predicted offset to obtain the coordinates of the predicted lane line.

[0072] In a specific implementation, the lane line detection device predicts the offset between the initial lane line and the target lane line, adjusts the coordinates of the initial lane line according to the predicted offset, and obtains the coordinates of the predicted lane line to realize the detection of the lane line in front of the target vehicle.

[0073] This embodiment obtains the image information in front of the target vehicle and extracts the features of the image information to obtain the corresponding image feature information; predicts the starting point and the end point of the lane line according to the image feature information to obtain the starting point and the end point of the lane line; generates an initial lane line according to the starting point and the end point; predicts the offset between the initial lane line and the target lane line, adjusts the coordinates of the initial lane line according to the predicted offset, and obtains the coordinates of the predicted lane line to detect the lane line in front of the target vehicle. The present invention obtains the image information in front of the target vehicle during the driving process, extracts the features of the image information to obtain the corresponding feature image information, obtains the starting point and the end point of the lane line according to the image feature information, connects the starting point and the end point to generate the corresponding initial lane line, predicts the offset between the initial lane line and the target lane line, adjusts the coordinates of the initial lane line according to the predicted offset, and obtains the coordinates of the predicted lane line to detect the lane line in front of the target vehicle, so that the deviation between the initial lane line and the target lane line can be predicted, thereby effectively improving the efficiency and accuracy of lane line detection.

[0074] In one embodiment, if Figure 3 Based on the first embodiment, a second embodiment of the lane line detection method of the present invention is proposed, wherein step S20 includes:

[0075] Step S201, obtaining a convolutional layer head network, performing channel conversion on the image feature information according to the head network, and obtaining a heat map for key point detection.

[0076] It should be understood that the convolutional layer head network refers to a neural network composed of several convolutional units, and the parameters of each convolutional unit are optimized through the back propagation algorithm. The purpose of the convolution operation is to extract different features of the input, because the first convolutional layer of the neural network may extract relatively low-level features such as edges, lines, and corners, while the multi-layer head network can iteratively extract more complex features from relatively low-level features.

[0077] It can be understood that the heat map is obtained by performing channel conversion on the image feature information through the head network. Since the heat map is obtained from the image feature information, the size of the obtained heat map is the same as the size of the feature map obtained by the feature information. The heat map includes points and peaks on the heat map.

[0078] In a specific implementation, a lane line detection device obtains a convolutional layer head network, performs channel conversion on the image feature information according to the head network, and obtains a heat map for key point detection.

[0079] Step S202, predictively analyze the heat map according to the key point preset strategy, obtain the peak point of the heat map, and use the coordinates of the peak point as the starting point of the lane line.

[0080] It should be understood that each position in the heat map corresponds to a score, which indicates the probability that the position is the starting point of the lane line. Non-Maximum Suppression (NMS) is used to remove duplicate values ​​around the correct predicted value. The first N peak coordinates of the heat map are taken as the predicted lane line starting point positions. The N here is determined by the maximum number of lane lines in the image. In addition, only when the score of the peak position is greater than a certain threshold T is it a valid lane line starting point. The value of the threshold T is determined by the experimenter based on the experimental results. The number of predicted lane lines L is determined by the number of valid lane line starting points.

[0081] In a specific implementation, the lane line detection device analyzes the heat map according to the key point preset strategy, obtains the peak point of the heat map, and uses the coordinates of the peak point as the starting point of the lane line.

[0082] Step S203: predicting the end point of the lane line according to the image feature information to obtain the end point of the lane line.

[0083] It should be understood that the end point of the lane line is obtained through the lane line probability map and the position and width of the vanishing point, wherein the value corresponding to each point in the probability map represents the probability that the position belongs to the lane line, and the probability map is obtained by mapping the image feature information through a head network composed of convolutional layers.

[0084] In a specific implementation, a lane line detection device predicts the end point of the lane line according to the image feature information to obtain the end point of the lane line.

[0085] Reference Figure 4 , Figure 4This is a flow chart of lane line detection in the present invention. The specific process is that during the driving process of the target vehicle, the road condition image information in front of the vehicle is collected by a camera installed on the vehicle, and the lane line detection program reads the original image to start the lane line detection. The road condition image information in front is extracted according to the backbone network model to obtain image information features. The feature information is used to obtain the initial lane line generation. According to the feature information, the preset key point prediction strategy and the vanishing point of the lane line, the corresponding starting point and end point are obtained, and the starting point and end point are connected to generate the initial lane line. Specifically, based on the predicted starting point and end point of the lane line, the number of predicted lane lines L is determined, and the straight line connecting the predicted starting point and end point of the lane line is used as the initial lane line. The offset prediction from the initial lane line collection point to the lane line boundary is predicted based on the LDNet network.

[0086] Reference Figure 5 , Figure 5 The lane line detection implementation scheme diagram of an embodiment of the lane line detection method of the present invention is a specific implementation scheme diagram, wherein the starting point and the end point of the initial lane line are obtained according to different neural network branches, and the starting point of the initial lane is obtained by two 1*1 convolution layer head networks, and the head network is used to perform channel conversion on the extracted image feature information to obtain the corresponding channel heat map, and the peak point on the channel heat map is used as the initial point of the initial lane line. The end point of the lane line is obtained by two 1*1 convolution layer head networks, and the head network is used to perform mapping conversion on the extracted image feature information to obtain the corresponding channel probability map, and the channel probability map is scanned to find the first row with lane line points, and the width occupied by the current lane line points is compared with the width occupied by the next row of lane line points, and the comparison operation is repeated until the width occupied by the current lane line points is greater than half of the width occupied by the next row of lane line points, and the row of the vanishing point is obtained according to the width of the current row and the vanishing point, and the row of the vanishing point is divided into L-1 equal parts to obtain the end point of the initial lane line.

[0087] This embodiment obtains a heat map for key point detection by acquiring a convolutional layer head network, and performs channel conversion on the image feature information according to the convolutional layer head network; performs predictive analysis on the heat map according to the key point preset strategy to obtain the peak point of the heat map, and uses the coordinates of the peak point as the starting point of the lane line; predicts the end point of the lane line according to the image feature information to obtain the end point of the lane line. By acquiring a convolutional layer head network, and performing channel conversion on the image feature information according to the convolutional layer head network, a heat map for key point detection is obtained, and the heat map is predicted and analyzed by the key point prediction strategy to obtain the corresponding peak point, and the coordinates of the peak point are selected to obtain the corresponding coordinates, and the score of the peak coordinates is compared with the preset threshold. If the score of the peak coordinates is greater than the preset threshold, the coordinates of the peak point are used as the starting point of the lane line, thereby effectively improving the accuracy of obtaining the starting point of the lane line.

[0088] In one embodiment, if Figure 6 Based on the first embodiment, a third embodiment of the lane line detection method of the present invention is proposed, wherein step S20 includes:

[0089] Step S204: predicting the starting point of the lane line according to the image feature information to obtain the starting point of the lane line.

[0090] It should be understood that the starting point of the lane line is obtained according to the peak value of the channel heat map, wherein the channel heat map is obtained by performing channel conversion on the image feature information according to the head network composed of two convolutional layers, and the size of the channel heat map is the same as the size of the feature map composed of the image feature information.

[0091] In a specific implementation, a lane line detection device predicts the starting point of the lane line according to the image feature information to obtain the starting point of the lane line.

[0092] Step S205: obtaining a convolutional layer head network, and performing mapping conversion on the image feature information according to the head network to obtain a probability map for estimating the position of the lane line vanishing point.

[0093] It should be understood that the probability map refers to the image feature information mapped by the head network information. The value corresponding to each point in the probability map represents the probability that the position belongs to the lane line point. For example, there are two points A and B in the probability map, and the coordinates of point A are (x a ,y a ), the coordinates of point B are (x b ,y b), it can be seen from the probability map that the probability of point A belonging to the lane line point is 60%, and the probability of point B belonging to the lane line point is 95%. The mapping conversion refers to mapping the image feature information on the image after obtaining the image feature information to obtain a probability map for estimating the position of the lane line vanishing point, which can intuitively obtain the information in various probability maps.

[0094] In a specific implementation, a lane line detection device obtains a convolutional layer head network, maps and transforms the image feature information according to the head network, and obtains a probability map for estimating the position of the lane line vanishing point.

[0095] Step S206: predicting the end point of the lane line according to the probability map and the vanishing point.

[0096] It can be understood that the prediction of the end point is specifically to scan the probability map, obtain the width occupied by the current lane line point, and obtain the width occupied by the next lane line point corresponding to the scanned probability map. If the width occupied by the current lane line point is greater than half of the width occupied by the next lane line point, then according to the width of the current line and the vanishing point, the line where the lane line vanishing point is located and its width are obtained, and the line of the lane line vanishing point is divided into L-1 equal parts to predict the end point of the lane line. For example, if the width occupied by the current lane line point is greater than half of the width occupied by the next lane line point, then according to the width of the current line and the vanishing point, the line where the lane line vanishing point is located and its width are obtained, and the line of the lane line vanishing point is divided into L-1 equal parts to predict the end point of the lane line. For example, the value corresponding to each point in the probability map represents the probability that the position belongs to the lane line point. By selecting an appropriate probability threshold (the value can be set to 0.65), we can obtain the lane line point map corresponding to the input image. Then, we start scanning the lane point map from the first row. Here we define the width between the first and last lane points as the width of the lane point of the i-th row. After finding the row with lane points, we assume that this row is the row where the lane vanishes. For robustness and noise elimination, we compare the width of the lane points of this row with the width of the lane points of the next row. If the width of the lane points of this row is less than half of the width of its lane points, then its next row is set as the row where the lane vanishes, and its lane points are compared with the width of the lane points of the next row. Repeat this process until the predicted lane end point is generated, which is the first lane point of the row where the lane vanishes.

[0097] In a specific implementation, a lane line detection device obtains the end point of the lane line according to the probability map and the vanishing point.

[0098] In this embodiment, the starting point of the lane line is predicted according to the image feature information to obtain the starting point of the lane line; a convolutional layer head network is obtained, and the image feature information is mapped and transformed according to the head network to obtain a probability map for estimating the position of the vanishing point of the lane line; and the end point of the lane line is obtained according to the probability map and the vanishing point. The starting point of the lane line is predicted according to the image feature information to obtain the starting point of the lane line, a convolutional layer head network is obtained, and the image feature information is mapped and transformed according to the head network to obtain a probability map for estimating the position of the vanishing point of the lane line, and the probability map is scanned to obtain the width of the corresponding current row, and the width of the next row corresponding to the scanned probability map is obtained, and the width of the current row is compared with the width of the next row. If the width of the current row is greater than half of the width of the next row, the row where the vanishing point of the lane line is located is obtained according to the width of the current row and the vanishing point, and the row where the vanishing point of the lane line is divided into L-1 equal parts, thereby predicting the end point of the lane line.

[0099] In addition, an embodiment of the present invention further proposes a storage medium, on which a lane line detection program is stored. When the lane line detection program is executed by a processor, the steps of the lane line detection method described above are implemented.

[0100] Since the storage medium adopts all the technical solutions of all the above embodiments, it has at least all the beneficial effects brought by the technical solutions of the above embodiments, which will not be described one by one here.

[0101] In addition, refer to Figure 7 The embodiment of the present invention further provides a lane line detection device, the lane line detection device comprising:

[0102] The feature extraction module 10 is used to obtain image information in front of the target vehicle and perform feature extraction on the image information to obtain corresponding image feature information.

[0103] It should be understood that the image information obtained in the forward direction of the target vehicle refers to the image information collected by a preset acquisition device during the driving process of the target vehicle. The image information can be the ground information passed by the target vehicle, or the vegetation information within the line of sight of the target vehicle, or other information. All image information collected by the preset acquisition device is the image information in the forward direction of the target vehicle. The preset acquisition device can be a driving recorder, an image acquisition recorder, and other devices with a shooting function, and this embodiment does not limit this.

[0104] It can be understood that the image feature information refers to the information obtained by extracting features from the image information. When extracting features from the image information, a preset backbone network model is required. The preset backbone network model is a fully convolutional neural network model, which can be a deep residual network model (Residual Network, ResNet), or an hourglass network model (Hourglass Network, Hourglass), or a deep fusion network model (DeepLayer Aggregation, DLA). This embodiment does not limit this. In this embodiment, the deep residual network model is taken as an example for explanation. The function of the deep residual network model is to take image information as input, process it through the deep residual network model, and output image feature information for use in predicting the initial point and end point of the subsequent lane line.

[0105] In a specific implementation, a lane line detection device obtains image information in front of a target vehicle, and performs feature extraction on the image information to obtain corresponding image feature information.

[0106] The prediction module 20 is used to predict the starting point and the end point of the lane line according to the image feature information to obtain the starting point and the end point of the lane line.

[0107] It should be understood that the starting point of the lane line is obtained by predicting and analyzing the heat map according to the preset key point prediction strategy. The preset key point prediction strategy refers to a key point prediction strategy based on a heat map. Specifically, the preset key point prediction strategy is used to convert the image feature information into a heat map for key point detection, and then the starting point coordinates of the lane line are obtained according to the peak value of the heat map.

[0108] In a specific implementation, a lane line detection device predicts the starting point and the end point of the lane line according to the image feature information to obtain the starting point and the end point of the lane line.

[0109] The generating module 30 is used to generate an initial lane line according to the starting point and the end point.

[0110] It should be understood that the initial lane line refers to the lane line obtained by connecting the initial point and the end point, and in order to prevent the initial lane line from intersecting during the process of generating the lane line, when obtaining the starting point of the lane line, it is necessary to sort the obtained starting points in counterclockwise order so that the generation of the initial lane line can be carried out in an orderly manner without the occurrence of lane line intersection.

[0111] In a specific implementation, the lane line detection device generates an initial lane line according to the starting point and the end point.

[0112] The adjustment module 40 is used to predict the offset between the initial lane line and the target lane line, and adjust the coordinates of the initial lane line according to the predicted offset to obtain the coordinates of the predicted lane line to achieve detection of the lane line in front of the target vehicle.

[0113] It should be understood that the offset between the predicted initial lane line and the target lane line is calculated by the LDNet algorithm based on the image feature information. After obtaining the offset between the initial lane line and the target lane line, the coordinates of the initial lane line are adjusted according to the predicted offset to obtain the coordinates of the predicted lane line to achieve detection of the lane line. For example, the offset calculated by the LDNet algorithm based on the image feature information is A, the value corresponding to the offset A is (Δx, Δy), the initial lane line is B, and the coordinates corresponding to the initial lane line point are (x, y), then the coordinates corresponding to the detected lane line point are (x+Δx, y+Δy).

[0114] It can be understood that the LDNet algorithm is divided into three parts, namely, feature learning block, fusion block and prediction block. The input is the position of the initial lane line and the image feature information extracted by the backbone network, and the output is the offset of each point pointing to the boundary of the target lane line. Specifically, after obtaining the initial lane line, a lane line point is selected on the initial lane line, and a corresponding feature vector is constructed according to the selected lane line point. The input feature is the concatenation of the image feature corresponding to the point and the coordinates of the point. The offset between the initial lane line and the target lane line is predicted based on the input feature vector. The coordinates of the initial lane line are adjusted according to the predicted offset to obtain the coordinates of the predicted lane line.

[0115] In a specific implementation, the lane line detection device predicts the offset between the initial lane line and the target lane line, adjusts the coordinates of the initial lane line according to the predicted offset, and obtains the coordinates of the predicted lane line to realize the detection of the lane line in front of the target vehicle.

[0116] This embodiment obtains the image information in front of the target vehicle and extracts the features of the image information to obtain the corresponding image feature information; predicts the starting point and the end point of the lane line according to the image feature information to obtain the starting point and the end point of the lane line; generates an initial lane line according to the starting point and the end point; predicts the offset between the initial lane line and the target lane line, adjusts the coordinates of the initial lane line according to the predicted offset, and obtains the coordinates of the predicted lane line to detect the lane line in front of the target vehicle. The present invention obtains the image information in front of the target vehicle during the driving process, extracts the features of the image information to obtain the corresponding feature image information, obtains the starting point and the end point of the lane line according to the image feature information, connects the starting point and the end point to generate the corresponding initial lane line, predicts the offset between the initial lane line and the target lane line, adjusts the coordinates of the initial lane line according to the predicted offset, and obtains the coordinates of the predicted lane line to detect the lane line in front of the target vehicle, so that the deviation between the initial lane line and the target lane line can be predicted, thereby effectively improving the efficiency and accuracy of lane line detection.

[0117] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of the present invention. In practical applications, technicians in this field can select part or all of them according to actual needs to achieve the purpose of the present embodiment, and no limitation is made here.

[0118] In addition, for technical details not fully described in this embodiment, reference can be made to the lane line detection method provided in any embodiment of the present invention, and will not be repeated here.

[0119] In one embodiment, the extraction module 10 is also used to obtain a preset acquisition device deployed on the target vehicle, and to collect image information in front of the target vehicle according to the preset acquisition device; to obtain a preset backbone network model, and to perform feature extraction on the collected image information according to the preset backbone network model to obtain corresponding image feature information.

[0120] In one embodiment, the prediction module 20 is also used to obtain a convolutional layer head network, perform channel conversion on the image feature information according to the head network, and obtain a heat map for key point detection; analyze the heat map according to the key point preset strategy to obtain the peak point of the heat map, and use the coordinates of the peak point as the starting point of the lane line; predict the end point of the lane line according to the image feature information to obtain the starting point and end point of the lane line.

[0121] In one embodiment, the prediction module 20 is also used to select the coordinates of the peak points of the heat map to obtain the coordinates of the corresponding peak points; obtain a preset threshold to determine whether the score of the peak coordinates is greater than the preset threshold; if the score of the peak coordinates is greater than the preset threshold, execute the step of using the coordinates of the peak point as the starting point of the lane line; based on the number of detected lane line starting points, determine the number L of predicted lane lines.

[0122] In one embodiment, the prediction module 20 is further used to predict the starting point of the lane line according to the image feature information to obtain the starting point of the lane line; obtain a convolutional layer head network, map and transform the image feature information according to the head network, and obtain a probability map for estimating the position of the vanishing point of the lane line; and obtain the end point of the lane line according to the probability map and the vanishing point.

[0123] In one embodiment, the prediction module 20 is further used to scan the probability map to obtain the width occupied by the current lane line point; obtain the width occupied by the next row of lane line points corresponding to the scanned probability map, and if the width of the current row is greater than half of the width of the next row, then obtain the row where the lane line vanishing point is located and its width based on the width of the current row and the vanishing point. Divide the row of the lane line vanishing point into L-1 equal parts to predict the end point of the lane line.

[0124] In one embodiment, the adjustment module 40 is also used to obtain image feature information and LDNet algorithm. Based on the image feature information, LDNet predicts the offset between the initial lane line and its corresponding target lane line; the coordinates of the initial lane line are adjusted according to the predicted offset to obtain the coordinates of the predicted lane line, so as to realize the detection of the lane line in front of the target vehicle.

[0125] Other embodiments of the lane line detection device of the present invention or implementation methods thereof may refer to the above-mentioned method embodiments, which will not be repeated here.

[0126] In addition, it should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0127] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0128] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory (ROM) / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0129] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A lane line detection method, characterized in that: The lane line detection method comprises the following steps: Acquire image information in front of the target vehicle, and perform feature extraction on the image information to obtain corresponding image feature information; Predicting the starting point and the end point of the lane line according to the image feature information to obtain the starting point and the end point of the lane line; Generate an initial lane line according to the starting point and the end point; Predicting an offset between an initial lane line and a target lane line, adjusting the coordinates of the initial lane line according to the predicted offset, and obtaining the coordinates of the predicted lane line to detect the lane line in front of the target vehicle; The method predicts an offset between the initial lane line and the target lane line, and adjusts the coordinates of the initial lane line according to the predicted offset to obtain the coordinates of the predicted lane line, so as to detect the lane line in front of the target vehicle, including: Selecting lane line points on the initial lane line, and constructing corresponding feature vectors according to the lane line points; Acquire the image feature corresponding to the lane line point according to the image feature information and the feature vector; Concatenating the image features and the coordinates of the lane line points into input features; Based on the LDNet algorithm, the offset between the initial lane line and the target lane line is predicted according to the input features; wherein the LDNet algorithm is divided into three parts, namely, a feature learning block, a fusion block and a prediction block; The coordinates of the initial lane line are adjusted according to the predicted offset to obtain the coordinates of the predicted lane line, so as to detect the lane line in front of the target vehicle.

2. The lane line detection method according to claim 1, characterized in that: The step of acquiring image information in front of the target vehicle and performing feature extraction on the image information to obtain corresponding image feature information includes: Acquire a preset acquisition device arranged on the target vehicle, and acquire image information in front of the target vehicle according to the preset acquisition device; A preset backbone network model is obtained, and features are extracted from the collected image information according to the preset backbone network model to obtain corresponding image feature information.

3. The lane line detection method according to claim 1, characterized in that: Predicting the starting point and the end point of the lane line according to the image feature information to obtain the starting point and the end point of the lane line includes: Obtaining a convolutional layer head network, performing channel conversion on the image feature information according to the head network, and obtaining a heat map for key point detection; Analyze the heat map according to a preset key point prediction strategy to obtain a peak point of the heat map, and use the coordinates of the peak point as the starting point of the lane line; The end point of the lane line is predicted according to the image feature information to obtain the end point of the lane line.

4. The lane line detection method according to claim 3, characterized in that: Before taking the coordinates of the peak point as the starting point of the lane line, the method further includes: Selecting the coordinates of the peak points of the heat map to obtain the coordinates of the corresponding peak points; Obtain a preset threshold value, and determine whether the score of the coordinates of the peak point is greater than the preset threshold value; If the score of the coordinates of the peak point is greater than a preset threshold, executing the step of using the coordinates of the peak point as the starting point of the lane line; Based on the number of detected lane line starting points, the number of predicted lane lines L is determined.

5. The lane line detection method according to claim 1, characterized in that: The predicting the starting point and the end point of the lane line according to the image feature information to obtain the starting point and the end point of the lane line includes: Predicting the starting point of the lane line according to the image feature information to obtain the starting point of the lane line; Obtaining a convolutional layer head network, and performing mapping conversion on the image feature information according to the head network to obtain a probability map for estimating the position of the lane line vanishing point; An end point of the lane line is predicted according to the probability map and the vanishing point.

6. The lane line detection method according to claim 5, characterized in that: The predicting the end point of the lane line according to the probability map and the vanishing point includes: Scan the probability map to obtain the width occupied by the current lane line point; Obtaining the width of the next lane line point corresponding to the scanned probability map, if the width of the current lane line point is greater than half of the width of the next lane line point, obtaining the row where the lane line vanishing point is located and its width according to the width of the current row and the vanishing point; The row of the lane line vanishing point is divided into L-1 equal parts, and the end point of the lane line is predicted.

7. A lane line detection device, characterized in that: The lane line detection device comprises: A feature extraction module is used to obtain image information in front of the target vehicle and perform feature extraction on the image information to obtain corresponding image feature information; A prediction module, used to predict the starting point and the end point of the lane line according to the image feature information to obtain the starting point and the end point of the lane line; A generating module, used for generating an initial lane line according to the starting point and the end point; An adjustment module is used to predict the offset between the initial lane line and the target lane line, and adjust the coordinates of the initial lane line according to the predicted offset to obtain the coordinates of the predicted lane line, so as to realize the detection of the lane line in front of the target vehicle; The adjustment module is also used to select lane line points on the initial lane line and construct corresponding feature vectors based on the lane line points; obtain image features corresponding to the lane line points based on image characteristic information and the feature vectors; connect the image features and the coordinates of the lane line points in series as input features; based on the LDNet algorithm, predict the offset between the initial lane line and the target lane line based on the input features; wherein the LDNet algorithm is divided into three parts, namely, a feature learning block, a fusion block and a prediction block; the coordinates of the initial lane line are adjusted according to the predicted offset to obtain the coordinates of the predicted lane line, so as to realize the detection of the lane line in front of the target vehicle.

8. A lane line detection device, characterized in that: The lane line detection device includes: a memory, a processor, and a lane line detection program stored in the memory and executable on the processor, wherein the lane line detection program is configured with steps for implementing the lane line detection method according to any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium stores a lane line detection program, and when the lane line detection program is executed by the processor, the steps of the lane line detection method according to any one of claims 1 to 6 are implemented.

Images