A lane line detection method, module, and medium

Abstract

This specification discloses a lane line detection method, module, and medium. The method includes: acquiring a target image containing lane lines; inputting the target image into a lane line key point detection model to obtain key point information of the lane lines in the target image; the lane line key point detection model is trained based on sample images, where key points of the lane lines are marked at equal intervals along the fitted lane line curve in the sample images; and performing clustering processing on the key point information to obtain the lane lines contained in the target image. This embodiment trains the lane line detection model based on sample images marked with key points of the lane lines. The lane line detection model can classify only the marked key points, resulting in faster detection speed compared to detection models that classify every pixel in the sample images.

Description

Technical Field

[0001] This application relates to the field of lane line recognition technology, and in particular to a lane line detection method, module and medium. Background Technology

[0002] In the field of computer vision, there is a wealth of research literature on lane detection, which can be widely applied to road recognition systems in the fields of ADAS (Advanced Driving Assistance System) and autonomous driving.

[0003] Traditional methods for detecting lane lines use manually designed features to extract low-level lane line features, such as shape, color, or texture, and then combine them using Hough transform or Kalman filter to generate lane lines. However, this method is easily affected by weather, road conditions, occlusion, and other factors.

[0004] Lane detection based on deep neural network models has become a mainstream research direction in recent years, such as semantic segmentation-based methods. Semantic segmentation methods use deep neural networks to classify each pixel in an image as either a lane line or not, then cluster the pixels classified as lane lines into multiple lane lines, and finally refine and fit the data to identify the lane lines. However, this method classifies every pixel in the image, calculating a large number of points unrelated to lane lines, resulting in high computational cost and slow lane line detection speed. Summary of the Invention

[0005] This specification provides a lane line detection method, module, and medium to solve the problem of slow detection speed in existing lane line detection methods.

[0006] To solve the above-mentioned technical problems, the embodiments in this specification are implemented as follows:

[0007] This specification provides an embodiment of a lane line detection method, including:

[0008] Acquire the image to be detected, which includes lane lines;

[0009] The image to be detected is input into the lane line key point detection model to obtain the key point information of the lane line in the image to be detected. The lane line key point detection model is trained based on sample images, and the key points of the lane line are marked at equal intervals along the fitted lane line curve in the sample images.

[0010] Clustering is performed on the key point information to obtain the lane lines contained in the image to be detected.

[0011] Optionally, the image to be detected includes the original image acquired by the acquisition device;

[0012] Alternatively, the image to be detected includes an original image acquired by the acquisition device and a vanishing point region image; the vanishing point region image is obtained by cropping the original image based on the vanishing points of the lane lines in the original image.

[0013] Optionally, the lane line key point detection model includes a key point location detection head and a key point clustering information detection head; the key point information includes key point location information obtained based on the key point location detection head and key point clustering information obtained based on the key point clustering information detection head;

[0014] The step of clustering the key point information to obtain the lane lines contained in the image to be detected specifically includes:

[0015] Based on the key point location information and the key point clustering information, the clustering information values ​​of the key points of the lane lines in the image to be detected are obtained;

[0016] Clustering is performed on the clustering information values ​​to obtain the lane lines contained in the image to be detected.

[0017] Optionally, the lane line key point detection model further includes a key point position adjustment detection head; the key point information also includes key point position adjustment information obtained based on the key point position adjustment detection head;

[0018] The step of obtaining the clustering information values ​​of the key points of the lane lines in the image to be detected based on the key point location information and the key point clustering information specifically includes:

[0019] Based on the key point position adjustment information, the key point position information is adjusted to obtain the adjusted key point position information;

[0020] Based on the adjusted key point location information and the key point clustering information, the clustering information value of the key points of the lane lines in the image to be detected is obtained.

[0021] Optionally, the lane line key point detection model includes a key point attribute detection head; the key point information includes key point attribute information obtained based on the key point attribute detection head.

[0022] The method further includes:

[0023] Based on the key point attribute information, the attributes of the lane lines contained in the image to be detected are determined.

[0024] Optionally, the method further includes:

[0025] Acquire historical images containing lane lines collected by the acquisition device;

[0026] The key points of the lane lines in the historical image are marked at equal intervals along the fitted lane line curve to obtain the sample image;

[0027] Based on the sample images, a preset lane line key point detection model is trained to obtain the lane line key point detection model.

[0028] Optionally, the step of labeling key points of the lane lines in the historical image at equal intervals along the fitted lane line curve to obtain the sample image specifically includes:

[0029] Based on the vanishing points of lane lines in the historical image, the historical image is cropped to obtain a vanishing point region image;

[0030] The vanishing point region image is stitched together with the historical image to obtain the processed historical image;

[0031] The key points of the lane lines in the processed historical image are marked at equal intervals along the fitted lane line curve to obtain the sample image.

[0032] Optionally, the preset lane line key point detection model includes a backbone network, a preset key point location detection head, and a preset key point clustering information detection head; the sample image includes first annotation information for representing key point location information and second annotation information for representing key point clustering information;

[0033] The process of training a preset lane line key point detection model based on the sample images to obtain the lane line key point detection model specifically includes:

[0034] The sample image is input into the backbone network to obtain a first recognition result representing key point location information and a second recognition result representing key point clustering information, which are extracted from the sample image by the backbone network.

[0035] The Gaussian kernel of the first recognition result is taken as the ground value, and the preset key point location detection head is trained using the FocalLoss loss function.

[0036] Using the second identification result as the true value, the preset key point clustering information detection head is trained using the Discriminative Loss function.

[0037] Optionally, the preset lane line key point detection model further includes a preset key point attribute detection head; the sample image also includes third annotation information for representing key point attribute information;

[0038] The step of training a preset lane line key point detection model based on the sample images to obtain the trained lane line key point detection model specifically includes:

[0039] The sample image is input into the backbone network to obtain a third recognition result, which is extracted from the sample image and used to represent key point attribute information.

[0040] The third recognition result is used as the ground truth, and the preset key point attribute detection head is trained using the cross-entropy loss function.

[0041] Optionally, the preset lane line key point detection model further includes a preset key point position adjustment detection head; the sample image also includes fourth annotation information for representing key point position adjustment information;

[0042] The process of training a preset lane line key point detection model based on the sample images to obtain the lane line key point detection model specifically includes:

[0043] The sample image is input into the backbone network to obtain a fourth recognition result, which is extracted from the sample image and used to represent the key point position adjustment information.

[0044] The fourth identification result is used as the true value, and the detection head is trained using a regression loss function to adjust the preset key point positions.

[0045] This specification provides an embodiment of a lane line detection module, comprising:

[0046] An acquisition unit is used to acquire an image to be detected that includes lane lines;

[0047] An image input unit is used to input the image to be detected into a lane line key point detection model to obtain the key point information of the lane line in the image to be detected; the lane line key point detection model is trained based on sample images, and the key points of the lane line are marked at equal intervals along the fitted lane line curve in the sample images.

[0048] The processing unit is used to perform clustering processing on the key point information to obtain the lane lines contained in the image to be detected.

[0049] This specification provides an embodiment of a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the steps of the method described above.

[0050] One embodiment of this specification achieves the following beneficial effects: A lane line key point detection model is trained using sample images annotated with each key point of the lane line. The trained lane line detection model can classify only the annotated key points, resulting in faster detection speed compared to existing detection models that classify every pixel in the sample image. Furthermore, this embodiment annotates each key point of the lane line in the sample image at equal intervals along the fitted lane line curve, resulting in higher lane line detection accuracy compared to existing techniques that annotate key points based on the direction of the road where the lane line is located or a direction perpendicular to the road. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 A flowchart of a lane line detection method provided in the embodiments of this specification;

[0053] Figure 2 This is a schematic diagram of key points for directional labeling based on the road perpendicular to the lane lines, provided as an embodiment of this specification.

[0054] Figure 3 This is a schematic diagram showing the equally spaced annotations of key points provided in the embodiments of this specification;

[0055] Figure 4 A schematic diagram of a historical image with key points marked, provided for an embodiment of this specification;

[0056] Figure 5 This is a schematic diagram of historical images containing vanishing points provided in the embodiments of this specification;

[0057] Figure 6 A schematic diagram of the processed historical image provided in the embodiments of this specification;

[0058] Figure 7 A structural diagram of a preset lane line key point detection model provided in the embodiments of this specification;

[0059] Figure 8 A schematic diagram of the key point heat map provided in the embodiments of this specification;

[0060] Figure 9 A schematic diagram showing the fusion of clustering information values ​​of key points provided in the embodiments of this specification into the same coordinate system;

[0061] Figure 10 This is a schematic diagram illustrating the clustering of clustering information values ​​provided in the embodiments of this specification;

[0062] Figure 11 This is a schematic diagram of lane line detection results provided in the embodiments of this specification;

[0063] Figure 12 This is a schematic diagram illustrating the key point position adjustment information provided in the embodiments of this specification;

[0064] Figure 13 This is a structural diagram of a lane line detection module provided in an embodiment of this specification;

[0065] Figure 14 This is a structural diagram of a computer device provided as an embodiment of this specification. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of one or more embodiments of this specification clearer, the technical solutions of one or more embodiments of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of one or more embodiments of this specification.

[0067] The technical solutions provided in the various embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0068] Traditional methods for detecting lane lines use manually designed features to extract low-level lane line features, such as shape, color, or texture, and then combine them using Hough transform or Kalman filter to generate lane lines. However, this method is easily affected by weather, road conditions, occlusion, and other factors.

[0069] Lane detection based on deep neural network models has become a mainstream research direction in recent years, such as semantic segmentation-based methods. Semantic segmentation methods use deep neural networks to classify each pixel in an image as either a lane line or not, then cluster the pixels classified as lane lines into multiple lane lines, and finally refine and fit the data to identify the lane lines. However, this method classifies every pixel in the image, calculating a large number of points unrelated to lane lines, resulting in high computational cost and slow lane line detection speed.

[0070] To address the shortcomings of existing technologies, this solution provides the following embodiments:

[0071] Figure 1This is a flowchart illustrating a lane line detection method provided in an embodiment of this specification. From a programming perspective, the execution entity of the process can be a program hosted on an application server or an application client. Figure 1 As shown, the method may include the following steps:

[0072] Step 102: Obtain the image to be detected, which includes lane lines.

[0073] In step 102, an image can be acquired using an acquisition device to obtain the image to be detected. The acquisition device can be a vehicle-mounted camera, such as a vehicle-mounted monocular camera.

[0074] In step 102, the image to be detected contains lane lines.

[0075] Step 104: Input the image to be detected into the lane line key point detection model to obtain the key point information of the lane line in the image to be detected; the lane line key point detection model is trained based on sample images, and the key points of the lane line are marked at equal intervals along the fitted lane line curve in the sample images.

[0076] In step 104, the lane line key point detection model can detect key points of lane lines in the sample image to be detected, thereby obtaining the key point information of lane lines in the image. The training process of the lane line key point detection model may include:

[0077] Acquire historical images containing lane lines collected by the acquisition device;

[0078] The key points of the lane lines in the historical image are marked at equal intervals along the fitted lane line curve to obtain the sample image;

[0079] Based on the sample images, a preset lane line key point detection model is trained to obtain the lane line key point detection model.

[0080] In step 104, the acquisition device can be a vehicle-mounted camera, which acquires historical images containing lane lines.

[0081] In step 104, key points of lane lines in historical images are labeled, which can be based on the direction of the road where the lane lines are located or the direction perpendicular to the road where the lane lines are located.

[0082] Figure 2 This is a schematic diagram of key points for directional annotation based on the road perpendicular to the lane lines, provided in the embodiments of this specification. Figure 2 As shown, the road is divided into equal intervals along a direction perpendicular to the lane lines, where... Figure 2 The dashed lines in the diagram represent road dividing lines, and the points where the lane lines lie on the road dividing lines are marked as key points.

[0083] However, in existing technologies, key points are labeled based on the direction perpendicular to the road where the lane line is located. This results in the labeled key points not being able to reflect the shape of the lateral lane line well, that is, not being able to reflect the shape of the lane line with a small angle relative to the road direction well. This leads to poor detection accuracy of the model for the lateral lane line.

[0084] Similarly, based on the direction annotation key points of the road where the lane line is located, the shape of the longitudinal lane line cannot be well reflected, that is, the shape of the lane line with a large angle relative to the road direction cannot be well reflected, which leads to poor detection accuracy of the longitudinal lane line by the model.

[0085] Therefore, in this embodiment, key points of the lane line can also be marked at equal intervals along the fitted lane line curve. Figure 3 This is a schematic diagram illustrating the equally spaced annotations of key points provided in the embodiments of this specification, such as... Figure 3 As shown, key points are marked at equal intervals along the fitted lane line curve. The absolute distance between any two adjacent key points can be fixed, or the lane line length between any two adjacent key points can be fixed; no restrictions are imposed here. Marking key points along the fitted lane line curve at equal intervals can effectively reflect the shape of the lateral and longitudinal lane lines, thereby improving the model's accuracy in detecting lane lines.

[0086] In addition, when labeling keypoints, it is also necessary to label them with first labeling information to indicate their location, second labeling information to indicate their clustering, and third labeling information to indicate their attributes. Keypoint location information indicates the location of the keypoint; keypoint clustering information indicates the clustering of the keypoints, i.e., the keypoint category, such as the lane number to which the keypoint belongs; and keypoint attribute information indicates the keypoint's attributes, such as dashed, solid, yellow, or white. If the keypoint attribute is dashed, the lane line containing that keypoint can be a dashed line; if the keypoint attribute is yellow, the lane line containing that keypoint can be a yellow line.

[0087] Figure 4 This is a schematic diagram of a historical image with key points annotated, provided in the embodiments of this specification. Figure 4 As shown, Figure 4 Each key point can contain key point location information, key point clustering information, and key point attribute information.

[0088] In one embodiment, the image with labeled key points can also be preprocessed, such as by random rotation or Gaussian noise, to improve the detection accuracy of the trained lane line key point detection model.

[0089] Furthermore, in this embodiment, the image captured by the vehicle-mounted camera is from the vehicle's perspective, so the lane lines in the image may have vanishing points. Geometrically, the vanishing point of a straight line is the intersection of a line passing through the center of the image sensor and parallel to that line with the image plane. In this embodiment, the vanishing point of a lane line is the intersection of a line passing through the center of the image sensor and parallel to that lane line with the image plane. To further explain the vanishing point simply, it is the end of the lane line in the image. Figure 5 This is a schematic diagram showing historical images containing vanishing points provided in the embodiments of this specification. Figure 5 The vanishing point in the equation is the end of the lane line.

[0090] Since the vanishing point is the end of the lane lines in the image, the model may have difficulty detecting lane lines at this location. Therefore, to improve the model's detection accuracy of lane lines at the vanishing point, this embodiment processes historical images before obtaining sample images. Specifically, this processing may include:

[0091] Based on the vanishing points of lane lines in the historical image, the historical image is cropped to obtain a vanishing point region image;

[0092] The vanishing point region image is stitched together with the historical image to obtain the processed historical image.

[0093] Figure 6 This is a schematic diagram of the processed historical image provided in the embodiments of this specification, such as... Figure 6 As shown, Figure 6 It includes stitched images of the vanishing point region and historical images. The vanishing points in the images can be obtained manually or determined using methods such as Hough line fitting; no restrictions are imposed here.

[0094] Since the lane detection model is trained on an image containing vanishing point regions, the image to be detected by the lane detection model can include not only the original image captured by the acquisition device, but also the vanishing point region image in the original image.

[0095] Figure 7 A structural diagram of the preset lane line key point detection model provided in the embodiments of this specification, such as... Figure 7 As shown, the preset lane line key point detection model can include a backbone network and a head network. The backbone network is responsible for extracting features from the image. The head network is responsible for acquiring the output content, using the features extracted by the backbone network to make predictions.

[0096] The head network may include a pre-defined keypoint location detection head and a pre-defined keypoint clustering information detection head. The training process of the pre-defined lane line keypoint detection model may include:

[0097] The sample image is input into the backbone network to obtain a first recognition result representing key point location information and a second recognition result representing key point clustering information, which are extracted from the sample image by the backbone network.

[0098] The Gaussian kernel of the first recognition result is taken as the ground value, and the preset key point location detection head is trained using the FocalLoss loss function.

[0099] Using the second identification result as the true value, the preset key point clustering information detection head is trained using the Discriminative Loss function.

[0100] In existing technologies, the lane line category and lane line attributes are coupled together. Therefore, different lane line attributes can cause the detected lane lines to be disconnected.

[0101] To avoid this situation, the preset lane line key point detection model in this embodiment may further include a preset key point attribute detection head, and the key point attribute detection head and the key point clustering information detection head are decoupled. In this case, the training process of the preset lane line key point detection model may further include:

[0102] The sample image is input into the backbone network to obtain a third recognition result, which is extracted from the sample image and used to represent key point attribute information.

[0103] The third recognition result is used as the ground truth, and the preset key point attribute detection head is trained using the cross-entropy loss function.

[0104] In this embodiment, the size of the historical image and the input image size of the preset lane line key point detection model may differ. For example, if the historical image size is 1920*1080, while the input image size of the preset lane line key point detection model is 512*628, then the 1920*1080 historical image needs to be compressed to 512*628 before being input into the preset lane line key point detection model. Similarly, when processing lane lines at vanishing points in the historical image is required, the historical image is also compressed. For instance, if the historical image size is still 1920*1080, but the vanishing point region image size is 300*200, then the historical image needs to be compressed to 512*288, and the vanishing point region image size enlarged to 512*340. The two images are then stitched together into a single 512*628 image before being input into the preset lane line key point detection model.

[0105] In this embodiment, the first annotation information representing the key point location information in the sample image can be an integer based on the annotation of the historical image before size reduction. For example, the first annotation information can be (123, 234). The key point location information output by the preset lane line key point detection model can also be an integer. However, after the historical image is compressed, the first annotation information may be reduced to a decimal according to the compression ratio of the historical image, which will lead to a deviation between the detected key point location information and the first annotation information.

[0106] To eliminate this bias and improve the accuracy of key point location information, the preset lane line key point detection model may further include a preset key point location adjustment detection head, and the sample image may also include fourth annotation information to represent the key point location adjustment information. The training process of the preset lane line key point detection model may further include:

[0107] The sample image is input into the backbone network to obtain a fourth recognition result, which is extracted from the sample image and used to represent the key point position adjustment information.

[0108] The fourth identification result is used as the true value, and the detection head is trained using a regression loss function to adjust the preset key point positions.

[0109] In this embodiment, the L1Loss regression loss function or the L2Loss regression loss function can be used to train the preset key point positions to adjust the detection head, and no limitation is made here.

[0110] Furthermore, the training process for the pre-defined lane line key point detection model may also include:

[0111] Based on the outputs of the preset key point location detection head, the preset key point clustering information detection head, the preset key point attribute detection head, and the preset key point position adjustment detection head, the preset lane line key point detection model is optimized using the backpropagation algorithm to obtain the lane line key point detection model.

[0112] Finally, the lane line key point detection model is used to detect the image to be detected, and the key point information of the lane lines in the image to be detected is output.

[0113] Step 106: Perform clustering processing on the key point information to obtain the lane lines contained in the image to be detected.

[0114] Specifically, in step 106, the key point information may include key point location information and key point clustering information. Clustering the key point information to obtain the lane lines contained in the image to be detected may specifically include:

[0115] Based on key point location information and key point clustering information, the clustering information values ​​of key points of lane lines in the image to be detected are obtained.

[0116] Clustering is performed on the clustering information values ​​to obtain the lane lines contained in the image to be detected.

[0117] In this embodiment, the key point location detection head can output a key point heatmap. Figure 8 This is a schematic diagram of a key point heatmap provided in the embodiments of this specification. The key point heatmap includes key point location information. For example... Figure 8 As shown, Figure 8 The key points are divided into two rows. The top row contains six key points on the left and right, which represent the key point location information of the vanishing point region in the image to be detected output by the key point location detection head. The bottom row contains six key points on the left and right, which represent the key point location information of the image to be detected output by the key point location detection head.

[0118] Then, based on the key point heatmap, the local maximum point is selected by max pooling, and then the points that are globally greater than the preset threshold are selected from the local maximum points to obtain the key point location information.

[0119] In this embodiment, the key point clustering information detection head outputs the clustering information value of the key points. The clustering information value of the key points is a coordinate representation that maps the clustering information of the key points to a high-dimensional abstract space. The clustering information value of the key points contains the clustering information of the key points.

[0120] Then, the clustering information values ​​of each key point are fused into the same coordinate system. Figure 9 This diagram illustrates the fusion of clustering information values ​​of key points provided in the embodiments of this specification into the same coordinate system. It should be noted that... Figure 9 The key points in the text are the key points before classification.

[0121] Then, density-based clustering algorithms, such as DBSCAN (Density-Based Spatial Clustering of Applications with Noise), are used to cluster the clustering information values ​​under the same coordinate system. Figure 10 This is a schematic diagram illustrating the clustering of clustering information values ​​provided in the embodiments of this specification. Figure 10 In the game, if different colored key points can be used to represent different categories, then red key points can belong to lane 1 and green key points can belong to lane 2.

[0122] Finally, based on the key point location information, the clustering information values ​​after clustering are mapped back to the key point location information after clustering to obtain the lane lines contained in the image to be detected. Figure 11 This is a schematic diagram of lane line detection results provided in the embodiments of this specification. Figure 11 In the diagram, the three red key points in the lower left corner represent lane 1, the three red key points in the upper left corner represent the lane line at the end of lane 1, the three green key points in the lower right corner represent lane 2, and the three green key points in the upper right corner represent the lane line at the end of lane 2.

[0123] In this embodiment, the key point information may further include key point position adjustment information. The key point position adjustment information can be detected from image features extracted from the backbone network using the key point position adjustment detection head of the lane line key point detection model. Figure 12 This is a schematic diagram illustrating the key point position adjustment information provided in the embodiments of this specification, such as... Figure 12 As shown, Figure 12 Each point in the diagram can represent the position adjustment information of the corresponding key point.

[0124] The key point location information is then adjusted using the key point location adjustment information to make the key point location information more accurate, thereby obtaining more accurate clustering information values.

[0125] Furthermore, keypoint information can also include keypoint attribute information. Keypoint attribute information can be detected from image features extracted by the backbone network using the keypoint attribute detection head of the lane line keypoint detection model.

[0126] Then, based on the key point attribute information, the attributes of the lane lines contained in the image to be detected can be determined. Specifically, determining the lane line attributes can involve identifying whether the lane line is a dashed line, a solid line, a yellow lane line, a white lane line, etc.

[0127] In this embodiment, in order to obtain a curved representation of the lane line, the key points of the detected lane line can also be fitted into a curve through curve fitting, which can then be used by downstream users.

[0128] It should be understood that the order of some steps in the methods described in one or more embodiments of this specification may be interchanged according to actual needs, or some steps may be omitted or deleted.

[0129] This embodiment trains a lane line key point detection model using sample images of key points labeled with lane lines. The trained lane line detection model only classifies the labeled key points. At the same time, it uses image features extracted by the backbone network and combines them with a simple clustering method to distinguish each lane line, eliminating the need for various complicated manual post-processing. Then, the clustered key point information is backtracked to the image to be detected to achieve rapid lane line detection.

[0130] In this embodiment, key points of the lane lines in the sample image are labeled at equal intervals along the fitted lane line curve. Compared with labeling key points based on the direction of the road where the lane lines are located or the direction perpendicular to the road where the lane lines are located, the lane line detection accuracy is high.

[0131] In addition, the lane line key point detection model in this embodiment does not predict the shape of the lane line separately, but only relies on the features of the key points themselves, such as the key point location information and key point clustering information, to train the model. Thus, the lane line key point detection model in this embodiment can identify various complex lane lines, such as Y-shaped lane lines and broken lane lines.

[0132] Furthermore, this embodiment does not use complex algorithms. The lane line key point detection model only requires the most basic convolutions, such as the backbone network and the head network, which makes it compatible with most devices and will not be limited in deployment due to insufficient device computing power.

[0133] Based on the same idea, the embodiments of this specification also provide modules corresponding to the above methods. Figure 13 The embodiments provided in this specification correspond to Figure 1 A structural diagram of a lane line detection module, such as... Figure 13 As shown, this module may include:

[0134] The acquisition unit 130 is used to acquire the image to be detected, which includes lane lines.

[0135] The image input unit 132 is used to input the image to be detected into the lane line key point detection model to obtain the key point information of the lane line in the image to be detected; the lane line key point detection model is trained based on sample images, and the key points of the lane line are marked at equal intervals along the fitted lane line curve in the sample images.

[0136] The processing unit 134 is used to perform clustering processing on the key point information to obtain the lane lines contained in the image to be detected.

[0137] Optionally, the image to be detected may include an original image acquired by the acquisition device; or, the image to be detected may include an original image acquired by the acquisition device and a vanishing point region image; the vanishing point region image is obtained by cropping the original image based on the vanishing points of the lane lines in the original image.

[0138] Optionally, the lane line key point detection model includes a key point location detection head and a key point clustering information detection head; the key point information includes key point location information obtained based on the key point location detection head and key point clustering information obtained based on the key point clustering information detection head; the processing unit 134 may specifically include:

[0139] The clustering information value acquisition subunit is used to obtain the clustering information value of the key points of the lane lines in the image to be detected based on the key point location information and the key point clustering information;

[0140] The clustering subunit is used to cluster the clustering information values ​​to obtain the lane lines contained in the image to be detected.

[0141] Optionally, the lane line key point detection model may further include a key point position adjustment detection head; the key point information further includes key point position adjustment information obtained based on the key point position adjustment detection head; the clustering information value acquisition subunit may specifically include:

[0142] The location information acquisition micro-unit is used to adjust the key point location information based on the key point location adjustment information to obtain the adjusted key point location information;

[0143] The clustering information value acquisition micro-unit is used to obtain the clustering information value of the key points of the lane lines in the image to be detected based on the adjusted key point location information and the key point clustering information.

[0144] In this embodiment, the lane line key point detection model includes a key point attribute detection head; the key point information includes key point attribute information obtained based on the key point attribute detection head; the module further includes:

[0145] An attribute determination unit is used to determine the attributes of lane lines contained in the image to be detected based on the key point attribute information.

[0146] Furthermore, the module may also include:

[0147] The historical image acquisition unit is used to acquire historical images containing lane lines collected by the acquisition device;

[0148] An annotation unit is used to annotate key points of lane lines in the historical image at equal intervals to obtain the sample image;

[0149] The model acquisition unit is used to train a preset lane line key point detection model based on the sample image to obtain the lane line key point detection model.

[0150] Furthermore, the annotation unit may specifically include:

[0151] The cropping subunit is used to crop the historical image based on the vanishing points of lane lines in the historical image to obtain a vanishing point region image.

[0152] The stitching subunit is used to stitch the vanishing point region image with the historical image to obtain the processed historical image;

[0153] The annotation sub-unit is used to annotate the key points of the lane lines in the processed historical image at equal intervals along the fitted lane line curve to obtain the sample image.

[0154] Furthermore, the preset lane line key point detection model includes a backbone network, a preset key point location detection head, and a preset key point clustering information detection head; the sample image includes first annotation information for representing key point location information and second annotation information for representing key point clustering information; the model acquisition unit specifically includes:

[0155] An input subunit is used to input the sample image into the backbone network to obtain a first recognition result representing key point location information and a second recognition result representing key point clustering information, which are extracted from the sample image by the backbone network.

[0156] The first training subunit is used to take the Gaussian kernel as the ground value of the first recognition result and train the preset key point location detection head using the Focal Loss loss function.

[0157] The second training subunit is used to train the preset key point clustering information detection head using the second recognition result as the ground truth and the Discriminative Loss function.

[0158] Furthermore, the preset lane line key point detection model also includes a preset key point attribute detection head; the sample image also includes third annotation information for representing key point attribute information; the input unit is further configured to input the sample image into the backbone network to obtain a third recognition result for representing key point attribute information extracted from the sample image by the backbone network; the model acquisition unit further includes:

[0159] The third training subunit is used to train the preset key point attribute detection head using the third recognition result as the ground truth and the cross-entropy loss function.

[0160] Furthermore, the preset lane line key point detection model also includes a preset key point position adjustment detection head; the sample image also includes fourth annotation information for representing key point position adjustment information; the input unit is also used to input the sample image into the backbone network to obtain the fourth recognition result for representing key point position adjustment information extracted from the sample image by the backbone network; the model acquisition unit also includes:

[0161] The fourth training subunit is used to train the preset key point position adjustment detection head using the fourth recognition result as the ground truth and a regression loss function.

[0162] Based on the same idea, this specification also provides devices corresponding to the above methods in its embodiments. Figure 14 The embodiments provided in this specification correspond to Figure 1 A schematic diagram of the structure of a computer device. For example... Figure 14 As shown, device 140 may include memory 141, processor 142, and computer program 143 stored on memory 141. Processor 142 executes computer program 143 to implement the steps of the above method.

[0163] Based on the same idea, embodiments of this specification also provide a computer-readable storage medium corresponding to the above method, wherein computer instructions are stored on the computer-readable storage medium, and when the computer instructions are executed by a processor, the steps of the above method are implemented.

[0164] Based on the same idea, this specification also provides a computer program product corresponding to the above method. The computer program product includes computer instructions, which, when executed by a processor, implement the steps of the above method.

[0165] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on its differences from other embodiments. In particular, for... Figure 14 As the computer device shown is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0166] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0167] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1A device that provides the functions specified in one or more boxes.

[0168] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0169] This application can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0170] The above description is merely an embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of this application should be included within the scope of the claims of this application.

Claims

1. A lane line detection method characterized by, The method comprises the following steps: acquiring a to-be-detected image containing lane lines; inputting the to-be-detected image into a lane line key point detection model to obtain key point information of lane lines in the to-be-detected image; the lane line key point detection model is obtained based on sample images, and each key point of the lane lines is labeled along a fitted lane line curve at equal intervals in the sample images; performing clustering processing on the key point information to obtain lane lines contained in the to-be-detected image; the sample images are obtained in the following manner: cutting a historical image containing lane lines based on vanishing points of the lane lines in the historical image to obtain a vanishing point region image; splicing the vanishing point region image and the historical image to obtain a processed historical image; labeling each key point of the lane lines in the processed historical image along a fitted lane line curve at equal intervals to obtain the sample images.

2. The lane line detection method according to claim 1, characterized by, The to-be-detected image comprises an original image collected by a collection device. Alternatively, the to-be-detected image comprises an original image collected by a collection device and a vanishing point region image obtained by cutting the original image based on vanishing points of lane lines in the original image.

3. The lane line detection method of claim 1, wherein, The lane line key point detection model comprises a key point position detection head and a key point clustering information detection head; the key point information comprises key point position information obtained based on the key point position detection head and key point clustering information obtained based on the key point clustering information detection head; The clustering processing on the key point information to obtain lane lines contained in the to-be-detected image specifically comprises the following steps: obtaining a clustering information value of a key point of the lane lines in the to-be-detected image based on the key point position information and the key point clustering information; performing clustering on the clustering information value to obtain lane lines contained in the to-be-detected image.

4. The lane line detection method according to claim 3, characterized by, The lane line key point detection model further comprises a key point position adjustment detection head; the key point information further comprises key point position adjustment information obtained based on the key point position adjustment detection head; The obtaining of the clustering information value of the key point of the lane lines in the to-be-detected image based on the key point position information and the key point clustering information specifically comprises the following steps: adjusting the key point position information based on the key point position adjustment information to obtain adjusted key point position information; obtaining the clustering information value of the key point of the lane lines in the to-be-detected image based on the adjusted key point position information and the key point clustering information.

5. The lane line detection method according to any one of claims 1 to 4, characterized by, The lane line key point detection model comprises a key point attribute detection head; the key point information comprises key point attribute information obtained based on the key point attribute detection head; The method further comprises the following steps: determining attributes of the lane lines contained in the to-be-detected image based on the key point attribute information.

6. The lane line detection method of claim 1, wherein, The method further comprises the following steps: acquiring a historical image containing lane lines collected by a collection device; labeling each key point of the lane lines in the historical image along a fitted lane line curve at equal intervals to obtain the sample images; training a preset lane line key point detection model based on the sample images to obtain the lane line key point detection model.

7. The lane line detection method according to claim 6, characterized by, The preset lane line key point detection model comprises a backbone network, a preset key point position detection head, and a preset key point clustering information detection head; the sample image comprises first annotation information for representing key point position information and second annotation information for representing key point clustering information; The preset lane line key point detection model is trained based on the sample image to obtain the lane line key point detection model, and the training specifically comprises: The sample image is input into the backbone network to obtain first recognition results for representing key point position information and second recognition results for representing key point clustering information which are extracted from the sample image and output by the backbone network; The first recognition results are taken as true values, and the preset key point position detection head is trained by using a Focal Loss loss function; The second recognition results are taken as true values, and the preset key point clustering information detection head is trained by using a Discriminative Loss discriminative loss function.

8. The lane line detection method of claim 7, wherein, The preset lane line key point detection model further comprises a preset key point attribute detection head, and the sample image further comprises third annotation information for representing key point attribute information; The preset lane line key point detection model is trained based on the sample image to obtain the trained lane line key point detection model, and the training specifically comprises: The sample image is input into the backbone network to obtain third recognition results for representing key point attribute information which are output by the backbone network; The third recognition results are taken as true values, and the preset key point attribute detection head is trained by using a cross-entropy loss function.

9. The lane line detection method of claim 7, wherein, The preset lane line key point detection model further comprises a preset key point position adjustment detection head, and the sample image further comprises fourth annotation information for representing key point position adjustment information; The preset lane line key point detection model is trained based on the sample image to obtain the lane line key point detection model, and the training specifically comprises: The sample image is input into the backbone network to obtain fourth recognition results for representing key point position adjustment information which are output by the backbone network; The fourth recognition results are taken as true values, and the preset key point position adjustment detection head is trained by using a regression loss function.

10. A lane line detection device characterized by comprising: comprise: An acquisition unit is configured to acquire a to-be-detected image containing a lane line; An image input unit is configured to input the to-be-detected image into a lane line key point detection model to obtain key point information of the lane line in the to-be-detected image; The lane line key point detection model is trained based on a sample image, and each key point of the lane line is annotated at equal intervals along a curve of the lane line after fitting in the sample image; A processing unit is configured to perform clustering processing on the key point information to obtain a lane line contained in the to-be-detected image; The sample image is obtained in the following manner: A vanishing point region image is obtained by cutting a historical image containing a lane line based on a vanishing point of the lane line in the historical image; stitch the vanishing point region image and the historical image to obtain a processed historical image; labeling each key point of the lane line in the processed historical image along the fitted lane line curve at equal intervals to obtain the sample image.

11. A computer readable storage medium having stored thereon computer instructions, wherein, The computer instructions, when executed by a processor, implement the steps of the method of any one of claims 1 to 9.

Images