Lane line recognition and model training methods, devices, equipment, and media

Abstract

The present disclosure provides a method, apparatus, device, and medium for lane line recognition and model training, which relate to the technical fields of autonomous driving, intelligent transportation, and high-precision map technology, and particularly to the technical field of high-precision map data processing. The specific implementation solution of the lane line recognition model training method is as follows: Obtain multiple groups of training samples; wherein, each group of training samples includes a reflection value base map sample of a road and the trajectory of the acquisition device that acquires the reflection value base map sample, and the reflection value base map sample is labeled with lane line annotation information; for each group of training samples, intercept a reflection value image block from the reflection value base map sample along the trajectory of the acquisition device, and input the reflection value image block into the lane line recognition model, so that the lane line recognition model can recognize the lane line recognition result in the reflection value base map sample according to the reflection value image block; determine the loss error according to the lane line recognition result and the lane line annotation information, and adjust the model parameters of the lane line recognition model according to the loss error.

Description

Technical Field

[0001] The present disclosure relates to the technical fields of autonomous driving, intelligent transportation, and high-precision map technology, and particularly to the technical field of high-precision map data processing. Background Art

[0002] High-precision maps play an important role in the processes of perception, positioning, decision-making, and control in autonomous driving. As a basic element of high-precision maps, lane lines are crucial in the generation process of high-precision maps.

[0003] In the prior art, when generating lane lines, the obtained road images are first input into a semantic segmentation network, and lane lines are generated based on the method of the semantic segmentation network. However, since the lane lines in the road images are worn or blocked, etc., the lane lines generated based on the semantic segmentation network are discontinuous, and subsequent manual connection of the lane lines is required to generate the lane lines.

[0004] Therefore, using the existing lane line generation method, the generation process is relatively complex, resulting in a low generation efficiency of lane lines. Summary of the Invention

[0005] The present disclosure provides a method, device, equipment, and medium for lane line recognition and model training.

[0006] According to a first aspect of the present disclosure, there is provided a method for training a lane line recognition model, including:

[0007] Obtaining multiple groups of training samples; wherein each group of training samples includes a reflection value base map sample of a road and the trajectory of a collection device that collects the reflection value base map sample, and the reflection value base map sample is labeled with lane line annotation information;

[0008] For each group of training samples, intercepting a reflection value image block from the reflection value base map sample along the trajectory of the collection device, and inputting the reflection value image block into the lane line recognition model, so that the lane line recognition model recognizes the lane line recognition result in the reflection value base map sample according to the reflection value image block;

[0009] Determining a loss error according to the lane line recognition result and the lane line annotation information, and adjusting the model parameters of the lane line recognition model according to the loss error.

[0010] According to a second aspect of the present disclosure, there is provided a lane line recognition method, including:

[0011] Obtaining a reflection value base map to be recognized and the trajectory of a collection device that collects the reflection value base map;

[0012] Intercept a reflection value image block from the reflection value base map along the trajectory of the acquisition device, and input the reflection value image block into a trained lane line recognition model, so that the lane line recognition model recognizes the lane line recognition result in the reflection value base map according to the reflection value image block; wherein, the lane line recognition model is obtained by training an initial lane line recognition model based on multiple groups of training, and each group of training includes a reflection value base map of a road and the trajectory of the acquisition device that acquires the reflection value base map, and the reflection value base map is labeled with lane line annotation information.

[0013] According to a third aspect of the present disclosure, there is provided a training device for a lane line recognition model, including:

[0014] An acquisition module, configured to acquire multiple groups of training samples; wherein, each group of training samples includes a reflection value base map sample of a road and the trajectory of the acquisition device that acquires the reflection value base map sample, and the reflection value base map sample is labeled with lane line annotation information;

[0015] A preprocessing module, configured to, for each group of training samples, intercept a reflection value image block from the reflection value base map sample along the trajectory of the acquisition device, and input the reflection value image block into the lane line recognition model, so that the lane line recognition model recognizes the lane line recognition result in the reflection value base map sample according to the reflection value image block;

[0016] A training model, configured to determine a loss error according to the lane line recognition result and the lane line annotation information, and adjust the model parameters of the lane line recognition model according to the loss error.

[0017] According to a fourth aspect of the present disclosure, there is provided a lane line recognition device, including:

[0018] An acquisition module, configured to acquire a reflection value base map to be recognized and the trajectory of the acquisition device that acquires the reflection value base map;

[0019] A recognition module, configured to intercept a reflection value image block from the reflection value base map along the trajectory of the acquisition device, and input the reflection value image block into a trained lane line recognition model, so that the lane line recognition model recognizes the lane line recognition result in the reflection value base map according to the reflection value image block; wherein, the lane line recognition model is obtained by training an initial lane line recognition model based on multiple groups of training, and each group of training includes a reflection value base map of a road and the trajectory of the acquisition device that acquires the reflection value base map, and the reflection value base map is labeled with lane line annotation information.

[0020] According to a fifth aspect of the present disclosure, there is provided an electronic device, including:

[0021] At least one processor; and

[0022] A memory communicatively connected to the at least one processor; wherein,

[0023] the memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is enabled to execute the method according to any one of the above.

[0024] According to a sixth aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the method according to any one of the above.

[0025] According to a seventh aspect of the present disclosure, there is provided a computer program product including a computer program, and when the computer program is executed by a processor, the method according to any one of the above is implemented.

[0026] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The drawings are used to better understand the solution and do not constitute a limitation to the present disclosure. Among them:

[0028] Figure 1 is a flowchart of a method for training a lane line recognition model provided by an exemplary embodiment of the present disclosure;

[0029] Figure 2 is a schematic diagram of a scenario for intercepting a reflection value image block in a reflection value base map sample provided by an exemplary embodiment of the present disclosure;

[0030] Figure 3 is a network architecture diagram of a lane line recognition model provided by an exemplary embodiment of the present disclosure;

[0031] Figure 4 is a result comparison diagram of lane lines extracted by a lane line recognition model obtained by an exemplary embodiment of the present disclosure and lane lines extracted by other methods;

[0032] Figure 5 is a flowchart of a lane line recognition method provided by an exemplary embodiment of the present disclosure;

[0033] Figure 6 is a module schematic diagram of a device for training a lane line recognition model provided by an exemplary embodiment of the present disclosure;

[0034] Figure 7Schematic diagram of a lane line recognition device provided by an exemplary embodiment of the present disclosure;

[0035] Figure 8 Block diagram of an electronic device provided by an exemplary embodiment of the present disclosure. Detailed implementation manners

[0036] The following describes exemplary embodiments of the present disclosure with reference to the accompanying drawings. Various details of the embodiments of the present disclosure are included to facilitate understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0037] Figure 1 Flowchart of a method for training a lane line recognition model provided by an exemplary embodiment of the present disclosure. The method for training the lane line recognition model includes the following steps:

[0038] Step 101: Obtain multiple groups of training samples.

[0039] Each group of training samples includes a reflection value base map sample of a road and the trajectory of the acquisition device of the reflection value base map sample, and the reflection value base map sample is labeled with lane line annotation information.

[0040] The number of training samples can be determined according to actual situations. It can be understood that the more the number of training samples and the richer the road forms, the higher the accuracy and stronger the robustness of the trained lane line recognition model.

[0041] The reflection value base map sample is obtained from the point cloud data of the road. Specifically, the point cloud data is projected into a two-dimensional space to obtain a reflection value base map, and the reflection value base map is a two-dimensional grayscale map.

[0042] The point cloud data can be obtained based on the laser measurement principle and / or the photogrammetry principle. The point cloud data obtained according to the laser measurement principle includes three-dimensional coordinates (XYZ) and laser reflection information; the point cloud data obtained according to the photogrammetry principle includes three-dimensional coordinates (XYZ); combining the laser measurement and photogrammetry principles to obtain a point cloud, including three-dimensional coordinates (XYZ) and laser reflection information. Representing the point cloud data according to the reflection information in the point cloud, the reflection value base map corresponding to the point cloud data can be obtained.

[0043] Specifically, taking the laser measurement principle as an example, during the process of the acquisition device driving on the road, the lidar set on the acquisition device is used to obtain point cloud data. The point cloud data includes multiple point clouds, and each point in the point cloud contains the three-dimensional coordinate information and reflection value of the point. Among them, the reflection value is related to the material of the object; then, the point cloud data is rasterized on a two-dimensional plane; for example, it is divided into M*N grids, and each grid contains a point set. By taking the average value of the reflection values of the points in the point set corresponding to each grid, the reflection value of each grid can be obtained; then, all grids are processed to obtain a reflection value grid with a resolution of M*N. The reflection value grid with a resolution of M*N is called a reflection value base map sample.

[0044] In one embodiment, a reflection value base map sample is obtained according to the point cloud data collected by the acquisition device driving along the road once, that is, based on a single loop of point cloud data. Obtaining a reflection value base map sample based on a single loop of point cloud data has a small calculation amount.

[0045] In one embodiment, a reflection value base map sample is obtained according to multiple groups of point cloud data. The acquisition device drives back and forth along the road multiple times or drives unidirectionally along the road multiple times. During each driving process, a group of point cloud data of the road is collected, and the obtained multiple groups of point cloud data are rasterized on a two-dimensional plane to obtain a reflection value base map sample of the road.

[0046] Obtaining a reflection value base map sample based on multiple groups of point cloud data can avoid or reduce the lane line pressing deviation problem caused by the ghost problem of the spliced point cloud and the problems of lane line wear and occlusion, reduce or even eliminate the noise in the reflection value base map sample, make the features (lane line geometric information) extracted by the lane line recognition model more accurate, and thus improve the efficiency and accuracy of model training.

[0047] During the process of collecting point cloud data, the acquisition device can obtain positioning information in real time, and the trajectory of the cloud acquisition device can be generated according to this positioning information. Among them, the positioning information can be obtained through, but is not limited to, the Global Positioning System (GPS). The acquisition device can be, but is not limited to, a point cloud acquisition vehicle, a drone, etc.

[0048] Step 102: For each group of training samples, intercept a reflection value image block from the reflection value base map sample along the trajectory of the acquisition device, and input the reflection value image block into the lane line recognition model to identify the lane line recognition result in the reflection value base map sample according to the lane line recognition model.

[0049] Figure 2Schematic diagram of intercepting a reflection value image block from a reflection value base map sample provided by an exemplary embodiment of the present disclosure. The horizontal and vertical intersecting lines in the reflection value base map sample represent roads. Intercepting the reflection value image block from the reflection value base map sample according to the trajectory of the acquisition device is equivalent to preprocessing the reflection value base map sample based on the trajectory of the acquisition device to obtain multiple reflection value image blocks. A small rectangular frame in the figure represents a reflection value image block (block). The size of the reflection value image block is determined according to actual needs. Generally, the short side of the reflection value image block is greater than the road width in the image, and the long side of the reflection value image block extends in the same direction as the road.

[0050] In one implementation, the preprocessing includes trajectory clustering. Based on trajectory clustering, the approximate positions of the lane lines are found. Specifically, by clustering multiple trajectories through distance, reflection value image blocks within a certain threshold range are selected according to the center point of each cluster. Selecting reflection value image blocks through trajectory clustering can filter out invalid data and improve the accuracy and efficiency of lane line extraction.

[0051] In an embodiment, the lane line recognition model includes a feature extraction layer and an output layer. The feature extraction layer is used to extract lane line features from the reflection value image block to obtain at least two feature layers. The output layer is used to predict the lane line recognition result according to at least two feature layers. In step 102, the reflection value image block is input into the lane line recognition model, triggering the feature extraction layer to extract lane line features from the reflection value image block to obtain at least two feature layers; triggering the output layer to predict the lane line recognition result according to at least two feature layers. The number of feature layers can be set according to actual situations, and can be 2 feature layers, 3 feature layers, 4 feature maps, or even more.

[0052] In an embodiment, the network architecture of the lane line recognition model adopts the network architecture of an existing lane line recognition model. Training the network architecture of the existing lane line recognition model with reflection value image blocks eliminates the need to build the network architecture of the lane line recognition model by oneself, which is convenient to operate.

[0053] In an embodiment, the network architecture of the existing lane line recognition model is optimized, and the optimized network architecture is trained with reflection value image blocks, so that the trained lane line recognition model can recognize lane lines based on the characteristics of the reflection value base map sample, improving the accuracy of lane line recognition. Optimizing the network architecture includes: optimizing the feature layer and the output layer.

[0054] In one embodiment, the feature extraction layer includes a Deep Residual Network (ResNet) and a Transformer encoder. The feature extraction layer can not only extract the image features of each reflected value image patch, but also extract the correlation features between the reflected value image patches. The image features and the correlation features jointly represent the geometric information of the lane line. The output layer can predict the lane line recognition result based on the image features and the correlation features, which is beneficial to improving the accuracy of the lane line recognition model in recognizing lane lines from the reflected value images.

[0055] In one embodiment, the output layer can detect lane information in two directions along the feature layer. Specifically, the output layer is triggered to calculate the first lane line information along the row direction of the feature layer and the second lane line information along the column direction of the feature layer, and predict the lane line recognition result based on the first lane line information and the second lane line information. The lane line recognition result includes the vector information of the lane line.

[0056] The lane lines in the reflected value base map sample can be horizontal or vertical; the longitudinal lane line information (the first lane line information) is detected from the row direction, and the horizontal lane line (the second lane line information) is detected from the column direction. The first lane line information and the second lane line information can be calculated in parallel or sequentially, and the embodiments of the present disclosure do not make special limitations on this.

[0057] In the embodiments of the present disclosure, based on the existing lane line recognition model, an additional network structure layer is added, that is, two network structures are used to detect lane information in two directions along the feature layer respectively, and the lane line recognition result is predicted based on the lane information detected in the two directions. For example, the longitudinal lane line predicts the lane line information from the row direction, and the horizontal lane line predicts the lane line information from the column direction, so that the prediction result of the lane line recognition result is more complete and the recognition efficiency is higher.

[0058] Figure 3 It is a network architecture diagram of a lane line recognition model provided by an exemplary embodiment of the present disclosure. The feature extraction layer is used as the backbone network of the lane line recognition model, and a cascaded ResNet and Transformer encoder are adopted. The output layer includes two parts: Proposal head and Conditional shape head.

[0059] Each reflected value image patch is input into the feature extraction layer, so that the feature extraction layer extracts the image features of the reflected value image patch and the correlation features between the reflected value image patches, and generates four feature layers according to the image features and the correlation features (the number of feature layers is not limited to 4 in actual use).

[0060] The Proposal head part is used to select one feature layer from four feature layers, divide the selected feature layer into several grids (let the number of grids be Hp*Wp), and output two feature maps. One is a heatmap with a shape of 1*Hp*Wp, representing the starting point of the lane line; the other is a parameter map with a shape of Cp*Hp*Wp, corresponding to the heatmap one by one, and outputting a set of convolution parameters corresponding to the lane line instance in the grid for convolution operations in the Conditional shape head to calculate the lane line information corresponding to the grid.

[0061] The Conditional shape head part is used to solve the lane line curve (lane line recognition result). Specifically, it detects lane information in two directions along the feature layer, predicts the lane line recognition result based on the lane information detected in the two directions, and finally predicts the lane line recognition result. The final implementation process is similar to the related technology and will not be elaborated here.

[0062] In one embodiment, the trigger output layer selects the feature layer of the target layer from at least two feature layers and predicts the lane line recognition result according to the feature layer of the target layer.

[0063] See Figure 3 , the feature extraction layer outputs four feature layers. The four feature maps correspond to the high-level feature, the sub-high-level feature, the sub-low-level feature, and the low-level feature from top to bottom. The high-level feature is used to extract high-level semantic information (whether it is a lane line), and the low-level feature is used to extract accurate position information. In the prior art, generally the first feature layer (the topmost feature map) or the second feature layer (the second feature map) is used for subsequent processing. The high-level feature is more suitable for the scenario of lane line recognition based on visual images and not suitable for the reflected value base map. Therefore, in the embodiments of the present disclosure, the lower-level feature layer is selected to predict the lane line recognition result, generally the feature layer with more accurate represented position information, which can be Figure 3 the third feature layer in

[0064] In the embodiments of the present disclosure, based on the characteristics of the reflected value base map, the feature layer of the target layer is selected to predict the lane line recognition result, so that the prediction result of the lane line recognition result is more accurate.

[0065] Step 103: Determine the loss error according to the lane line recognition result and the lane line annotation information, and adjust the model parameters of the lane line recognition model according to the loss error.

[0066] Iteratively train the lane line recognition model until the iteration stop condition is reached. The trained lane line recognition model can be used to recognize lane lines in the reflection value image. Among them, the iteration stop condition can include, but is not limited to, that the loss error is less than the error threshold or the number of iterations reaches the number threshold. The error threshold and the number threshold can be set according to the actual situation.

[0067] In the embodiments of the present disclosure, the learning method is used to train the lane line recognition model, which solves the problem of scenarios that cannot be covered by rule-based post-processing and improves the efficiency of the map production process.

[0068] In one embodiment, the loss error is calculated based on the following loss function:

[0069] L total = L point + αL row + βL rang + γL offset + δL state ;

[0070] Among them, α, β, γ, and δ are all coefficients, which can be set to 1, 1, 0.4, and 1, respectively; L point represents the loss function value of predicting the starting position of the lane line; L 阳w represents the loss function value of predicting the row position; L rang represents the loss function value of predicting the longitudinal lane line length range; L offset represents the loss function value of predicting the offset of the lane line position; L state represents the loss function value of predicting the binary state (indicating whether to continue or stop predicting the lane line).

[0071] In one embodiment, the loss error is calculated based on the following loss function:

[0072] L total = L point + α(L xrow + L yrow + L dis ) + β(L xrang + L yrang ) + γL offset + δL state ;

[0073] Among them, L xrow represents the loss function value of predicting the position of each point on the lane line in each row; L yrow represents the loss function value of predicting the position of each point on the lane line in each column; L xrang represents the loss function value of predicting the length range of the lane line along the row direction; L yrangThe loss function value characterizing the length range prediction of the lane line along the column direction; L dis The loss function value characterizing the length prediction of each point. The so-called point length is the length from each point on the lane line to the two endpoints of the lane line. One of the two endpoints is the starting position, and the other is the ending position.

[0074] In the embodiments of the present disclosure, the loss along the row direction of the points included in the lane line, the loss along the row direction of the points, and the loss of the length of each point are added to the loss function, so that the lane line model can predict the length from each point on the lane line to the two endpoints of the lane line, generate a vectorized lane line starting from the lane line endpoints, thereby generating a vectorized lane line in the road to be processed, avoiding the subsequent manual connection of the lane line due to the discontinuous phenomenon of the lane line, thus reducing the complexity in the lane line generation process and improving the generation efficiency of the lane line.

[0075] In the embodiments of the present disclosure, the lane line recognition module is trained based on the reflection value base map sample and the trajectory of the corresponding point cloud acquisition device. The trained lane line recognition model can perform lane line recognition on the reflection value base map and directly output the detection result, realizing end-to-end lane line recognition, without the need to manually connect the lane line, which can improve the lane line recognition efficiency and avoid the errors caused by manually connecting the lane line; and since visual images are not used, the problems of distortion of the camera itself and inaccurate lane lines extracted due to the lack of depth information of the camera can be avoided.

[0076] See Figure 4 , the figure shows a comparison diagram of the lane lines extracted by the lane line recognition model obtained by using the training method of the lane line recognition model provided in the embodiments of the present disclosure and the lane lines extracted by the existing technology method. From left to right in the figure are the lane lines extracted based on point cloud data, the lane lines extracted based on image data, and the lane lines extracted by the lane line recognition model obtained by using the model training method provided in the embodiments of the present disclosure. It can be seen that the lane lines extracted based on point cloud data (sparse line segments in the figure) are relatively sparse, and the lane lines extracted based on image data (broken lines in the figure) are also intermittent, while the lane lines extracted by the lane line recognition model obtained based on the embodiments of the present disclosure (continuous lines in the figure) are relatively complete, and when the lane line is a white dotted line, continuous lane lines can also be well detected without ghosting problems. That is to say, the lane line recognition model obtained by using the model training method provided in the embodiments of the present disclosure solves the scenarios that cannot be covered by rule-based processing, such as bifurcated lane lines, lane white dotted line connection lines, and scenarios where lane lines are worn and blurred. The embodiments of the present disclosure can accurately predict the two bifurcated lane lines, lane white dotted line connection lines, and scenarios where lane lines are worn and blurred, with an accuracy rate of 96% and a recall rate of 95% under the 20 cm evaluation accuracy.

[0077] Figure 5 The flowchart of a lane line recognition method provided for an exemplary embodiment of the present disclosure. The lane line recognition method includes the following steps:

[0078] Step 501: Obtain the reflection value base map to be recognized and the trajectory of the acquisition device that acquires the reflection value base map.

[0079] The reflection value base map to be recognized is obtained based on the point cloud data of the road. The specific implementation process is similar to that of the reflection value base map sample and will not be elaborated here.

[0080] In one embodiment, the reflection value base map to be recognized is obtained based on the point cloud data collected by the acquisition device during one drive along the road, that is, based on single-loop point cloud data.

[0081] In one embodiment, the reflection value base map is obtained based on multiple sets of point cloud data. The acquisition device drives back and forth or unidirectionally along the road multiple times, and a set of point cloud data of the road is collected during each drive. The obtained multiple sets of point cloud data are rasterized on a two-dimensional plane to obtain the reflection value base map.

[0082] Obtaining the reflection value base map sample based on multiple sets of point cloud data can avoid or reduce the lane line pressing deviation problem caused by the double-image problem of the spliced point cloud and the problems of lane line wear and occlusion, reduce or even eliminate the noise in the reflection value base map sample, make the features (lane line geometric information) extracted by the lane line recognition model more accurate, and thus improve the efficiency and accuracy of model training.

[0083] Step 502: Intercept reflection value image blocks from the reflection value base map along the trajectory of the acquisition device, and input the reflection value image blocks into the trained lane line recognition model, so that the lane line recognition model can recognize the lane line recognition result in the reflection value base map according to the reflection value image blocks.

[0084] Among them, the lane line recognition model is trained based on multiple sets of training for the initial lane line recognition model. Each set of training includes the reflection value base map of the road and the trajectory of the acquisition device that acquires the reflection value base map, and the reflection value base map is marked with lane line annotation information. The model training process of the lane line recognition model refers to any of the above embodiments and will not be elaborated here.

[0085] The lane line recognition result output by the lane line recognition model can be used to create a high-precision map.

[0086] In the embodiments of the present disclosure, end-to-end lane line recognition is realized based on the lane line recognition model, which can improve the efficiency and accuracy of lane line recognition, and further improve the efficiency of high-precision map creation.

[0087] In one embodiment, the lane line recognition model includes a feature extraction layer and an output layer; predicting the lane line recognition result of the reflection value base map according to the lane line recognition model includes: triggering the feature extraction layer to extract image features and the correlation features between the respective reflection value image blocks from the reflection value image blocks, and obtaining at least two feature layers according to the image features and the correlation features; triggering the output layer to predict the lane line recognition result according to the at least two feature layers.

[0088] In the embodiments of the present disclosure, extracting the image features and the correlation features from the reflection value image blocks jointly characterizes the geometric information of the lane lines, so that the output layer predicts the lane line recognition result according to the image features and the correlation features, which is beneficial to improving the accuracy of the lane line recognition model for lane line recognition of the reflection value image.

[0089] In one embodiment, the output layer can detect lane information in two directions along the feature layer. Specifically, the output layer calculates the first lane line information along the row direction of the feature layer, calculates the second lane line information along the column direction of the feature layer, and predicts the lane line recognition result according to the first lane line information and the second lane line information.

[0090] In the embodiments of the present disclosure, on the basis of the existing lane line recognition model, an additional network structure layer is added, that is, the lane information is detected in two directions along the feature layer through two network structures, and the lane line recognition result is predicted based on the lane information detected in the two directions, so that the prediction result of the lane line recognition result is more accurate.

[0091] In one embodiment, triggering the output layer to select the feature layer of the target layer from the at least two feature layers, and predicting the lane line recognition result according to the feature layer of the target layer.

[0092] In the embodiments of the present disclosure, based on the characteristics of the reflection value base map, the feature layer of the target layer is selected to predict the lane line recognition result, so that the prediction result of the lane line recognition result is more accurate.

[0093] Corresponding to the foregoing embodiments of the training method and the lane line recognition method of the lane line recognition model, the present disclosure also provides embodiments of a training device for the lane line recognition model and a lane line recognition device.

[0094] Figure 6 The following is a schematic module diagram of a training device for a lane line recognition model provided by an exemplary embodiment of the present disclosure. The training device for the lane line recognition model includes:

[0095] An acquisition module 61, configured to acquire multiple groups of training samples; wherein each group of training samples includes a reflection value base map sample of a road and the trajectory of an acquisition device that acquires the reflection value base map sample, and the reflection value base map sample is labeled with lane line annotation information;

[0096] A preprocessing module 62, configured to, for each group of training samples, intercept a reflected value image block from the reflected value base map sample along the trajectory of the acquisition device, and input the reflected value image block into a lane line recognition model, so that the lane line recognition model recognizes the lane line recognition result in the reflected value base map sample according to the reflected value image block;

[0097] A training model 63, configured to determine a loss error according to the lane line recognition result and the lane line annotation information, and adjust the model parameters of the lane line recognition model according to the loss error.

[0098] Optionally, the lane line recognition model includes a feature extraction layer and an output layer;

[0099] The feature extraction layer is configured to extract image features and correlation features between each reflected value image block from the reflected value image block, and obtain at least two feature layers according to the image features and the correlation features;

[0100] The output layer is configured to predict the lane line recognition result according to the at least two feature layers.

[0101] Optionally, the output layer is specifically configured to:

[0102] Calculate first lane line information along the row direction of the feature layer, and calculate second lane line information along the column direction of the feature layer;

[0103] Predict the lane line recognition result according to the first lane line information and the second lane line information.

[0104] Optionally, the output layer is specifically configured to: trigger the output layer to select a feature layer of a target layer from the at least two feature layers, and predict the lane line recognition result according to the feature layer of the target layer.

[0105] Optionally, the loss error is a weighted result of the position loss function value of the points included in the lane line, the loss function value of the points along the row direction, the loss function value of the points along the column direction, and the loss function value of the point length.

[0106] Optionally, each reflected value base map sample is obtained according to multiple groups of point cloud data of the road; the trajectories of the acquisition devices for acquiring each group of point cloud data are the same.

[0107] Figure 7 The figure is a schematic diagram of modules of a lane line recognition device provided by an exemplary embodiment of the present disclosure. The lane line recognition device includes:

[0108] An acquisition module 71, configured to acquire a reflected value base map to be recognized and the trajectory of the acquisition device that acquires the reflected value base map;

[0109] An identification module 72, configured to intercept a reflected value image block from the reflected value base map along the trajectory of the acquisition device, and input the reflected value image block into a trained lane line identification model, so that the lane line identification model identifies the lane line identification result in the reflected value base map according to the reflected value image block; wherein, the lane line identification model is obtained by training an initial lane line identification model based on multiple groups of training, and each group of training includes the reflected value base map of the road and the trajectory of the acquisition device that acquires the reflected value base map, and the lane line annotation information is marked on the reflected value base map.

[0110] Optionally, the lane line identification model includes a feature extraction layer and an output layer;

[0111] The feature extraction layer is configured to extract image features and association features between each reflected value image block from the reflected value image block, and obtain at least two feature layers according to the image features and the association features;

[0112] The output layer is configured to predict the lane line identification result according to the at least two feature layers.

[0113] Optionally, the output layer is specifically configured to:

[0114] Calculate first lane line information along the row direction of the feature layer, and calculate second lane line information along the column direction of the feature layer;

[0115] Predict the lane line identification result according to the first lane line information and the second lane line information.

[0116] Optionally, the output layer is specifically configured to:

[0117] Trigger the output layer to select a feature layer of the target layer from the at least two feature layers;

[0118] Predict the lane line identification result according to the feature layer of the target layer.

[0119] Optionally, the reflected value base map is obtained according to multiple groups of point cloud data of the road; the trajectories of the acquisition devices for acquiring each group of point cloud data are the same.

[0120] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the descriptions of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of the present disclosure. A person of ordinary skill in the art can understand and implement it without creative work.

[0121] In the technical solution of the present disclosure, the processing of the point cloud data involved, such as collection, storage, use, processing, transmission, provision, and disclosure, all comply with the provisions of relevant laws and regulations and do not violate public order and good customs.

[0122] According to the embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0123] Figure 8 FIG. shows a schematic block diagram of an exemplary electronic device 800 that can be used to implement the embodiments of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, a personal digital processor, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are only examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0124] As Figure 8 shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 802 or the computer program loaded from the storage unit 808 into the random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other through a bus 804. The input / output (I / O) interface805 is also connected to the bus 804.

[0125] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as a keyboard, mouse, etc.; output unit 807, such as various types of displays, speakers, etc.; storage unit 808, such as a disk, optical disc, etc.; and communication unit 809, such as a network card, modem, wireless communication transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0126] Computing unit 801 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of computing unit 801 include but are not limited to a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Computing unit 801 executes the various methods and processes described above, such as the training method of the lane line recognition model and the lane line recognition method. For example, in some embodiments, the training method of the lane line recognition model and the lane line recognition method can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed onto device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by computing unit 801, one or more steps of the training method of the lane line recognition model and the lane line recognition method described above can be executed. Alternatively, in other embodiments, computing unit 801 can be configured to execute the training method of the lane line recognition model and the lane line recognition method by any other suitable means (e.g., by means of firmware).

[0127] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGA), application-specific integrated circuits (ASIC), application-specific standard products (ASSP), systems-on-chip (SOC), complex programmable logic devices (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: implemented in one or more computer programs, the one or more computer programs can be executed and / or interpreted on a programmable system including at least one programmable processor, the programmable processor can be a special or general-purpose programmable processor, can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0128] The program code for implementing the methods of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the program codes are executed by the processor or controller, the functions / operations specified in the flowchart and / or block diagram are implemented. The program codes can be executed entirely on the machine, partially on the machine, executed partially on the machine as an independent software package and partially on a remote machine, or executed entirely on a remote machine or server.

[0129] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0130] In order to provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which the user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0131] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected to each other by digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), and the Internet.

[0132] A computer system can include a client and a server. The client and the server are generally far from each other and typically interact through a communication network. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, a server of a distributed system, or a server incorporating a blockchain.

[0133] It should be understood that the various forms of the processes shown above can be used, with steps reordered, added, or deleted. For example, the steps recited in this disclosure can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and no limitations are imposed herein.

[0134] The above specific embodiments do not constitute a limitation on the protection scope of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the protection scope of this disclosure.

Claims

1. A training method for a lane line recognition model, comprising: Obtaining multiple groups of training samples; wherein each group of training samples includes a reflection value base map sample of a road and the trajectory of the acquisition device that acquires the reflection value base map sample, and the reflection value base map sample is labeled with lane line annotation information; For each group of training samples, intercepting reflection value image blocks from the reflection value base map sample along the trajectory of the acquisition device, and inputting the reflection value image blocks into the lane line recognition model, so that the lane line recognition model recognizes the lane line recognition result in the reflection value base map sample according to the reflection value image blocks; Determining a loss error according to the lane line recognition result and the lane line annotation information, and adjusting the model parameters of the lane line recognition model according to the loss error; Wherein, the lane line recognition model includes a feature extraction layer and an output layer; The predicting the lane line recognition result of the reflection value base map sample according to the lane line recognition model includes: Triggering the feature extraction layer to extract image features and correlation features between the respective reflection value image blocks from the reflection value image blocks, and obtaining at least two feature layers according to the image features and the correlation features; Triggering the output layer to predict the lane line recognition result according to the at least two feature layers.

2. The training method of the lane line recognition model according to claim 1, wherein, The triggering the output layer to predict the lane line recognition result according to the at least two feature layers includes: Triggering the output layer to calculate first lane line information along the row direction of the feature layer and calculate second lane line information along the column direction of the feature layer; Predicting the lane line recognition result according to the first lane line information and the second lane line information.

3. The training method of the lane line recognition model according to claim 1, wherein, The triggering the output layer to predict the lane line recognition result according to the at least two feature layers includes: Triggering the output layer to select a feature layer of a target layer from the at least two feature layers; Predicting the lane line recognition result according to the feature layer of the target layer.

4. The training method of the lane line recognition model according to claim 1, wherein, The loss error includes a weighted result of the position loss function value of the points included in the lane line, the loss function value of the points along the row direction, the loss function value of the points along the column direction, and the loss function value of the point length.

5. The training method of the lane line recognition model according to any one of claims 1-4, wherein, Each reflection value base map sample is obtained according to multiple groups of point cloud data of the road; the trajectories of the acquisition devices for acquiring each group of point cloud data are the same.

6. A lane line recognition method, comprising: Obtaining a reflection value base map to be recognized and the trajectory of the acquisition device that acquires the reflection value base map; Intercepting reflection value image blocks from the reflection value base map along the trajectory of the acquisition device, and inputting the reflection value image blocks into a trained lane line recognition model, so that the lane line recognition model recognizes the lane line recognition result in the reflection value base map according to the reflection value image blocks; wherein, the lane line recognition model is obtained by training an initial lane line recognition model based on multiple groups of training, and each group of training includes a reflection value base map of a road and the trajectory of the acquisition device that acquires the reflection value base map, and the reflection value base map is labeled with lane line annotation information; Wherein, the lane line recognition model includes a feature extraction layer and an output layer; Predicting the lane line recognition result of the reflection value base map according to the lane line recognition model includes: Triggering the feature extraction layer to extract image features and the correlation features between each reflection value image block from the reflection value image block, and obtaining at least two feature layers according to the image features and the correlation features; Triggering the output layer to predict the lane line recognition result according to the at least two feature layers.

7. The lane line recognition method according to claim 6, wherein, The triggering the output layer to predict the lane line recognition result according to the at least two feature layers includes: Triggering the output layer to calculate the first lane line information along the row direction of the feature layer and calculate the second lane line information along the column direction of the feature layer; Predicting the lane line recognition result according to the first lane line information and the second lane line information.

8. The lane line recognition method according to claim 6, wherein, The triggering the output layer to predict the lane line recognition result according to the at least two feature layers includes: Triggering the output layer to select the feature layer of the target layer from the at least two feature layers; Predicting the lane line recognition result according to the feature layer of the target layer.

9. The lane line recognition method according to any one of claims 6-8, wherein, The reflection value base map is obtained according to multiple groups of point cloud data of the road; the trajectories of the acquisition devices for acquiring each group of point cloud data are the same.

10. A training device for a lane line recognition model, including: An acquisition module, configured to acquire multiple groups of training samples; wherein, each group of training samples includes a reflection value base map sample of a road and the trajectory of an acquisition device for acquiring the reflection value base map sample, and the reflection value base map sample is marked with lane line annotation information; A preprocessing module, configured to, for each group of training samples, intercept reflection value image blocks from the reflection value base map sample along the trajectory of the acquisition device, and input the reflection value image blocks into the lane line recognition model, so that the lane line recognition model recognizes the lane line recognition result in the reflection value base map sample according to the reflection value image blocks; A training model, configured to determine a loss error according to the lane line recognition result and the lane line annotation information, and adjust the model parameters of the lane line recognition model according to the loss error; Wherein, the lane line recognition model includes a feature extraction layer and an output layer; The feature extraction layer is configured to extract image features and the correlation features between each reflection value image block from the reflection value image block, and obtain at least two feature layers according to the image features and the correlation features; The output layer is configured to predict the lane line recognition result according to the at least two feature layers.

11. The training device for the lane line recognition model according to claim 10, wherein, The output layer is specifically configured to: Calculate the first lane line information along the row direction of the feature layer and calculate the second lane line information along the column direction of the feature layer; Predict the lane line recognition result according to the first lane line information and the second lane line information.

12. The training device for the lane line recognition model according to claim 10, wherein, The output layer is specifically configured to: trigger the output layer to select the feature layer of the target layer from the at least two feature layers, and predict the lane line recognition result according to the feature layer of the target layer.

13. The training device for the lane line recognition model according to claim 10, wherein, The loss error is the weighted result of the position loss function value of the points included in the lane line, the loss function value of the points along the row direction, the loss function value of the points along the column direction, and the loss function value of the point length.

14. The training device for a lane line recognition model according to any one of claims 10-13, wherein, Each reflected value base map sample is obtained according to multiple groups of point cloud data of the road; the trajectories of the acquisition devices for acquiring each group of point cloud data are the same.

15. A lane line recognition device, comprising: An acquisition module, configured to acquire a reflected value base map to be recognized and the trajectory of the acquisition device that acquires the reflected value base map; A recognition module, configured to intercept a reflected value image block from the reflected value base map along the trajectory of the acquisition device, and input the reflected value image block into a trained lane line recognition model, so that the lane line recognition model recognizes the lane line recognition result in the reflected value base map according to the reflected value image block; wherein, the lane line recognition model is obtained by training an initial lane line recognition model based on multiple groups of training, and each group of training includes a reflected value base map of a road and the trajectory of the acquisition device that acquires the reflected value base map, and the reflected value base map is labeled with lane line annotation information; Wherein, the lane line recognition model includes a feature extraction layer and an output layer; The feature extraction layer is configured to extract image features and correlation features between each reflected value image block from the reflected value image block, and obtain at least two feature layers according to the image features and the correlation features; The output layer is configured to predict the lane line recognition result according to the at least two feature layers.

16. The lane line recognition device according to claim 15, wherein, The output layer is specifically configured to: Calculate first lane line information along the row direction of the feature layer, and calculate second lane line information along the column direction of the feature layer; Predict the lane line recognition result according to the first lane line information and the second lane line information.

17. The lane line recognition device according to claim 15, wherein, The output layer is specifically configured to: Trigger the output layer to select a feature layer of a target layer from the at least two feature layers; Predict the lane line recognition result according to the feature layer of the target layer.

18. The lane line recognition device according to any one of claims 15-17, wherein, The reflected value base map is obtained according to multiple groups of point cloud data of the road; the trajectories of the acquisition devices for acquiring each group of point cloud data are the same.

19. An electronic device, comprising: At least one processor; And A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor, so that the at least one processor can execute the method according to any one of claims 1-9.

20. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to execute the method according to any one of claims 1-9.

21. A computer program product, comprising a computer program, where the computer program, when executed by a processor, implements the method according to any one of claims 1-9.

Images