Model training methods, media, software products, and electronic devices

Abstract

This application relates to the field of electronic devices, specifically to a model training method, medium, program product, and electronic device. The model training method includes: acquiring sample images of a target road; inputting the sample images into a model to be trained to obtain a predicted centerline of the target road; selecting a target labeled centerline corresponding to the predicted centerline from at least one labeled centerline of the target road; obtaining two target labeled lane lines corresponding to the target labeled centerline; and adjusting the model parameters of the model to be trained when the predicted centerline and the two target labeled lane lines do not satisfy a positional matching relationship. According to the model training method provided in this application, the labeled lane lines corresponding to the labeled centerline can be used to train the model to be trained, enabling the model to maintain good training results even when the quality of the centerline annotations in the sample images is poor.

Description

Technical Field

[0001] This application relates to the field of electronic equipment technology, and in particular to a model training method, medium, program product and electronic equipment. Background Technology

[0002] In the current training process of centerline prediction models, supervised learning methods such as feature regression or mask segmentation are usually used to train the model.

[0003] Since centerlines may not be actual road markings, it's necessary to annotate lane centerlines in road sample images before training the model. Currently, lane centerlines are typically annotated manually. However, manually annotated lane centerlines can have significant errors compared to ideal centerlines. For example, errors by the annotator can result in centerlines that are not strictly centered or are distorted. These issues reduce the data quality of the sample images, thereby lowering the training accuracy of the centerline prediction model and leading to poor accuracy and stability in predicting complex scenarios such as winding lanes. Summary of the Invention

[0004] This application provides a model training method, medium, program product, and electronic device. The following describes this application from multiple aspects, and the embodiments and beneficial effects of the following aspects can be referenced each other.

[0005] In a first aspect, this application provides a model detection method, the method comprising: acquiring a sample image of a target road; inputting the sample image into a model to be trained to obtain a predicted centerline of the target road; selecting a target labeled centerline corresponding to the predicted centerline from at least one labeled centerline of the target road; acquiring two target labeled lane lines that satisfy a positional matching relationship with the target labeled centerline; and adjusting the model parameters of the model to be trained when the predicted centerline does not satisfy a positional matching relationship with the two target labeled lane lines.

[0006] According to an embodiment of this application, a model training method is provided. In this method, the model to be trained is trained by introducing an additional matching relationship between two target lane lines and the predicted center line. This avoids the problem of poor model training effect caused by low quality of target lane line annotations.

[0007] In some embodiments, the target road has multiple labeled center lines, and the target labeled center line corresponding to the predicted center line is selected from at least one labeled center line of the target road, including: determining the distance between each labeled center line and the predicted center line respectively; and taking the labeled center line with the shortest distance to the predicted center line as the target labeled center line.

[0008] According to embodiments of this application, a method is provided for determining a target annotation centerline from multiple annotation centerlines based on the distance between the predicted centerline and the annotation centerline. This improves the consistency between the selected annotation centerline and the predicted centerline, thereby enhancing the reliability of the training samples.

[0009] In some embodiments, the distance between the labeled centerline and the predicted centerline is determined by: determining P first sampling points of the labeled centerline and P second sampling points corresponding to the P first sampling points on the predicted centerline, where P is an integer greater than 2; calculating the distance difference between the P first sampling points and the corresponding second sampling points respectively, and taking the average of the distance differences as the distance between the labeled centerline and the predicted centerline.

[0010] According to an embodiment of this application, a method for calculating the distance between the predicted center line and the labeled center line is provided. The distance between the predicted center line and the labeled center line is represented by the average distance of multiple sampling points, which avoids noise from extreme sampling points and better reflects the overall distance between the predicted center line and the labeled center line.

[0011] In some embodiments, the predicted centerline has a first distance, a first curvature similarity, and a first direction similarity with the first target labeled lane line among the two target labeled lane lines, and the predicted centerline has a second distance, a second curvature similarity, and a second direction similarity with the second target labeled lane line among the two target labeled lane lines, wherein the centerline is either the target labeled centerline or the predicted centerline; and the position matching relationship includes: the distance difference between the first distance and the second distance is less than a distance threshold; the curvature similarity difference between the first curvature similarity and the second curvature similarity is less than a curvature threshold; and the direction similarity difference between the first direction similarity and the second direction similarity is less than a direction threshold.

[0012] According to an embodiment of this application, a method for calculating matching relationships is provided. This method uses three indicators—distance, curvature similarity, and direction similarity—used to measure the similarity between the predicted centerline and the target labeled lane line, together to determine the matching relationship, thereby reducing matching errors caused by misjudgment of a single indicator.

[0013] In some embodiments, the first distance is determined by determining M third sampling points on the prediction centerline and M fourth sampling points corresponding to the M third sampling points on the first target lane line, where M is an integer greater than 2;

[0014] Calculate the distance difference between each of the M third sampling points and the corresponding fourth sampling point, and take the average of the distance differences as the first distance.

[0015] In some embodiments, the first curvature similarity is determined by calculating the curvature difference between M third sampling points and the corresponding fourth sampling point, and taking the average of the curvature differences as the first curvature similarity.

[0016] In some embodiments, the first directional similarity is determined in the following manner:

[0017] Calculate the angle between the tangent vector directions of the M third sampling points and the corresponding fourth sampling points, and take the average of the angles between the tangent vector directions as the first direction similarity.

[0018] In some embodiments, obtaining two target lane lines that satisfy the positional matching relationship with the target lane line includes: selecting candidate lane lines that satisfy the positional matching relationship with the target lane line from multiple lane lines; if there are two candidate lane lines selected, the candidate lane lines are used as target lane lines; if there are more than two candidate lane lines selected, the two candidate lane lines on both sides of the target lane line that are closest to the target lane line are used as target lane lines.

[0019] Secondly, embodiments of this application provide a readable storage medium including one or more programs, which, when executed on an electronic device, cause the electronic device to implement the method of any embodiment of the first aspect.

[0020] Secondly, this application provides a program product that, when executed on an electronic device, causes the electronic device to implement the method of any embodiment of the first aspect.

[0021] Thirdly, this application provides an electronic device, comprising: at least one memory for storing one or more programs; and at least one processor for executing one or more programs to cause the electronic device to implement the method of any embodiment in the first aspect. Attached Figure Description

[0022] Figure 1 A schematic diagram of the predicted centerline provided in the embodiments of this application;

[0023] Figure 2An exemplary flowchart of the model training method provided in the embodiments of this application;

[0024] Figure 3 An exemplary flowchart of the model training method provided in the embodiments of this application;

[0025] Figure 4 An exemplary flowchart of a method for determining a first target lane line provided in an embodiment of this application;

[0026] Figure 5 A block diagram of an electronic device provided in an embodiment of this application;

[0027] Figure 6 This is a block diagram of a system-on-a-chip provided in an embodiment of this application. Detailed Implementation

[0028] The embodiments of this application include, but are not limited to, a model training method, a medium, a program product, and an electronic device. The method provided by the embodiments of this application can train the model to be trained, so as to ensure the training quality of the model even when the sample image quality is poor.

[0029] For ease of understanding, this application first explains the terms that may be involved in the embodiments.

[0030] 1. Lane markings: In some embodiments, lane markings typically refer to physical traffic markings applied to the road surface using paint, lights, or other means. Lane markings can divide a road into multiple lanes and indicate the boundary of each lane.

[0031] 2. Centerline: In some embodiments, the centerline is used to represent the geometric center of the lane. The centerline is typically not a physical traffic marking on the lane. In some embodiments, lane markings may be virtual markings estimated by electronic devices based on road lane lines, traffic flow, etc.

[0032] In some embodiments, the centerline can be used to indicate the future trajectory of the vehicle. Since predicting the vehicle's trajectory is a crucial component of driver assistance models, centerline prediction methods are currently widely used in these models.

[0033] In some current technical solutions, an image of the target road can be input into a centerline prediction model, and the centerline of each lane in the target road can be predicted based on the centerline prediction model. The centerline prediction model can be obtained through supervised learning training methods. Supervised learning is a training method that optimizes the model's parameters using the residuals between the model's predicted values ​​and the ideal values.

[0034] In some current technical solutions, labeled values ​​can often be used as substitutes for ideal values. That is, the centerline labeled in the sample image is used as the ideal value, and the centerline prediction model is trained based on the residual between the labeled centerline and the predicted centerline.

[0035] It is understood that in the embodiments of this application, the center line of the annotation, which is the ideal value, should be a smooth curve or straight line that is strictly centered on the road. However, in some embodiments, the center line of the annotation value, which is used as a substitute for the ideal value, is usually marked manually by visual inspection. Due to problems such as visual errors of the human eye, errors inevitably occur between the marked value and the ideal value.

[0036] However, in supervised learning, since the quality of the sample data directly determines the training effect of the model, the above-mentioned error problem will cause the centerline prediction model to produce a large deviation in the prediction results.

[0037] Figure 1 This is a schematic diagram of the predicted centerline provided in an embodiment of this application. Figure 1 As shown, Figure 1 The illustration shows a road 100 in an urban environment, consisting of only one lane in each direction. Road 100 includes two lane lines: a first lane line 110 and a second lane line 120. The ideal centerline 130 is the ideal location of the centerline on road 100. The predicted centerline 140 is the centerline predicted by the centerline prediction model based on sample images of road 100. The labeled centerline 150 can be a labeled lane line on road 100 obtained through manual annotation.

[0038] Understandable. Figure 1 An error occurred between the labeled centerline 150 and the ideal centerline 130; the labeled centerline 150 is shifted to the left compared to the ideal centerline 130. Since the parameters of the centerline prediction model are adjusted using the left-shifted labeled centerline 150, the predicted centerline 140 will also exhibit the same left-shifting problem.

[0039] It is understandable that if the predicted center line 140, which deviates to the left as described above, indicates the vehicle's future trajectory, it may cause the vehicle to gradually deviate from the center of the lane, thereby increasing the probability of certain types of traffic accidents. For example, it may increase the likelihood of the vehicle hitting a road guardrail.

[0040] As mentioned above, when the centerline prediction model is trained using only the labeled centerlines in the sample images, if the quality of the sample image data used for training is poor, the prediction quality of the trained model will be reduced, resulting in a large error between the predicted centerline output by the model and the ideal centerline.

[0041] Therefore, this application proposes a method such as Figure 2 The model training method shown. Figure 2 As shown, the method includes:

[0042] S100: Obtain a sample image of the target road.

[0043] S110: Input the sample image into the model to be trained to obtain the predicted center line of the target road.

[0044] S120: Select the target labeled centerline corresponding to the predicted centerline from at least one labeled centerline of the target road.

[0045] S130: Obtain the two target lane lines corresponding to the target center line.

[0046] S140: If the predicted centerline and the two target lane marking lines do not satisfy the position matching relationship, adjust the model parameters of the model to be trained.

[0047] As can be understood, the method described above first determines the predicted centerline based on sample images of the target road. Then, a labeled centerline matching the predicted centerline is determined, and based on the correspondence between the labeled centerline and the labeled lane lines, two target labeled lane lines corresponding to the predicted centerline are identified. Finally, the positional matching relationship between the predicted centerline and the two target labeled lane lines is determined, and based on this positional matching relationship, the parameters of the model to be trained are fine-tuned to complete the model training process.

[0048] For example, a sample image of road 100 can be input into the centerline prediction model, and the predicted value of the centerline of the target image can be obtained through the centerline prediction model, which is the predicted centerline 140.

[0049] Since the sample image of road 100 only includes one labeled centerline 150 and two labeled lane lines (i.e., the first lane line 110 and the second lane line 120), the labeled centerline 150 can be determined as the target centerline corresponding to the predicted centerline 140, and the first lane line 110 and the second lane line 120 can be determined as the two target labeled lane lines corresponding to the labeled centerline 150.

[0050] Since the predicted centerline 140 deviates from the ideal centerline 130 (which is the center position of road 100), the predicted centerline 140 and the two target lane marking lines do not meet the matching relationship. Therefore, the centerline prediction model can be updated by adjusting the parameters of the model to be trained, so that the predicted centerline predicted by the centerline prediction model in the future matches the two target lane marking lines better, thus completing the training process of the centerline prediction model.

[0051] It is understandable that, according to the above method, when the quality of the labeled images used in the training process is poor, the center line with higher reliability can be used for auxiliary training, thereby avoiding model parameter errors caused by poor sample image quality and improving the prediction accuracy of the center line prediction model for lane center lines when the sample images are poor.

[0052] Figure 3 An exemplary flowchart of a model training method provided in this application embodiment is used to illustrate... Figure 2 The schematic diagram of the model training method shown is further refined below. A specific model training method is described below based on some embodiments of this application. (Reference) Figure 3 The model training process includes:

[0053] S200: Obtain a sample image of the target road.

[0054] In some embodiments, the target road may include various types of roads in real-world scenarios, such as highways, viaducts, mountain roads, or internal roads, which are not limited herein.

[0055] It is understood that the target road can be divided into multiple lanes for traffic by lane lines, such as single lane, two lane, four lane or eight lane, etc. The specific number of lanes included in the target road is not limited in this application.

[0056] In some embodiments, in order to train the lane line prediction model, sample images of the target road must first be acquired.

[0057] It is understood that the above sample images are used to train the model and can be obtained in various ways, such as by image acquisition devices such as drones, vehicle cameras or high-speed monitoring, or by obtaining them from image databases. This application does not limit the methods used.

[0058] In some embodiments, for training the centerline prediction model, the sample images also include labeled lane lines and labeled centerlines for each lane of the target road. These labeled lane lines and labeled centerlines can be obtained manually or automatically through a machine learning model; this application does not impose any limitation on this.

[0059] For example, in the case of manual labeling, the labeler can determine the center position of each lane in the target road by visual estimation and mark the center line at the center position of each lane.

[0060] S210: Input the sample image into the model to be trained to obtain the predicted center line of the target road.

[0061] In some embodiments, the model to be trained is a supervised learning-based model, including a deep learning model or a machine learning model. For example, a deep learning-based Convolutional Neural Network (CNN) model.

[0062] It is understandable that since supervised learning models are trained using the residuals between predicted and ideal values ​​(labeled values), it is necessary to obtain the predicted center line of the model to be trained based on the sample images before training the model.

[0063] For example, Figure 1 The sample image of road 100 is input into the convolutional neural network model to predict the prediction centerline 140 corresponding to road 100.

[0064] S220: Determine whether the number of labeled centerlines for each lane corresponding to the predicted centerline is greater than 1.

[0065] It is understandable that if the judgment result of S220 is yes, it means that the number of labeled center lines for each lane corresponding to the center line is greater than 1, and S230 is executed to determine the target labeled center line from multiple labeled center lines; if the judgment result is no, it means that the number of labeled center lines for each lane corresponding to the center line is 1, and S230A is executed to determine the only labeled center line as the target labeled center line corresponding to the predicted center line.

[0066] It is understood that in some embodiments, each lane of the target road includes at least two lane lines on the left and right and a center line for indicating the center position of the lane. That is, the minimum number of lane lines is 2 and the minimum number of center lines is 1.

[0067] For example, Figure 1 The target road shown consists of only one road 100, and road 100 includes only one center line 150 and two target lane lines (first lane line 110 and second lane line 120).

[0068] It is understood that in some other embodiments, in addition to the two lane lines and one center line mentioned above, multiple additional lane lines and center lines can be marked for each lane. In this case, the number of lane lines marked for each lane can be greater than 2, and the number of center lines marked can be greater than 1.

[0069] Therefore, for each lane corresponding to a predicted centerline, the number of marked centerlines can be one or more. Thus, when determining the target marked centerline, it is also necessary to determine the number of marked centerlines for each lane corresponding to the predicted centerline, so that the target marked centerline can be determined in different ways depending on the number of marked centerlines.

[0070] S230: Select the target labeled centerline corresponding to the predicted centerline from at least one labeled centerline of the target road.

[0071] Understandably, based on the judgment result of S220, in S230, there are multiple labeled center lines corresponding to the predicted center line of the lane. In this case, it is necessary to determine one labeled center line from among the multiple labeled center lines as the target labeled center line corresponding to the predicted lane center line for use in the subsequent training process.

[0072] In some embodiments, the target label centerline corresponding to the predicted lane centerline can be determined by the distance between the predicted lane centerline and the label centerline. Specifically, the method for determining the target label centerline includes: determining the distance between each label centerline and the predicted centerline; and selecting the label centerline with the shortest distance to the predicted centerline as the target label centerline.

[0073] It is understandable that the closer the distance between the labeled center line and the predicted center line, the closer the labeled center line is to the location of the predicted center line. In this way, the labeled center line closest to the predicted center line can be determined as the target labeled center line, reducing the initial residual between the predicted center line and the target labeled center line, so as to accelerate the convergence speed of the model to be trained in the subsequent training process.

[0074] It is understood that the method for determining the distance between the marked centerline and the predicted centerline will be described in detail below, and will not be repeated here.

[0075] S230A: Determine the unique labeled centerline of the lane corresponding to each predicted centerline as the target labeled centerline.

[0076] It is understandable that, based on the judgment result of S220, there is only one annotation center line used to determine the target annotation center line in S230A. Therefore, the only annotation center line can be directly determined as the target annotation center line corresponding to the predicted center line.

[0077] S240: Determine whether the number of lane lines corresponding to the center line of the target lane is greater than 2.

[0078] It is understandable that if the judgment result of S240 is yes, it means that there are more than 2 lane lines corresponding to the lane where the target center line is located. S250 is executed to determine the two target lane lines corresponding to the target center line from the multiple lane lines. If the judgment result is no, it means that there are 2 lane lines corresponding to the lane where the target center line is located. S250A is executed to determine these two lane lines as the target lane lines corresponding to the target center line.

[0079] It is understandable that in S200, sample images obtained through manual annotation may have errors between the annotated center line and the ideal center line.

[0080] In some embodiments, to avoid poor training performance of the model to be trained due to the aforementioned error problems, this application embodiment also introduces two target label lane lines corresponding to the target label center line to assist the center line prediction model in training.

[0081] It is understandable that, according to S220, the number of lane lines corresponding to the center line of the target can be 2 or any positive integer greater than 2.

[0082] Therefore, before determining the two target lane lines corresponding to the target center line, it is necessary to judge the different situations of the number of lane lines, so as to select the target lane line determination method corresponding to different numbers of lane lines.

[0083] S250: Obtain two target lane lines that satisfy the positional matching relationship with the target centerline.

[0084] It is understandable that, based on the judgment result of S240, in S250, the number of lane lines corresponding to the lane marked by the target center line is greater than 2.

[0085] In this case, two lane lines can be identified from multiple lane lines to serve as the two target lane lines corresponding to the target lane center line.

[0086] It is understood that the two target label lane lines determined above include the first target label lane line located on the first side of the target label center line, and the second target label lane line located on the other side of the target label center line.

[0087] In some embodiments, in order to determine the first target lane line, multiple first lane lines located on the first side of the target lane centerline can be obtained as candidate lane lines for determining the first target lane.

[0088] In some embodiments, after determining the first lane marking line, the first target lane marking line can be determined from the first lane marking line based on the positional relationship between the first lane marking line and the target lane marking center line.

[0089] For example, the first lane line that is closest to the first target lane line, has the highest curvature similarity, and the highest direction similarity can be determined from the first lane lines and used as the first target lane line.

[0090] How to determine the first target lane line with the closest distance, highest curvature similarity, and highest direction similarity will be described in detail below, and will not be repeated here.

[0091] It is understandable that, based on the above method, the first target lane marking line located on the first side can be determined. The method for determining the second target lane marking line located on the other side can refer to the method for determining the first target lane marking line, and will not be elaborated here.

[0092] S250A: Define the two lane lines corresponding to the center line of the target lane as the target lane lines.

[0093] It is understandable that, based on the judgment result of S240, in S250A, the number of lane lines corresponding to the center line of the target is two.

[0094] In this case, there are only two lane marking lines used to determine the target lane marking lines. Therefore, these two lane marking lines can be directly identified as the two target lane marking lines corresponding to the target marking center line.

[0095] S260: Determine whether the predicted centerline and the two target lane marking lines satisfy a matching relationship.

[0096] It is understandable that if the judgment result of S260 is negative, since the predicted center line does not match the two target lane lines, S270 can be executed to adjust the model to be trained. The specific method for determining the position matching relationship will be described in detail later, and will not be repeated here. If the judgment result of S260 is positive, the predicted center line predicted by the center line prediction model can match the two target lane lines, and S280 can be executed to complete the training process of the model to be trained.

[0097] It's understandable that lane markings, being physical traffic markings on lanes, are less prone to error during annotation. However, center lines are annotated using methods like visual inspection or algorithmic calculation, making them more susceptible to error. Therefore, to avoid center line annotation errors, during training, two target lane markings can be used as ideal values, and the predicted center line as the predicted value. The residual between the ideal and predicted values ​​can then be used to adjust the training model, ensuring a matching relationship between the two target lane markings and the predicted center line, thus completing the training process.

[0098] S270: Adjust the model parameters of the model to be trained.

[0099] It is understandable that, based on the judgment result of S260, the predicted centerline does not satisfy the matching relationship with the two target lane marking lines in S270. In this case, the model parameters of the training model can be tuned based on the distance difference, orientation similarity difference, and shape similarity difference between the predicted centerline and the two target lane marking lines. For example, the model parameters of the training model can be tuned using the backpropagation algorithm to continuously reduce the distance difference, orientation similarity difference, and shape similarity difference between the predicted centerline and the two target lane marking lines.

[0100] It is understandable that when the difference in distance, direction similarity, and shape similarity between the predicted centerline and the two labeled lane lines is less than the prediction accuracy threshold, S280 can be executed to complete the model training process.

[0101] In some embodiments, the prediction accuracy threshold as described above includes the accuracy threshold for distance difference. For example, when the ratio of the distance difference to the distance between the predicted centerline and the two target lane lines is less than 0.1%, the accuracy of the model to be trained in terms of distance difference can be considered to have met the requirements.

[0102] It is understood that the accuracy thresholds mentioned above may also include accuracy thresholds for orientation similarity differences and / or accuracy thresholds for shape similarity differences, which will not be elaborated upon here.

[0103] S280: Model training complete.

[0104] In some embodiments, the distance between the labeled centerline and the predicted centerline is determined by:

[0105] S231: Determine the P first sampling points of the marked center line, and predict the P second sampling points corresponding to the P first sampling points on the center line.

[0106] It is understandable that, since the center line or lane line is a continuous curve located in the space of the sample image, directly calculating the distance or calculating the distance pixel by pixel would be computationally intensive. Therefore, when calculating the distance between the labeled center line and the predicted center line, the distance between the sampling points on the labeled center line and the predicted center line can be used as an approximation.

[0107] Therefore, P first sampling points can be determined on the labeled center line, and P second sampling points corresponding to the labeled center line can be determined on the predicted center line for subsequent distance determination. Here, P can be an integer greater than 2, such as 2, 3, 5, 10, 20, 50, or 100, etc., and this application does not impose any limitation on this.

[0108] In some embodiments, a first sampling interval for the first sampling point and a second sampling interval for the second sampling point can be determined based on the lengths of the labeled center line and the predicted center line, respectively. P first sampling points are determined on the labeled center line according to the first sampling interval, and P second sampling points are determined on the predicted center line according to the second sampling interval.

[0109] For example, if the length of both the labeled centerline and the predicted centerline is 100 meters, and 20 sampling points are obtained (i.e., P=20), 5 meters can be used as the first sampling interval and the second sampling interval. A first sampling point is determined every 5 meters on the labeled centerline, and a second sampling point is determined every 5 meters on the predicted centerline.

[0110] S232: Calculate the distance difference between each of the P first sampling points and the corresponding second sampling point, and use the average of the distance differences as the distance between the labeled center line and the predicted center line.

[0111] It is understood that, in some embodiments, in order to use the distance difference between P first sampling points and corresponding second sampling points to jointly represent the distance between the labeled center line and the predicted center line, the average value of the distance difference between P first sampling points and corresponding second sampling points can be determined, and the above average value can be used as the distance between the labeled center line and the predicted center line.

[0112] For example, when P=5, if the distance differences between the 5 first sampling points and the corresponding second sampling points are 1 meter, 3 meters, 2 meters, 4 meters and 3 meters respectively, the distance between the marked center line and the predicted center line can be obtained by calculating the average of the above distance differences (1+3+2+4+3) / 5=2.6 meters.

[0113] In some embodiments, the two target label lane lines include a first target label lane line and a second target label lane line. Figure 4 The method for determining the lane lines of the first target is shown, specifically including:

[0114] S251: Obtain multiple first-line annotation lane lines located on the first side of the target annotation centerline.

[0115] It is understood that, since the target lane centerline is located at the center of its corresponding lane, the multiple lane markings corresponding to the lane containing the target lane centerline can include a first lane marking located on the first side of the target lane centerline and multiple second lane markings located on the second side of the target lane marking. Specifically, the method for determining the first and second lane markings will be described later, and will not be repeated here.

[0116] It is understood that in the introduction of S251 to S255, the process of determining the target label center line from the label center line will be described using the first target label lane line as an example. The method of determining the second target label line can refer to the method of determining the first target label line, and will not be repeated here.

[0117] S252: From multiple first-marked lane lines, select the i first-marked lane lines with the smallest distance to the center line of the target mark to form a first lane line set.

[0118] In some embodiments, i is a positive integer greater than or equal to 2. For example, i can be an integer such as 2, 3, 5, 10, 20, or 50, which is not limited herein.

[0119] It is understandable that, since the distance between the target lane line and the center line is less than the distance between the center line and most lane lines, the first lane lines can be initially screened by the distance between the target center line and the first lane lines to obtain the first lane line combination composed of the i first lane lines closest to the target center line.

[0120] In some embodiments, the method for calculating the distance can refer to the method for calculating the first distance described later, and will not be repeated here.

[0121] S253: From multiple first-labeled lane lines, select j first-labeled lane lines with the smallest curvature similarity to the target label centerline to form a second lane line set.

[0122] In some embodiments, j is a positive integer greater than or equal to 2. For example, j can be an integer such as 2, 3, 5, 10, 20, or 50, which is not limited herein.

[0123] It's understandable that curvature similarity can be used to measure the consistency between the first lane marking and the target lane marking's centerline. For example, a higher curvature similarity indicates that the curves and undulations between the first lane marking and the target lane marking's centerline are more similar. Thus, by filtering based on curvature similarity, lane markings with inconsistent curvature can be eliminated.

[0124] In some embodiments, the method for calculating curvature similarity can refer to the method for calculating the first curvature similarity described later, and will not be repeated here.

[0125] S254: From multiple first-labeled lane lines, select the k first-labeled lane lines with the smallest directional similarity to the target label centerline to form a third lane line set.

[0126] In some embodiments, k is a positive integer greater than or equal to 2. For example, k can be an integer such as 2, 3, 5, 10, 20, or 50, which is not limited herein.

[0127] As can be understood, directional similarity measures the consistency of the pointing directions of the first lane label and the target label centerline in a local area. For example, a higher directional similarity indicates a smaller angle between the pointing directions of the tangent vectors of the first lane label and the target label centerline at each sampling point; in other words, the first lane label and the target label centerline tend to be parallel within a local range at a certain sampling point. Thus, by filtering based on directional similarity, lane labels that are inconsistent in their local pointing directions can be eliminated.

[0128] S255: Determine the first target lane line based on the first lane line set, the second lane line set, and the third lane line set.

[0129] It is understandable that, based on S251 to S254, three sets of lane lines were determined that have the smallest distance from the target annotation centerline, the highest curvature similarity, and the highest direction similarity.

[0130] In some embodiments, in order to determine a first target labeled lane line that simultaneously satisfies distance, curvature similarity and orientation similarity, a first intersection of a first set of lane lines, a second set of lane lines and a third set of lane lines can be determined.

[0131] It can be understood that the sample image includes at least two lane lines located on either side of the target annotation centerline within the lane corresponding to the target annotation centerline. Therefore, the first intersection obtained above includes at least one lane line.

[0132] Therefore, if there is only one first lane line corresponding to the first intersection, then one first lane line in the first intersection can be used as the first target lane line.

[0133] If the number of first labeled lane lines corresponding to the first intersection is equal to or greater than two, it is necessary to filter the first labeled lane lines in the first intersection to obtain the first target labeled lane line.

[0134] For example, the first lane line closest to the center line of the target label in the first intersection can be used as the first target label lane line.

[0135] In some embodiments, the matching relationship can be used to represent the consistency of similarity between the predicted centerline and the two target labeled lane lines. The similarity between the predicted centerline and the two target labeled lane lines can be represented by evaluating the distance, curvature similarity, and orientation similarity between the predicted lane centerline and the two target labeled lane lines in the aforementioned three dimensions (distance, curvature similarity, and orientation similarity).

[0136] For example, the closer the distance between the predicted centerline and the lane line labeled with the first target, and the higher the curvature similarity and direction similarity, the more similar the predicted centerline and the lane line labeled with the first target are.

[0137] For ease of distinction, this application uses the distance, curvature similarity, and direction similarity between the predicted centerline and the lane line marked on the first target as the first distance, the first curvature similarity, and the first direction similarity. The distance, curvature similarity, and direction similarity between the predicted centerline and the lane line marked on the second target as the second distance, the second curvature similarity, and the second direction similarity.

[0138] It is understandable that, since the matching relationship is used to represent the consistency of similarity between the predicted centerline and the two target labeled lane lines, in order to determine whether the predicted lane centerline and the two target labeled lane lines satisfy the position matching relationship, it is necessary to obtain the first similarity between the predicted centerline and the first target labeled lane line, and the second similarity between the predicted centerline and the second target labeled lane line, so as to determine whether the predicted centerline and the two target labeled lane lines satisfy the matching relationship based on the consistency of the first similarity and the second similarity.

[0139] In some embodiments, the first similarity may include a first distance between the predicted centerline and the first target labeled lane line, a first curvature similarity, and a first direction similarity, and the second similarity may include a second distance between the predicted centerline and the second target labeled lane line, a second curvature similarity, and a second direction similarity.

[0140] It is understood that since the first similarity and the second similarity can be calculated in the same way, the following will use the first distance, the first curvature similarity and the first direction similarity as examples to introduce the method of determining the first similarity as described above.

[0141] In some embodiments, since the first distance, the first curvature similarity, and the first orientation similarity can be determined by the distance, curvature similarity, and orientation similarity at each sampling point on the predicted centerline and the first target labeled lane line, multiple sampling points are determined on the predicted centerline and the first target labeled lane line before determining the first distance, the first curvature similarity, and the first orientation similarity.

[0142] In some embodiments, the method for determining the sampling points as described above includes: determining M third sampling points on the prediction center line and M fourth sampling points corresponding to the M third sampling points on the first target lane line, wherein M is an integer greater than 2, for example, M is an integer such as 2, 3, 5, 10 or 20, which is not limited in this application.

[0143] In some embodiments, to ensure that the determined M third sampling points and M fourth sampling points are evenly distributed on the predicted center line and the first target lane marking line, respectively, the third sampling interval of the third sampling points and the fourth sampling interval of the fourth sampling points can be determined according to the lengths of the predicted center line and the first lane marking line. Furthermore, M first sampling points are determined on the predicted center line according to the third sampling interval, and M fourth sampling points are determined on the first lane marking line according to the fourth sampling interval.

[0144] For example, if the predicted centerline is 100 meters long and the first target lane line is 120 meters long, with M=20 sampling points, 5 meters can be used as the third sampling interval and 6 meters as the fourth sampling interval. A third sampling point can be determined every 5 meters on the predicted centerline and a fourth sampling point can be determined every 6 meters on the first target lane line.

[0145] It is understood that, in some embodiments, the first distance, the first curvature similarity, and the first direction similarity between the predicted centerline and the first labeled lane line are determined by the distance, curvature difference, and the angle between the tangent vector directions at M sampling points on the predicted centerline and the first labeled lane line.

[0146] Therefore, in order to obtain the first distance, the first curvature similarity, and the first direction similarity, the distance, curvature difference, and the angle between the tangent vector directions of the M third sampling points and the corresponding fourth sampling points can be calculated respectively.

[0147] In some embodiments, the fourth sampling point corresponding to the third sampling point may be the fourth sampling point that is closest to the third sampling point. Specifically, the closest fourth sampling point can be determined by formula (1).

[0148] (1)

[0149] in, It is located at the center line of the prediction The m-th third sampling point on the first target label line, q is located on the first target label line. The fourth sampling point on, yes The minimum distance between q and , where m is a positive integer increasing from 1 to M.

[0150] It can be understood that, according to formula (1), the fourth sampling point q that is closest to the i-th third sampling point can be determined. Furthermore, the distance, curvature difference, and angle between the third sampling point and the corresponding fourth sampling point can be determined.

[0151] In some embodiments, the method for calculating the distance at the m-th third sampling point has been shown by formula (1), and will not be repeated here.

[0152] In some embodiments, the curvature difference at the m-th third sampling point can be obtained by calculating the curvature difference between the m-th third sampling point and the fourth sampling point q, which is closest to the m-th third sampling point.

[0153] For example, the curvature at the m-th third sampling point is 0.0042. The curvature at the fourth sampling point q, which is closest to the m-th third sampling point, is 0.0007. In this case, the curvature difference can be 0.0042 - 0007 = 0.0035. .

[0154] In some embodiments, the angle between the tangent vector directions at the m-th third sampling point can be obtained by calculating the angle between the tangent vector at the m-th third sampling point and the tangent vector at the fourth sampling point q, which is closest to the m-th third sampling point.

[0155] For example, if the angle between the tangent vector at the m-th third sampling point and the tangent vector at the fourth sampling point q, which is closest to the m-th third sampling point, is 30°, then in this case, the angle between the directions of the tangent vectors can be 30°.

[0156] According to the above embodiment, the distance, curvature difference, and angle between the tangent vector directions of the M third sampling points and the corresponding fourth sampling points can be obtained. Furthermore, the first distance, the first curvature similarity, and the first direction similarity can be obtained based on the average distance, the average curvature difference, and the average angle between the tangent vector directions, respectively.

[0157] For example, in the calculation of the average distance, when P=5, the curvature difference between the five third sampling points and the corresponding fourth sampling point is 0.0042. -0.0018 0.0025 0.0007 With –0.0030 Therefore, the average curvature difference is (0.0042–0.0018+0.0025+0.0007–0.0030) / 5=0.00052 .

[0158] It is understandable that the calculation of the average value of the curvature difference and the average value of the angle between the tangent vector directions can be referred to the example above, and will not be repeated here.

[0159] In some embodiments, when determining the target annotation centerline, it is also determined which side of the predicted centerline the first target annotation centerline and the second target annotation centerline are located on, respectively. For example, if one of the directions pointed to by the tangent vectors of each sampling point of the target annotation centerline is determined as the first direction, the first side can be determined as the left side of the target annotation centerline, and the second side can be determined as the right side of the target annotation centerline.

[0160] In some embodiments, in order to determine the lane line located to the left or right of the target annotation center line, the pointing vector from the first sampling point on the target annotation center line to the second sampling point on the lane line can be obtained according to formula (2).

[0161] (2)

[0162] in, It points to a vector. These are the coordinates of the p-th sampling point on the target annotation centerline C. These are the coordinates of the p-th sampling point on lane line L. Here, p is a positive integer increasing sequentially from 1 to P; for example, p can be 1, 2, 3, or P.

[0163] In some embodiments, a normal vector pointing to the left of the target lane line and perpendicular to the tangent vector can be determined. Therefore, when and When the directions are the same, the lane line L is to the left of the target lane line. Therefore, by statistically analyzing the directions of all pointing vectors from the target lane center line C to lane line L, we can determine whether lane line L is to the left or right of the target lane center line based on the direction that appears most frequently.

[0164] In some embodiments, in order to determine and Whether the directions are the same can be determined by formula (3) to determine the pointing vector. In the normal vector The sign of the projection is determined based on the sign value. and Are the directions the same?

[0165] (3)

[0166] in, Pointer vector In the normal vector Projection on It is the normal vector. It is a pointer vector.

[0167] For example, for a target annotation centerline corresponding to 50 sampling points, if the target annotation centerline and a certain lane line have 30 sampling points... A positive sign indicates that the lane marking is located to the left of the target lane marking centerline as described above. Therefore, this lane marking can be identified as the first lane marking on the left.

[0168] This application also provides a computer-readable storage medium. The computer-readable storage medium can be any storage medium (e.g., magnetic medium, optical medium, semiconductor medium, etc.) capable of storing and / or retrieving data by a computing device. The computer-readable storage medium includes instructions that direct the computing device to implement the model training method provided in this application.

[0169] This application also provides a program product, which may be a software or program product including instructions, capable of running on a computing device or stored on any usable medium. When the computer program product runs on at least one computing device, the at least one computing device implements the model training method provided in this application.

[0170] This application also provides an electronic device, including one or more processors and one or more memories, wherein the one or more processors can be used to execute instructions to implement the model training method provided in this application.

[0171] For example, Figure 5 A block diagram of an electronic device 400 according to an embodiment of this application is shown. The electronic device 400 may include one or more processors 401 coupled to a controller hub 403. In at least one embodiment, the controller hub 403 communicates with the processor 401 via a multi-branch bus such as a Front Side Bus (FSB), a point-to-point interface such as a QuickPath Interconnect (QPI), or a similar network interface 406. The processor 401 executes instructions controlling general types of data processing operations. In some embodiments, the controller hub 403 may include, but is not limited to, a Graphics & Memory Controller Hub (GMCH) (not shown) and an Input / Output Hub (IOH) (which may be on separate chips) (not shown), wherein the GMCH may include memory and a graphics controller and is coupled to the IOH.

[0172] Electronic device 400 may also include a coprocessor 402 and a memory 404 coupled to a controller hub 403. Alternatively, one or both of the memory and the GMCH may be integrated within the processor (as described in the embodiments of this application), with memory 404 and coprocessor 402 directly coupled to processor 401 and controller hub 403, which is located on a single chip with IOH. Memory 404 may be, for example, dynamic random access memory (DRAM), phase change memory (PCM), or a combination of both.

[0173] The memory 404 may include one or more tangible, non-transitory computer-readable media for storing data and / or instructions. The computer-readable storage medium stores instructions, specifically, temporary and permanent copies of those instructions. The instructions may include, when executed by at least one of the processors, causing the electronic device 400 to perform, as... Figure 5 The instructions for the method are shown. When the instructions are executed on a computer, the computer performs the model training method disclosed in the embodiments of this application.

[0174] In one embodiment, coprocessor 402 is a dedicated processor, such as, for example, a high-throughput many integrated core (MIC) processor, a network or communication processor, a compression engine, a graphics processor, a general-purpose computing on graphics processing units (GPGPU), or an embedded processor, etc. Optional properties of coprocessor 402 are indicated by dashed lines. Figure 5 middle.

[0175] In one embodiment, electronic device 400 may further include a Network Interface Controller (NIC) 406. The network interface 406 may include a transceiver for providing a radio interface for electronic device 400 to communicate with any other suitable device, such as a front-end module, antenna, etc. In various embodiments, the network interface 406 may be integrated with other components of electronic device 400. The network interface 406 can implement the functions of the communication unit in the above embodiments.

[0176] Electronic device 400 may further include input / output (I / O) devices 405. I / O 405 may include: a user interface designed to enable a user to interact with electronic device 400; a peripheral component interface designed to enable peripheral components to also interact with electronic device 400; and / or sensors designed to determine environmental conditions and / or location information related to electronic device 400.

[0177] It is worth noting that, Figure 5 This is merely an example. That is, although... Figure 5 The illustration shows an electronic device 400 including multiple devices such as a processor 401, a controller hub 403, and a memory 404. However, in actual applications, devices using the methods of the embodiments of this application may include only a portion of the devices in the electronic device 400. For example, it may include only the processor 401 and the network interface 406. Figure 5 The properties of the optional devices are shown by dashed lines.

[0178] Figure 6 A block diagram of a System-on-Chip (SoC) 500 applied to an electronic device according to an embodiment of this application is shown. Figure 6 In the diagram, similar components share the same reference numerals. Additionally, dashed boxes are an optional feature for more advanced SoCs. Figure 6 In this embodiment, SoC 500 includes: an interconnect unit 550 coupled to processor 510; a system proxy unit 580; a bus controller unit 590; an integrated memory controller unit 540; a group or one or more coprocessors 520, which may include integrated graphics logic, an image processor, an audio processor, and a video processor; a static random-access memory (SRAM) unit 530; and a direct memory access (DMA) unit 560. In one embodiment, coprocessor 520 includes a dedicated processor, such as, for example, a network or communication processor, a compression engine, general-purpose computing on graphics processing units (GPGPU), a high-throughput MIC processor, or an embedded processor.

[0179] Static Random Access Memory (SRAM) cell 530 may include one or more tangible, non-transitory computer-readable media for storing data and / or instructions. The computer-readable storage medium stores instructions, specifically, temporary and permanent copies of those instructions. These instructions may include: instructions that, when executed by at least one processor, cause the SoC to implement the methods shown in the embodiments of this application. When the instructions are executed on a computer, they cause the computer to perform the model training methods disclosed in the embodiments of this application.

[0180] It should be noted that the terminology used in the embodiment section of this application is only for explaining specific embodiments of this application and is not intended to limit this application. In the description of the embodiments of this application, unless otherwise stated, " / " means "or", for example, A / B can mean A or B; "and / or" in this document is merely a description of the relationship between associated obstacles, indicating that three relationships can exist, for example, A and / or B can represent: A alone, A and B simultaneously, and B alone. In addition, in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more, "at least one" or "one or more" means one, two or more.

[0181] It should be noted that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.

[0182] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0183] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product may include one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium, or a semiconductor medium, etc.

[0184] Those skilled in the art will understand that implementing all or part of the processes in the above embodiments can be accomplished by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium can include various media capable of storing program code, such as read-only memory, random access memory, magnetic disks, or optical disks.

[0185] The above description is merely a specific embodiment of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be determined by the protection scope of the claims.

Claims

1. A model training method, characterized in that, The method includes: Obtain sample images of the target road; The sample image is input into the model to be trained to obtain the predicted centerline of the target road; Select the target labeled centerline corresponding to the predicted centerline from at least one labeled centerline of the target road; Obtain two target lane lines corresponding to the target annotation center line, wherein the two target lane lines are located on both sides of the target annotation center line; If the predicted center line and the two target lane marking lines do not satisfy the position matching relationship, the model parameters of the model to be trained are adjusted.

2. The method according to claim 1, characterized in that, The target road has multiple marked center lines, and Selecting the target labeled centerline corresponding to the predicted centerline from at least one labeled centerline of the target road includes: The distance between each of the marked center lines and the predicted center line is determined; The label centerline that has the shortest distance from the predicted centerline is taken as the target label centerline.

3. The method according to claim 2, characterized in that, The distance between the marked center line and the predicted center line is determined in the following way: Determine P first sampling points on the marked center line and P second sampling points on the predicted center line corresponding to the P first sampling points, where P is an integer greater than 2; Calculate the distance difference between each of the P first sampling points and the corresponding second sampling point, and use the average of the distance differences as the distance between the labeled center line and the predicted center line.

4. The method according to claim 1, characterized in that, The two target lane marking lines include a first target lane marking line and a second target lane marking line. The predicted center line has a first distance, a first curvature similarity, and a first direction similarity with the first target lane marking line. The predicted center line has a second distance, a second curvature similarity, and a second direction similarity with the second target lane marking line. The location matching relationship includes: The distance difference between the first distance and the second distance is less than the distance threshold; The curvature similarity difference between the first curvature similarity and the second curvature similarity is less than the curvature threshold; The difference in directional similarity between the first directional similarity and the second directional similarity is less than the directional threshold.

5. The method according to claim 4, characterized in that, The first distance is determined in the following way: Determine M third sampling points on the prediction center line and M fourth sampling points on the first target label lane line corresponding to the M third sampling points, where M is an integer greater than 2; Calculate the distance between each of the M third sampling points and the corresponding fourth sampling point, and use the average of the distances as the first distance.

6. The method according to claim 5, characterized in that, The first curvature similarity is determined in the following way: Calculate the curvature difference between the M third sampling points and the corresponding fourth sampling points, and use the average value of the curvature difference as the first curvature similarity.

7. The method according to claim 5, characterized in that, The first directional similarity is determined in the following way: Calculate the angle between the tangent vector directions of the M third sampling points and the corresponding fourth sampling points, and use the average value of the angle between the tangent vector directions as the first directional similarity.

8. The method according to claim 1, characterized in that, The two target label lane lines include a first target label lane line located on the first side of the target label center line; and The step of obtaining the two target lane lines corresponding to the target center line includes: Obtain multiple first lane marking lines located on the first side of the target marking center line; From the plurality of first lane marking lines, select i first lane marking lines with the smallest distance to the center line of the target marking to form a first lane marking set, where i is a positive integer greater than or equal to 2; From the plurality of first lane marking lines, select j first lane marking lines with the highest curvature similarity to the center line of the target marking line to form a second lane marking set, where j is a positive integer greater than or equal to 2; From the plurality of first lane marking lines, select the k first lane marking lines with the highest directional similarity to the target marking center line to form a third lane marking set, where k is a positive integer greater than or equal to 2; The first target lane line is determined based on the first set of lane lines, the second set of lane lines, and the third set of lane lines.

9. The method according to claim 8, characterized in that, The step of determining the first target lane marking line based on the first lane line set, the second lane line set, and the third lane line set includes: Determine the first intersection of the first set of lane lines, the second set of lane lines, and the third set of lane lines; The number of first labeled lane lines in the first intersection is one, and one first labeled lane line in the first intersection is taken as the first target labeled lane line; If the number of first lane marking lines in the first intersection is equal to or greater than two, then the first lane marking line in the first intersection that is closest to the center line of the target marking is taken as the first target marking lane line.

10. A readable storage medium, characterized in that, The readable storage medium includes one or more programs that, when executed on an electronic device, cause the electronic device to perform the method according to any one of claims 1 to 9.

11. A program product, characterized in that, When the program product is executed on an electronic device, it causes the electronic device to perform the method described in any one of claims 1 to 9.

12. An electronic device, characterized in that, include: At least one memory for storing one or more programs; At least one processor is configured to execute the one or more programs to cause the electronic device to perform the method of any one of claims 1 to 9.

Images