Lane line detection model training method, lane line detection method, vehicle and medium

Abstract

The application relates to a lane line detection model training method, a lane line detection method, a vehicle and a storage medium. The lane line detection model training method comprises the following steps: acquiring a sample road image; calculating prediction data of each parameter of a lane line in the sample road image according to the sample road image, wherein the parameters comprise a horizontal coordinate of a starting point of the lane line, a vertical coordinate of the starting point of the lane line, an included angle between the lane line and a horizontal direction of the bottom of the sample road image, and a length of the lane line; acquiring a loss function corresponding to each parameter respectively; wherein the loss functions corresponding to at least two parameters are different; and training a lane line detection model according to the prediction data of each parameter, real data of each parameter and the loss function corresponding to each parameter. The method can improve the accuracy of lane line detection model training and thus improve the accuracy of lane line detection.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and in particular to a lane line detection model training method, a lane line detection method, a vehicle, and a storage medium. Background Technology

[0002] With the development of autonomous driving technology, lane detection technology has emerged. Objects on the road can be divided into two main categories: static objects and dynamic objects. Lane lines are one of the main static objects on the road. Better detection of lane lines can provide more accurate prior knowledge for downstream tasks of autonomous driving. Therefore, lane detection technology has become an important part of autonomous driving technology.

[0003] Traditional lane detection models use the smooth function to calculate the loss between predicted and actual lane data. However, when using the smooth function for loss calculation, if the loss value of one lane parameter is too large or calculated incorrectly, it will affect the regression of the loss values ​​of other parameters, thus affecting the accuracy of lane detection model training. Consequently, this leads to low accuracy in lane detection based on lane detection models. Summary of the Invention

[0004] Therefore, it is necessary to provide a lane line detection model training method, lane line detection method, vehicle, and storage medium that can improve the accuracy of lane line detection, addressing the aforementioned technical problems.

[0005] A method for training a lane detection model is disclosed. The lane detection model is used for lane detection. The method includes: acquiring a sample road image; calculating predicted data for each parameter of the lane line in the sample road image based on the sample road image, including the abscissa of the lane line starting point, the ordinate of the lane line starting point, the angle between the lane line and the bottom horizontal direction of the sample road image, and the lane line length; acquiring the loss function corresponding to each parameter; wherein at least two parameters have different loss functions; and training the lane detection model based on the predicted data of each parameter, the actual data of each parameter, and the loss function corresponding to each parameter.

[0006] In one or more embodiments, the lane line angle corresponds to a first loss function, the abscissa of the lane line starting point corresponds to a second loss function, the ordinate of the lane line starting point corresponds to a third loss function, and the lane line length corresponds to a fourth loss function. The lane line detection model is trained based on the predicted data of each parameter, the actual data of each parameter, and the corresponding loss function. This includes: calculating a first loss value based on the predicted data of the lane line angle, the actual data of the lane line angle, and the first loss function; calculating a second loss value based on the predicted data of the abscissa of the lane line starting point, the actual data of the abscissa of the lane line starting point, the first loss value, and the second loss function; calculating a third loss value based on the predicted data of the ordinate of the lane line starting point, the actual data of the ordinate of the lane line starting point, the first loss value, and the third loss function; calculating a fourth loss value based on the predicted data of the lane line length, the actual data of the lane line length, and the fourth loss function; and training the lane line detection model based on the first, second, third, and fourth loss values.

[0007] In one or more embodiments, the magnitude of the first loss value is inversely proportional to the magnitude of the second loss value; and / or the magnitude of the first loss value is inversely proportional to the magnitude of the third loss value.

[0008] In one or more embodiments, the value range of the first loss value, the second loss value, the third loss value, and the fourth loss value is [0, 1].

[0009] In one or more embodiments, the method for generating the second loss function includes: obtaining a first loss function and an exponential function with the natural constant e as the base; generating a negative exponent of the exponential function based on the first loss function; and generating a second loss function based on the exponential function and the negative exponent.

[0010] In one or more embodiments, the method for generating the third loss function includes: obtaining a first loss function and an exponential function with the natural constant e as the base; generating a negative exponent of the exponential function based on the first loss function; and generating a third loss function based on the exponential function and the negative exponent.

[0011] In one or more embodiments, the formula for the first loss function is:

[0012]

[0013] Where L1 is the first loss function, For the predicted lane line angle, θ j is the actual data of the lane line angle, and j is the index of the input sample road image.

[0014] In one or more embodiments, the formula for the second loss function is:

[0015]

[0016] Where L2 is the second loss function, The predicted data for the x-coordinate of the starting point of the lane line, x j is the actual x-coordinate of the starting point of the lane line, and j is the index of the input sample road image.

[0017] In one or more embodiments, the formula for the third loss function is:

[0018]

[0019] Where L3 is the third loss function, The predicted data for the ordinate of the starting point of the lane line, y j represents the actual ordinate of the starting point of the lane line, and j is the index of the input sample road image.

[0020] In one or more embodiments, the formula for the fourth loss function is:

[0021]

[0022] Where L4 is the fourth loss function, For the predicted lane length, l j is the actual data for lane line length, and j is the index of the input sample road image.

[0023] In one or more embodiments, calculating prediction data for each parameter of lane lines in a sample road image based on a sample road image includes: inputting the sample road image into a feature extraction backbone network to obtain a backbone feature map output by the feature extraction backbone network; inputting the backbone feature map into a feature pyramid network to obtain multiple first feature maps output by the feature pyramid network; wherein each first feature map has a different size; inputting the multiple first feature maps into a feature refinement network to obtain second feature maps corresponding to each first feature map output by the feature refinement network; and calculating prediction data for each parameter of lane lines in the sample road image based on each second feature map and a head attention mechanism.

[0024] In one or more embodiments, calculating predicted data for lane line angles based on sample road images includes: uniformly sampling multiple sampling points on the predicted lane lines; calculating the angle corresponding to each sampling point based on the predicted data of the abscissa of each sampling point, the abscissa of the starting point of the lane line, the number of sampling points, the height of the sample road image, and the arctangent function; and obtaining predicted data for lane line angles based on the angles corresponding to each sampling point.

[0025] A lane detection method includes: acquiring a road image to be detected; and performing lane detection based on the road image to be detected and a trained lane detection model; wherein the lane detection model is a model trained according to any of the above methods.

[0026] A vehicle includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the lane detection method described above.

[0027] A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the lane detection model training method described above.

[0028] The aforementioned lane detection model training method, lane detection method, vehicle, and storage medium, in the process of optimizing the model training using loss functions, respectively utilize the loss functions corresponding to each parameter of the lane line, as well as their corresponding predicted and real data, to calculate the loss. Moreover, the loss functions corresponding to the parameters of at least two lane lines have different structures. Therefore, by adopting this method, the problems of inaccurate regression and inaccurate training caused by single loss calculation can be avoided, thereby improving the training accuracy of the lane detection model and thus improving the lane detection accuracy. Attached Figure Description

[0029] Figure 1 This is a flowchart illustrating the lane detection model training method in one or more embodiments;

[0030] Figure 2 This is a flowchart illustrating the steps of training a lane detection model based on the predicted data of each parameter, the actual data of each parameter, and the loss function corresponding to each parameter in one or more embodiments.

[0031] Figure 3 This is a schematic diagram of the curves corresponding to the smoothl1 loss function, the first loss function, the second loss function, and the third loss function in one or more embodiments;

[0032] Figure 4 This is a schematic diagram showing the distribution of the calculation results of the fourth loss function in one or more embodiments;

[0033] Figure 5 This is a schematic diagram illustrating the training process of the lane detection model in one or more embodiments;

[0034] Figure 6 This is a flowchart illustrating the lane line detection method in one or more embodiments;

[0035] Figure 7This is a schematic diagram of the structure of an automated control system for a vehicle in one or more embodiments. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0037] In one or more embodiments, the lane detection model training method of this application can be applied to a computer device. It should be noted that the executing entity can be a virtual network interface card (NIC) resource configuration device. This device can be implemented as part or all of the computer device through software, hardware, or a combination of both. The computer device can be a terminal, client, or server. The server can be a single server or a server cluster composed of multiple servers. The terminal can be an in-vehicle terminal, smartphone, personal computer, tablet computer, wearable device, or other intelligent hardware device such as an intelligent robot. In the following embodiments of the lane detection model training method, the executing entity is always a computer device.

[0038] In one or more embodiments, such as Figure 1 As shown, a method for training a lane detection model is provided. The lane detection model is used for lane detection. Taking the application of this method to a computer device as an example, it can include the following steps:

[0039] Step S102: Obtain sample road images.

[0040] The sample road images refer to road images used as training samples for the lane detection model. The number of sample road images is unlimited and can be customized according to training needs. Sample road images can be acquired using an image acquisition device and may include vehicles, lanes, lane lines, and roadside scenery; that is, images reflecting the road conditions on which vehicles travel. Sample road images can be pre-processed and labeled before being used as samples for training the lane detection model. The format of the sample road images can be JPG, JPEG, or GIF, and the size is not limited; all can be adjusted according to the input parameters of the lane detection model.

[0041] Specifically, computer equipment can acquire multiple sample road images from a sample set as training samples for the lane detection model.

[0042] Step S104: Calculate the predicted data of each parameter of the lane line in the sample road image based on the sample road image. The parameters include the abscissa of the starting point of the lane line, the ordinate of the starting point of the lane line, the angle between the lane line and the bottom horizontal direction of the lane line in the sample road image, and the length of the lane line.

[0043] The parameters of the lane lines refer to the parameters used to describe the position, length, direction, and other related information of the lane lines in the sample image. Lane line parameters may include the x-coordinate of the lane line's starting point, the y-coordinate of the lane line's starting point, the angle between the lane line and the bottom horizontal direction of the sample road image, and the lane line length.

[0044] Specifically, the computer device can use sample road images as input to the lane detection model. The lane detection model can be a model built based on a neural network structure, such as the CLR Net (Cross Layer Refinement Network) model. Through the calculation of each network structure of the lane detection model, the predicted data of each parameter of the lane lines in the sample road image can be obtained.

[0045] Step S106: Obtain the loss function corresponding to each parameter; wherein, at least two parameters have different loss functions.

[0046] After calculating the predicted data for each parameter, the loss function corresponding to each parameter is called according to each parameter. The loss functions for at least two different parameters are constructed differently.

[0047] For example, the lane line angle can correspond to the angle loss function L1, the horizontal coordinate of the lane line starting point can correspond to the horizontal coordinate loss function L2, the vertical coordinate of the lane line starting point can correspond to the vertical coordinate loss function L3, and the lane line length can correspond to the length loss function L4. Here, L1, L2, L3, and L4 are loss functions corresponding to different parameters, and at least two of the loss functions L1, L2, L3, and L4 are constructed differently.

[0048] Step S108: Train the lane detection model based on the predicted data of each parameter, the real data of each parameter, and the loss function corresponding to each parameter.

[0049] In this context, "real data" refers to the lane line data in the sample road image, which is extracted in advance based on real information such as the position, size, and direction of lane lines in the sample road image. "Predicted data" refers to the lane line data in the sample road image predicted by the lane line detection model, which uses the sample road image as input.

[0050] In the optimization training of traditional lane detection models, the loss is calculated by directly applying the smoothl1 function to the predicted and real data. The model's loss calculation formula is as follows:

[0051]

[0052] in, This represents the predicted data for various parameters of the lane markings, y ij Let represent the actual data for each parameter of the lane line, where i represents the index of the lane line parameter, j represents the index of the sample road image, N is the number of lane line parameters, and K is the number of sample road images. As can be seen from the traditional model loss calculation formula, the smoothl1 function is uniformly used for optimization regardless of whether the parameters are the same. However, in reality, the deviations between the actual and predicted data for parameters such as the x-coordinate of the lane line's starting point, the y-coordinate of the lane line's starting point, the angle θ between the lane line and the bottom horizontal direction of the sample road image, and the lane line length (l) are not the same. Therefore, using the same loss function for gradient descent optimization is unreasonable.

[0053] Therefore, based on the parameters of the lane line, a corresponding loss function can be set for each parameter. Each parameter can then be optimized on a parameter-by-parameter basis using its respective loss function, as well as its corresponding predicted and real data. Finally, the entire lane line detection model can be optimized and trained based on the optimization results of each parameter.

[0054] For example, the loss calculation formula of the lane detection model in this application embodiment can be:

[0055]

[0056] Among them, L i Let i be the loss function corresponding to parameter i, N be the number of lane line parameters, and K be the number of sample road images. This represents the predicted data for various parameters of the lane markings, y ij This represents the actual data for each parameter of the lane line, where i represents the index of the lane line parameter and j represents the index of the sample road image. L i There are at least two different structures.

[0057] For example, the lane line angle corresponds to L1, the lane line starting point coordinates correspond to L2 and L3 (the horizontal coordinate corresponds to L2, and the vertical coordinate corresponds to L3), and the lane line length corresponds to L4. The lane line angle loss function (L1), the lane line starting point coordinate loss function (L2 and L3), and the lane line length loss function (L4) are constructed differently. The construction of L2 and L3 can be the same. Of course, the construction of the loss functions corresponding to each parameter can also be different.

[0058] The lane detection model training method described above calculates the loss using the loss function corresponding to each parameter of the lane line, as well as the corresponding predicted and real data, during the model optimization training process. Moreover, the loss function structures corresponding to the parameters of at least two lane lines are different. Therefore, this method can avoid the problems of inaccurate regression and inaccurate training caused by single loss calculation, thereby improving the training accuracy of the lane detection model and thus improving the lane detection accuracy.

[0059] In one or more embodiments, reference Figure 2 As shown, Figure 2 The diagram illustrates the steps of training a lane detection model based on predicted data for each parameter, actual data for each parameter, and the loss function corresponding to each parameter in one or more embodiments.

[0060] The lane line angle corresponds to the first loss function, the x-coordinate of the lane line starting point corresponds to the second loss function, the y-coordinate of the lane line starting point corresponds to the third loss function, and the lane line length corresponds to the fourth loss function. The lane line detection model is trained based on the predicted data for each parameter, the actual data for each parameter, and the corresponding loss function. This includes:

[0061] Step S202: Calculate the first loss value based on the predicted lane line angle data, the actual lane line angle data, and the first loss function;

[0062] Step S204: Calculate the second loss value based on the predicted data of the abscissa of the starting point of the lane line, the actual data of the abscissa of the starting point of the lane line, the first loss value, and the second loss function;

[0063] Step S206: Calculate the third loss value based on the predicted data of the ordinate of the starting point of the lane line, the actual data of the ordinate of the starting point of the lane line, the first loss value, and the third loss function;

[0064] Step S208: Calculate the fourth loss value based on the predicted lane length data, the actual lane length data, and the fourth loss function; and

[0065] Step S210: Train the lane detection model based on the first loss value, the second loss value, the third loss value, and the fourth loss value.

[0066] In the above embodiments, when calculating the loss of the two lane line parameters, the abscissa and ordinate of the lane line starting point, in addition to their respective loss functions, predicted data, and actual data, the loss calculation result (loss value) of the lane line angle is also incorporated. That is, before optimizing the loss of the abscissa and / or ordinate of the lane line starting point, the loss of the lane line angle is optimized first. By establishing a correlation between the lane line angle and the loss optimization of the abscissa or ordinate of the lane line starting point, optimizing the loss of the lane line angle first, and then optimizing the loss of the abscissa and / or ordinate of the lane line starting point, the optimization accuracy of the lane line starting point coordinate can be improved when the loss of the lane line angle is small.

[0067] Since the distances between sampling points on the predicted lane lines and sampling points on the real lane lines will be different when the predicted lane line angles are different, this distance reflects the predicted coordinates of the lane line starting point. Therefore, prioritizing the optimization of the lane line angle loss can better optimize the loss value of the lane line starting point coordinates.

[0068] In one or more embodiments, the magnitude of the first loss value is inversely proportional to the magnitude of the second loss value; and / or the magnitude of the first loss value is inversely proportional to the magnitude of the third loss value. Wherein, the first loss value is the loss value corresponding to the lane line angle, the second loss value is the loss value corresponding to the abscissa of the lane line starting point, and the third loss value is the loss value corresponding to the ordinate of the lane line starting point.

[0069] In the above embodiments, by limiting the inverse relationship between the loss value corresponding to the lane line angle and the loss value corresponding to the abscissa and / or ordinate of the lane line starting point, the loss contribution of the abscissa or ordinate of the lane line starting point can be reduced when the difference between the predicted and actual data of the lane line angle is too large; conversely, the loss contribution of the abscissa or ordinate of the lane line starting point can be increased when the difference between the predicted and actual data of the lane line angle is too small. This balances the loss value of the entire lane line detection model, preventing the loss of the entire lane line detection model from being too large and failing to regress well, thereby improving the training accuracy of the lane line detection model.

[0070] In one or more embodiments, the method for generating the second loss function includes: obtaining a first loss function and an exponential function with the natural constant e as the base; generating a negative exponent of the exponential function based on the first loss function; and generating a second loss function based on the exponential function and the negative exponent.

[0071] In one or more embodiments, the method for generating the third loss function includes: obtaining a first loss function and an exponential function with the natural constant e as the base; generating a negative exponent of the exponential function based on the first loss function; and generating a third loss function based on the exponential function and the negative exponent.

[0072] In the above embodiments, loss functions corresponding to the horizontal and vertical coordinates of the starting point of the lane line are constructed based on the loss function corresponding to the lane line angle. That is, a second loss function and / or a third loss function are constructed based on the first loss function. Moreover, through the correlation between the exponential function with the natural constant e as the base and its negative exponent, it is possible to restrict the loss calculation result of the lane line angle to be inversely proportional to the loss calculation result of the horizontal coordinate and / or vertical coordinate of the starting point of the lane line.

[0073] In one or more embodiments, the value ranges of the first loss value, the second loss value, the third loss value, and the fourth loss value are [0, 1].

[0074] In traditional methods, when the predicted data differs significantly from the actual data, the loss values ​​of each parameter become increasingly large. Therefore, after summing the loss values ​​of all parameters for the lane detection model, the overall loss value will also be substantial. If the loss value of any one parameter is too large, it will affect the regression of other loss values ​​and the overall model's loss value. The above embodiment, by limiting the range of the loss values ​​corresponding to each parameter to [0, 1], effectively restricts the range of the loss values ​​of each parameter, improving the regression efficiency of the overall model's loss calculation.

[0075] Below, with specific application examples, the calculation formulas for the loss functions of each parameter are given as examples. The following calculation formulas are only shown as examples. Other formulas generated by performing formal transformations, coefficient adjustments, etc., without departing from the concept of this application, are all within the protection scope of this application.

[0076] In one or more embodiments, the formula for the first loss function is:

[0077]

[0078] Where L1 is the first loss function, For the predicted lane line angle, θ j Here, j represents the actual lane line angle data, and j is the index of the input sample road image. As can be seen from the calculation formula of the first loss function above, L1 uses sin... 2 A function that can restrict the range of values ​​to [0, 1].

[0079] In one or more embodiments, the formula for the second loss function is:

[0080]

[0081] Where L1 is the first loss function and L2 is the second loss function. The predicted data for the x-coordinate of the starting point of the lane line, x j Here, represents the actual x-coordinate of the lane line starting point, and j is the index of the input sample road image. From the calculation formula of the second loss function above, it can be seen that since the value range of L1 is within [0, 1], therefore, e -value Since the range is within [0, 1], the range of the second loss value corresponding to the second loss function will also be limited to [0, 1]. Furthermore, L1 and L2 satisfy the following condition:

[0082] L1↑→e -value ↑→L2↓

[0083] L1↓→e -value ↓→L2↑

[0084] In one or more embodiments, the formula for the third loss function is:

[0085]

[0086] Where L1 is the first loss function and L3 is the third loss function. The predicted data for the ordinate of the starting point of the lane line, y j Here, represents the actual ordinate of the lane line starting point, and j is the index of the input sample road image. From the calculation formula of the third loss function above, it can be seen that since the value range of L1 is within [0, 1], therefore, e -value The range will be limited to [0, 1], therefore, the range of the second loss value corresponding to the second loss function will also be limited to [0, 1]. Furthermore, L1 and L3 satisfy the following condition:

[0087] L1↑→e -value ↑→L3↓

[0088] L1↓→e -value ↓→L3↑

[0089] In one or more embodiments, the formula for the fourth loss function is:

[0090]

[0091] Where L4 is the fourth loss function, For the predicted lane length, l j is the actual data for lane line length, and j is the index of the input sample road image.

[0092] Below, for reference Figure 3 and Figure 4 As shown, Figure 3 The diagram shows the curves corresponding to the smoothl1 loss function, the first loss function, the second loss function, and the third loss function. Curve 300 corresponds to the smoothl1 loss function, curve 301 corresponds to the first loss function, and curve 302 corresponds to either the second or third loss function. Figure 3 As can be seen, when using the smoothl1 loss function, the loss value increases as the difference between the predicted and actual data for each parameter increases. An excessively large loss value for any one parameter can negatively impact the overall model's loss optimization result. However, by using the loss functions corresponding to each parameter in this embodiment for loss calculation, the loss value for each parameter can be limited to the range [0,1]. Even if the difference between the predicted and actual data for a certain parameter is too large, it will not affect the regression of other parameters or the overall model's loss optimization result.

[0093] Figure 4 A schematic diagram showing the distribution of the calculation results of the fourth loss function in one or more embodiments is illustrated. From Figure 4 As can be seen, the fourth loss value 400 is distributed in the range of [0,1].

[0094] In one or more embodiments, calculating prediction data for each parameter of lane lines in a sample road image based on a sample road image includes: inputting the sample road image into a feature extraction backbone network to obtain a backbone feature map output by the feature extraction backbone network; inputting the backbone feature map into a feature pyramid network to obtain multiple first feature maps output by the feature pyramid network; wherein each first feature map has a different size; inputting the multiple first feature maps into a feature refinement network to obtain second feature maps corresponding to each first feature map output by the feature refinement network; and calculating prediction data for each parameter of lane lines in the sample road image based on each second feature map and a head attention mechanism.

[0095] For example, refer to Figure 5 As shown, Figure 5 A schematic diagram illustrating the training process of a lane detection model in one or more embodiments is shown. Figure 5 As shown, the lane detection model can be built based on the CLR Net model, which can specifically include the following steps:

[0096] 1. The input sample road image can be 800*320*3 image data. Wherein, the width is 800px, the height is 320px, and 3 represents the number of channels.

[0097] 2. The input sample road image first passes through a feature extraction backbone network, such as the ResNet34 deep residual network.

[0098] 3. Then, the backbone feature map output by the feature extraction backbone network is input into FPN (Feature Pyramid Networks), which outputs feature maps of various sizes as the first feature map. For example, it outputs feature maps of three sizes as the first feature map.

[0099] 4. The first feature maps of different sizes output by the FPN are then input into their respective feature refinement networks to obtain the refined feature maps corresponding to the first feature maps output by the feature refinement networks as second feature maps. For example, the second feature maps may include three different sizes of feature maps, with sizes of [B, 64, 10, 25], [B, 64, 20, 50], and [B, 64, 40, 100], where B is the batch size, 64 is the number of channels, 10, 20, and 40 are the width, and 25, 50, and 100 are the height.

[0100] 5. Input the second feature maps of different sizes into the corresponding head attention network, and calculate the prediction data of each parameter of the lane line based on the head attention mechanism to obtain the predicted data of the abscissa of the starting point of the lane line, the ordinate of the starting point of the lane line, the angle between the lane line and the bottom horizontal direction of the sample road image, and the lane line length.

[0101] 6. Optimize the model using the loss calculation method described in any of the above embodiments, and return the optimized result to update the learning parameters in the model, thereby achieving the training of the lane detection model.

[0102] In one or more embodiments, calculating predicted data for lane line angles based on sample road images includes: uniformly sampling multiple sampling points on the predicted lane lines; calculating the angle corresponding to each sampling point based on the predicted data of the abscissa of each sampling point, the abscissa of the starting point of the lane line, the number of sampling points, the height of the sample road image, and the arctangent function; and obtaining predicted data for lane line angles based on the angles corresponding to each sampling point.

[0103] In the above embodiments, the predicted lane angle can be calculated by dividing the data into multiple sampling points, thereby improving the accuracy of lane angle prediction. More specifically, the following calculation formula can be used as a reference:

[0104]

[0105] Where, if θ n >0, θ n=θ n Otherwise, θ n =1-|θ n In the above formula, x represents the x-coordinate of the predicted lane line starting point. n The x-coordinate of the nth sampling point is represented by θ, N represents the number of sampling points, img_h represents the height of the sample road image, arctan represents the arctangent function, and θ represents the x-coordinate of the nth sampling point. n The included angles corresponding to each sampling point are summed, and the average value is taken to obtain the predicted data of the lane line included angles.

[0106] Below, this application also provides a lane line detection method. This lane line detection method can be applied to a computer device, which can be a terminal or a server. The terminal can be an in-vehicle terminal, smartphone, personal computer, tablet computer, wearable device, or other intelligent hardware device such as an intelligent robot. The server can be a single server or a server cluster consisting of multiple servers.

[0107] In one or more embodiments, such as Figure 6 As shown, a lane line detection method is provided. Taking the application of this method to an in-vehicle terminal as an example, it may include the following steps:

[0108] S602: Acquire the image of the road to be detected;

[0109] S604: Perform lane line detection based on the road image to be detected and the trained lane line detection model.

[0110] The lane detection model can be trained according to the lane detection model training method of any one or more of the above embodiments. Using the lane detection model trained according to the lane detection model training method of any one or more of the above embodiments for lane detection can improve the accuracy of lane detection and better provide prior data for downstream tasks of autonomous vehicles.

[0111] It should be understood that, although Figures 1-2 and Figure 6 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order in which these steps are executed, and they can be performed in other orders. Furthermore, Figures 1-2 and Figure 6At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0112] In one or more embodiments, a vehicle is provided whose control system can be structured as follows: Figure 7 As shown, the system may include a processor 710, a memory 720, and a network interface 730 connected via a system bus. The processor 710 provides computing and control capabilities. The memory 720 includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs, and the internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface 730 is used for communication with external terminals or servers via a network connection. When the computer program is executed by the processor 710, it implements a lane detection method.

[0113] Figure 7 The vehicle control system shown is just an example and can have more than that. Figure 7 The more or fewer components shown can be combined into two or more components, or they can have different component configurations. For example, a display screen and an input device may also be included. The display screen can be a liquid crystal display screen or an e-ink display screen, and the input device can be a touch layer covering the display screen, or buttons, a trackball, or a touchpad set on the vehicle panel, or an external keyboard, touchpad, or mouse, etc. The various components shown in the figure can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0114] In one or more embodiments, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, it performs the following steps: acquiring a sample road image; calculating predicted data of each parameter of the lane line in the sample road image based on the sample road image, the parameters including the abscissa of the lane line starting point, the ordinate of the lane line starting point, the angle between the lane line and the bottom horizontal direction of the sample road image, and the lane line length; acquiring the loss function corresponding to each parameter respectively; wherein, at least two parameters have different loss functions; and training a lane line detection model based on the predicted data of each parameter, the real data of each parameter, and the loss function corresponding to each parameter.

[0115] In one or more embodiments, the lane line angle corresponds to a first loss function, the abscissa of the lane line starting point corresponds to a second loss function, the ordinate of the lane line starting point corresponds to a third loss function, and the lane line length corresponds to a fourth loss function. When a computer program is executed by a processor to train the lane line detection model based on the predicted data of each parameter, the actual data of each parameter, and the loss function corresponding to each parameter, the specific steps are implemented as follows: calculating a first loss value based on the predicted data of the lane line angle, the actual data of the lane line angle, and the first loss function; calculating a second loss value based on the predicted data of the abscissa of the lane line starting point, the actual data of the abscissa of the lane line starting point, the first loss value, and the second loss function; calculating a third loss value based on the predicted data of the ordinate of the lane line starting point, the actual data of the ordinate of the lane line starting point, the first loss value, and the third loss function; calculating a fourth loss value based on the predicted data of the lane line length, the actual data of the lane line length, and the fourth loss function; and training the lane line detection model based on the first loss value, the second loss value, the third loss value, and the fourth loss value.

[0116] In one or more embodiments, the computer program executed by the processor further performs the following steps: obtaining a first loss function and an exponential function with the natural constant e as the base; generating a negative exponent of the exponential function based on the first loss function; and generating a second loss function based on the exponential function and the negative exponent.

[0117] In one or more embodiments, the computer program executed by the processor further performs the following steps: obtaining a first loss function and an exponential function with the natural constant e as the base; generating a negative exponent of the exponential function based on the first loss function; and generating a third loss function based on the exponential function and the negative exponent.

[0118] In one or more embodiments, when a computer program is executed by a processor to calculate prediction data for each parameter of lane lines in a sample road image based on the sample road image, the specific steps are as follows: inputting the sample road image into a feature extraction backbone network to obtain a backbone feature map output by the feature extraction backbone network; inputting the backbone feature map into a feature pyramid network to obtain multiple first feature maps output by the feature pyramid network; wherein each first feature map has a different size; inputting the multiple first feature maps into a feature refinement network to obtain second feature maps corresponding to each first feature map output by the feature refinement network; and calculating prediction data for each parameter of lane lines in the sample road image based on each second feature map and a head attention mechanism.

[0119] In one or more embodiments, when the computer program is executed by a processor, it further performs the following steps: uniformly sampling multiple sampling points on the predicted lane line; calculating the included angle corresponding to each sampling point based on the predicted data of the abscissa of each sampling point, the abscissa of the starting point of the lane line, the number of sampling points, the height of the sample road image, and the arctangent function; and obtaining the predicted data of the included angle of the lane line based on the included angle corresponding to each sampling point.

[0120] In one or more embodiments, when the computer program is executed by a processor, it further performs the following steps: acquiring a road image to be detected; and performing lane detection based on the road image to be detected and a trained lane detection model; wherein the lane detection model is a model trained according to the method of any or more of the above embodiments.

[0121] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0122] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0123] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the characters in this article generally indicate that the preceding and following related objects have an "or" relationship.

[0124] Terms such as “first” and “second” used in this application are used only to distinguish one entity or operation from another, and do not necessarily require or imply any actual relationship, sequence, or difference in importance between these entities or operations.

[0125] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for training a lane detection model, wherein the lane detection model is used for lane detection, the method comprising: Obtain sample road images; The predicted data of each parameter of the lane line in the sample road image are calculated based on the sample road image. The parameters include the horizontal coordinate of the starting point of the lane line, the vertical coordinate of the starting point of the lane line, the angle between the lane line and the bottom horizontal direction of the lane line in the sample road image, and the length of the lane line. Obtain the loss function corresponding to each of the parameters; wherein at least two of the parameters have different loss functions; and The lane detection model is trained based on the predicted data of each parameter, the actual data of each parameter, and the loss function corresponding to each parameter; The lane line angle corresponds to a first loss function, the horizontal coordinate of the lane line starting point corresponds to a second loss function, the vertical coordinate of the lane line starting point corresponds to a third loss function, and the lane line length corresponds to a fourth loss function. Training the lane line detection model based on the predicted data of each parameter, the actual data of each parameter, and the loss function corresponding to each parameter includes: The first loss value is calculated based on the predicted data of the lane line angle, the actual data of the lane line angle, and the first loss function; The second loss value is calculated based on the predicted data of the abscissa of the starting point of the lane line, the actual data of the abscissa of the starting point of the lane line, the first loss value, and the second loss function. The third loss value is calculated based on the predicted data of the ordinate of the starting point of the lane line, the actual data of the ordinate of the starting point of the lane line, the first loss value, and the third loss function. The fourth loss value is calculated based on the predicted lane length data, the actual lane length data, and the fourth loss function; and The lane detection model is trained based on the first loss value, the second loss value, the third loss value, and the fourth loss value. The value ranges of the first loss value, the second loss value, the third loss value, and the fourth loss value are [0, 1].

2. The method according to claim 1, characterized in that, The magnitude of the first loss value is inversely proportional to the magnitude of the second loss value; and / or the magnitude of the first loss value is inversely proportional to the magnitude of the third loss value.

3. The method according to claim 1, characterized in that, The methods for generating the second loss function include: Obtain the first loss function and the exponential function with the natural constant e as the base; Generate the negative exponent of the exponential function based on the first loss function; and The second loss function is generated based on the exponential function and the negative exponent; and / or The method for generating the third loss function includes: Obtain the first loss function and the exponential function with the natural constant e as the base; Generate the negative exponent of the exponential function based on the first loss function; and The third loss function is generated based on the exponential function and the negative exponent.

4. The method according to claim 1, characterized in that, The formula for the first loss function is: ； in, Let the first loss function be... This is the predicted data for the lane line angle. This refers to the actual data of the lane line angle. The index of the input sample road image.

5. The method according to claim 4, characterized in that, The formula for the second loss function is: ； in, For the second loss function, The predicted data is the ordinate of the starting point of the lane line. The actual data for the x-coordinate of the starting point of the lane line. The index of the input sample road image.

6. The method according to claim 4, characterized in that, The formula for the third loss function is: ； in, For the third loss function, The predicted data is the ordinate of the starting point of the lane line. The actual data of the ordinate of the starting point of the lane line. The index of the input sample road image.

7. The method according to claim 1, characterized in that, The formula for the fourth loss function is: ； in, For the fourth loss function, This is the predicted data for the lane line length. The actual data of the lane line length. The index of the input sample road image.

8. The method according to any one of claims 1 to 7, characterized in that, The step of calculating the predicted data for each parameter of the lane lines in the sample road image based on the sample road image includes: The sample road image is input into the feature extraction backbone network to obtain the backbone feature map output by the feature extraction backbone network; The backbone feature map is input into the feature pyramid network to obtain multiple first feature maps output by the feature pyramid network; wherein each first feature map has a different size. Multiple first feature maps are input into a feature refinement network to obtain second feature maps corresponding to each first feature map output by the feature refinement network; and Based on each of the second feature maps and the head attention mechanism, the predicted data of each parameter of the lane line in the sample road image are calculated.

9. The method according to any one of claims 1 to 7, characterized in that, Calculating the predicted lane line angle based on the sample road image includes: Multiple sampling points are uniformly sampled along the predicted lane lines; The included angle corresponding to each sampling point is calculated based on the predicted data of the abscissa of the starting point of the lane line, the number of sampling points, the height of the sample road image, and the arctangent function; and The predicted data of the lane line angle is obtained based on the included angle corresponding to each sampling point.

10. A lane line detection method, the method comprising: Acquire the image of the road to be detected; as well as Lane detection is performed based on the road image to be detected and the trained lane detection model; wherein the lane detection model is the model trained according to any one of claims 1 to 9.

11. A vehicle comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method of claim 10.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 9, or the steps of the method according to claim 10.

Images