Lane line detection method and model training method, device, equipment and storage medium

Abstract

The application relates to the technical field of intelligent driving, and discloses a lane line detection method, a model training method, a device, equipment and a storage medium, the model training method comprises the following steps: lane line labeling is performed on an original-size training sample image to obtain a first lane line coordinate label; the original-size training sample image is reduced to a preset size to obtain a preset-size training sample image; the preset-size training sample image is used for detection training of a lane line detection model, a detection result is output, and lane line detection coordinate values are determined according to the detection result; quantitative error values are obtained according to the lane line detection coordinate values and the detection result; and the lane line detection coordinate values, the quantitative error values and the first lane line coordinate label are used for parameter adjustment of the lane line detection model. The application solves the quantitative error problem existing in lane line detection based on key points at present, and improves the accuracy of lane line detection.

Description

Technical Field

[0001] This invention relates to the field of intelligent driving technology, specifically to a lane line detection method, a model training method, a device, an equipment, and a storage medium. Background Technology

[0002] In the field of autonomous driving, lane detection is a crucial component. It provides autonomous vehicles with critical visual references, helping them avoid deviating from their lanes, avoid collisions with other vehicles, and react promptly in emergencies. This is essential for reducing the incidence of road traffic accidents and protecting the lives of drivers and passengers. In other words, the accuracy of lane detection is directly related to the safety of autonomous driving. Inaccurate lane detection can cause vehicles to deviate from their intended road trajectory, increasing the risk of accidents. Improving the accuracy of lane detection can reduce this risk, ensuring that vehicles always stay within the correct lane when driving on the road.

[0003] Existing heatmap-based lane detection algorithms generally suffer from a coordinate quantization error. This is primarily because the network can only output a heatmap that is downsampled to a size several times the original image size during learning. Since the final coordinates are determined by searching for the maximum value in the heatmap, and these coordinates are represented by integers, quantization errors occur when reconstructing the coordinates to the original image size. Clearly, quantization errors reduce the accuracy of lane detection. Summary of the Invention

[0004] In view of this, the present invention provides a lane line detection method, a model training method, an apparatus, a device, and a storage medium to solve the problem of low lane line detection accuracy caused by quantization errors in existing lane line detection methods that use heatmaps to predict lane coordinate points.

[0005] In a first aspect, the present invention provides a method for training a lane detection model, the method comprising:

[0006] Lane line annotations are performed on the original-size training sample images to obtain the first lane line coordinate labels;

[0007] The original-size training sample image is reduced to a preset size to obtain a training sample image of the preset size;

[0008] The lane line detection model is trained using training sample images of a preset size, the detection results are output, and the lane line detection coordinates are determined based on the detection results.

[0009] The quantization error value is obtained based on the lane line detection coordinates and detection results;

[0010] The parameters of the lane detection model are adjusted using the lane detection coordinates, quantization error values, and the first lane coordinate label.

[0011] In one optional implementation, the lane detection model is parameter-adjusted using lane detection coordinate values, quantization error values, and a first lane coordinate label, including:

[0012] Obtain the coordinate labels of the second lane line corresponding to the training sample image of the preset size;

[0013] The first loss value is obtained based on the lane line detection coordinates and the second lane line coordinate labels;

[0014] Based on the coordinate labels of the first lane line and the second lane line, obtain the quantization error label;

[0015] The second loss value is obtained based on the quantization error value and the quantization error label;

[0016] The parameters of the lane detection model are adjusted based on the first and second loss values.

[0017] In one optional implementation, obtaining a quantization error value based on lane line detection coordinates and detection results includes:

[0018] The predicted values ​​of the first element corresponding to the lane line detection coordinates and multiple adjacent second elements are obtained from the detection results. The predicted values ​​are output by the lane line detection model and are used to indicate the probability that the element is a lane line.

[0019] The first weight of the first element and the second element is determined based on the predicted values ​​of the first element and the second element.

[0020] The quantization error value is determined based on the first weight of the first element and the second element, and the relative position between the first element and the second element.

[0021] In one alternative implementation, the selection radius of the second element is a preset value, which is determined based on the radius during Gaussian processing, which is performed when obtaining the coordinate labels of the second lane line.

[0022] In one optional implementation, the lane detection model is parameter-adjusted based on a first loss value and a second loss value, including:

[0023] Obtain the second weights corresponding to the first and second loss values, respectively;

[0024] The third loss value is obtained by weighting and summing the first and second loss values ​​according to the second weight.

[0025] The parameters of the lane detection model are adjusted based on the third loss value.

[0026] In one optional implementation, the lane detection model is parameter-adjusted using lane detection coordinate values, quantization error values, and a first lane coordinate label, including:

[0027] Calculate the compensated lane line detection coordinates based on the lane line detection coordinates and quantization error values;

[0028] The compensated lane line detection coordinate values ​​are restored to the original lane line detection coordinate values.

[0029] The fourth loss value is calculated based on the original lane line detection coordinates and the first lane line coordinate labels;

[0030] The parameters of the lane detection model are adjusted based on the fourth loss value.

[0031] Secondly, the present invention provides a lane line detection method, the method comprising:

[0032] Acquire the image of the lane line to be detected;

[0033] The lane line image to be detected is reduced to a preset size to obtain a lane line image of the preset size;

[0034] Input a lane line image of a preset size into a pre-trained lane line detection model, output the detection results, and determine the lane line detection coordinates based on the detection results;

[0035] The quantization error value is obtained based on the lane line detection coordinates and detection results;

[0036] The coordinates of the lane lines are determined based on the lane line detection coordinates and quantization error values.

[0037] In one alternative implementation, the lane detection model is a model trained using the training method of the lane detection model of the first aspect described above or any corresponding implementation thereof.

[0038] Thirdly, the present invention provides a training apparatus for a lane line detection model, the apparatus comprising:

[0039] The first annotation module is used to annotate lane lines on the original-size training sample image to obtain the first lane line coordinate labels;

[0040] The first image size adjustment module is used to reduce the original size training sample image to a preset size to obtain a training sample image of the preset size.

[0041] The training module is used to train the lane line detection model using training sample images of preset size, output the detection results, and determine the lane line detection coordinate values ​​based on the detection results.

[0042] The first quantization error acquisition module is used to acquire quantization error values ​​based on lane line detection coordinates and detection results.

[0043] The model parameter adjustment module is used to adjust the parameters of the lane line detection model using lane line detection coordinate values, quantization error values, and the first lane line coordinate label.

[0044] Fourthly, the present invention provides a lane line detection device, the device comprising:

[0045] The image acquisition module is used to acquire images of the lane lines to be detected.

[0046] The second image size adjustment module is used to reduce the lane line image to be detected to a preset size to obtain a lane line image of the preset size.

[0047] The prediction module is used to input lane line images of a preset size into a trained lane line detection model, output detection results, and determine lane line detection coordinates based on the detection results.

[0048] The second quantization error acquisition module is used to acquire quantization error values ​​based on lane line detection coordinates and detection results.

[0049] The lane line determination module is used to determine the coordinates of the lane line based on the lane line detection coordinates and quantization error values.

[0050] Fifthly, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to perform the training method of the lane detection model of the first aspect or any corresponding embodiment thereof, or the lane detection method of the second aspect or any corresponding embodiment thereof.

[0051] In a sixth aspect, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to execute the training method of the lane detection model of the first aspect or any corresponding embodiment thereof, or the lane detection method of the second aspect or any corresponding embodiment thereof.

[0052] The lane detection method, model training method, apparatus, device, and storage medium provided in this invention not only consider the accuracy of model detection but also the quantization error, solving the problem of coordinate quantization error in lane detection methods based on key points (heatmaps) and improving the accuracy of lane detection. Attached Figure Description

[0053] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0054] Figure 1 This is a schematic diagram illustrating the changes in element coordinate values ​​after the image is scaled down;

[0055] Figure 2 This is a flowchart illustrating the training method of the lane line detection model according to an embodiment of the present invention;

[0056] Figure 3 This is a flowchart illustrating a training method for another lane detection model according to an embodiment of the present invention.

[0057] Figure 4 This is a schematic diagram of lane line elements determined based on the output of the lane line detection model;

[0058] Figure 5 yes Figure 4 A schematic diagram of one of the lane line elements shown;

[0059] Figure 6 This is a schematic flowchart of a lane line detection method according to an embodiment of the present invention;

[0060] Figure 7 This is a schematic diagram of lane line elements determined based on model detection results according to an embodiment of the present invention;

[0061] Figure 8 This is a schematic diagram illustrating the process of restoring lane line detection coordinate values ​​according to an embodiment of the present invention;

[0062] Figure 9 This is a structural block diagram of a training device for a lane line detection model according to an embodiment of the present invention;

[0063] Figure 10 This is a structural block diagram of a lane line detection device according to an embodiment of the present invention;

[0064] Figure 11 This is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. Detailed Implementation

[0065] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0066] Regarding quantization error, a detailed explanation is provided below: Please refer to [link / reference needed]. Figure 1 If the original image size is 256*256, if Figure 1 There is a lane line in the y-th row, with coordinates (65, y). Assuming the final output heatmap size after network learning is 64*64, then the original coordinates (65, y) should be scaled to (16, y') during network learning. The calculation formula is:

[0067]

[0068] This indicates rounding down, where S represents the downsampling factor, (x i ,y i () represents the coordinates of the original graph. This represents the coordinates on the heatmap after downsampling. In the example above, S = 256 / 64 = 4, x i =65, Quantization error refers to the error between the coordinates of a heatmap restored to their original coordinates and the original coordinates. In the example above, the heatmap coordinates... Restoring to the original coordinates is Quantization error is

[0069] This invention provides a training method for a lane detection model. During the model training process, the adjustment of model parameters and the evaluation of the model are based not only on the accuracy of model detection but also on the accuracy of quantization error compensation, thereby improving the accuracy of lane detection. This invention achieves the effect of improving lane detection accuracy through quantization error compensation.

[0070] According to an embodiment of the present invention, a training method for a lane detection model is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of executable computer instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0071] This embodiment provides a method for training a lane detection model, which can be used in computer devices, such as terminals or servers. Figure 2 This is a flowchart of a training method for a lane detection model according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps:

[0072] Step S201: Label the lane lines on the original size training sample image to obtain the first lane line coordinate label.

[0073] Here, there are many training sample images, and each training sample image needs to be labeled with lane lines. Gaussian attenuation can be used to obtain the first lane line coordinate labels during lane line labeling. The radius of the Gaussian attenuation can be set as needed, for example, it can be 5.

[0074] Step S202: Reduce the original size training sample image to a preset size to obtain a training sample image of the preset size.

[0075] Step S203: Using training sample images of a preset size, the lane detection model is trained, and the detection results are output. The lane detection coordinates are then determined based on these results. Specifically, input information is obtained from the training sample images of a preset size. This input information can be from the image itself or extracted feature information. This input information is fed into the lane detection model to be trained, and the detection results are output. The lane detection coordinates predicted by the model are then determined based on these results. The detection results can be the predicted values ​​for each element, indicating the probability that each element is a lane line. This can also be called the confidence score or the response value of a heatmap, which is obtained based on these predicted values. Based on the predicted values ​​for each element, one element can be identified as the lane line element predicted by the model, and its corresponding coordinates are the lane detection coordinates. For example, the element with the largest predicted value in each row can be identified as the lane line element, or an element greater than a preset threshold can be identified as the lane line element, or one or more elements greater than a preset threshold and with the largest predicted value can be identified as lane line elements.

[0076] The granularity of lane line detection, i.e., the element, can be a pixel. In this case, the lane line detection coordinate value is the pixel coordinate value.

[0077] Step S204: Obtain the quantization error value based on the lane line detection coordinates and detection results.

[0078] Specifically, the quantization error value is calculated according to a preset calculation method, with the lane line detection coordinates serving as one of the calculation bases. There are various calculation methods, but they all need to be aligned with the actual quantization error and be as close as possible to the actual quantization error. The following implementation examples will illustrate this.

[0079] Step S205: Adjust the parameters of the lane line detection model using the lane line detection coordinate values, quantization error values, and the first lane line coordinate label.

[0080] There are several ways to adjust the parameters of the lane detection model using lane detection coordinate values, quantization error values, and the first lane coordinate label. The following examples will illustrate these methods.

[0081] The lane detection model training method provided in this embodiment uses lane coordinate labels obtained by annotating training sample images of the original size before scaling down. When adjusting the parameters of the lane detection model, the lane detection coordinate values, quantization error values, and the first lane coordinate labels are considered simultaneously. In other words, not only the accuracy of model detection is considered, but also the quantization error. This solves the problem of coordinate quantization error in lane detection methods based on key points (heatmaps) and improves the accuracy of lane detection.

[0082] This embodiment provides a method for training a lane detection model, which can be used in computer devices, such as terminals or servers. Figure 3 This is a flowchart of a training method for a lane detection model according to an embodiment of the present invention, such as... Figure 3 As shown, the process includes the following steps:

[0083] Step S301: Label the lane lines on the original size training sample image to obtain the first lane line coordinate label.

[0084] Step S302: Reduce the original size training sample image to a preset size to obtain a training sample image of the preset size.

[0085] Step S303: Use training sample images of preset size to train the lane line detection model, output the detection results, and determine the lane line detection coordinate values ​​based on the detection results;

[0086] Step S304: Obtain the quantization error value based on the lane line detection coordinates and detection results;

[0087] In one optional implementation, step S304, which involves obtaining a quantization error value based on the lane line detection coordinates and detection results, includes:

[0088] Step S3041: Obtain the predicted values ​​of the first element corresponding to the lane line detection coordinates and multiple adjacent second elements from the detection results. The predicted values ​​are output by the lane line detection model and are used to indicate the probability that an element is a lane line. The predicted values ​​can also be called confidence scores, heatmap response values, etc. For example, elements with predicted values ​​greater than a preset threshold can be identified as lane line elements, or for a row of elements in an image, the element with the largest predicted value can be identified as a lane line element.

[0089] Specifically, the selection radius of the second element is a preset value, which is determined based on the radius obtained during Gaussian processing. Gaussian processing is the process performed when obtaining the coordinate labels of the second lane line. This preset value can be equal to the radius of the Gaussian processing.

[0090] For example, please refer to Figure 4 In the diagram, the green and blue dots represent the coordinates of the position (element) corresponding to the highest predicted value in the lane detection model's output. In other words, the detection result of the lane detection model indicates that the positions of the blue and green dots are lane line elements. Taking one of the green dot positions as an example, that green dot position is the first element. Please refer to [link to relevant documentation]. Figure 5 Based on a selection radius of 5, extract the 10 elements that are adjacent to each other on the left and right (i.e., the second element).

[0091] Step S3042: Determine the first weight of the first element and the second element based on the predicted values ​​of the first element and the second element;

[0092] Specifically, the normalized exponential function value (Softmax function value) of the predicted values ​​of the first and second elements can be calculated as the weight value.

[0093] Step S3043: Determine the quantization error value based on the first weight of the first element and the second element, and the relative position between the first element and the second element.

[0094] Specifically, we can first obtain the indices starting from 0 by sorting the first and second elements according to their coordinate positions. Then, we subtract the selection radius of the second element from each indice, and sum the resulting differences with the corresponding first weights to obtain the quantization error value. The specific calculation formula can be:

[0095]

[0096]

[0097] in, Here, T represents the quantization error value, and T is a temperature coefficient used to control the output value of the Softmax function. By choosing an appropriate temperature coefficient, the calculation of the quantization error value can focus more on the second element near the first element; for example, it can be 10.j Let be the predicted values ​​of the first and second elements, j be the sequence number starting from 0 obtained by sorting the first and second elements according to their coordinate positions, and r be the selection radius of the second element mentioned above.

[0098] Of course, the calculation method for quantization error is not limited to the above method. There are other calculation methods as long as the calculated quantization error value is as close as possible to the true quantization error value. Not only can it be calculated using a fitting function, but it can also be obtained using machine learning.

[0099] Step S305: Adjust the parameters of the lane line detection model using the lane line detection coordinate values, quantization error values, and the first lane line coordinate label.

[0100] In one optional implementation, step S305, which involves adjusting the parameters of the lane line detection model using the lane line detection coordinate values, the quantization error value, and the first lane line coordinate label, includes:

[0101] Step S3051: Obtain the coordinate labels of the second lane line corresponding to the training sample image of the preset size;

[0102] In this embodiment of the invention, both the first lane line coordinate label corresponding to the original size training sample image and the second lane line coordinate label corresponding to the reduced preset size training sample image are obtained.

[0103] Step S3052: Obtain the first loss value based on the lane line detection coordinate value and the second lane line coordinate label; the calculation of this loss value can refer to relevant technologies, specifically L2 loss, L1 loss, and smoothed L1 loss.

[0104] Step S3053: Obtain the quantization error label based on the first lane line coordinate label and the second lane line coordinate label;

[0105] The scale of the quantization error label here can be consistent with the scale of the original-size training sample images, for example... Figure 1 In the corresponding example, the quantization error label is 1; the scale of the quantization error label can be consistent with the scale of the training sample images of a preset size, for example... Figure 1 In the corresponding example, the quantization error label is 0.25.

[0106] Step S3054: Obtain the second loss value based on the quantization error value and the quantization error label;

[0107] In this embodiment of the invention, according to Figure 1 In the corresponding example, the quantization error label d is calculated. Taking the L2 loss as the second loss value, the specific calculation formula is as follows:

[0108]

[0109] Among them, L quant The second loss value is S, which is the downsampling factor, i.e., the reduction factor from the original size training sample image to the preset size training sample image.

[0110] Step S3055: Adjust the parameters of the lane detection model based on the first loss value and the second loss value.

[0111] Step S3055, which involves adjusting the parameters of the lane detection model based on the first and second loss values, includes:

[0112] Step S30551: Obtain the second weights corresponding to the first loss value and the second loss value, respectively;

[0113] The second loss value should not be too small, for example, greater than or equal to 3.

[0114] Step S30552: The first loss value and the second loss value are weighted and summed according to the second weight to obtain the third loss value;

[0115] Step S30553: Adjust the parameters of the lane detection model based on the third loss value.

[0116] Of course, the model training method with dual loss values ​​is not limited to the weighted summation method mentioned above. There are other model training methods as well. Please refer to the relevant technologies for details.

[0117] In this embodiment of the invention, the lane detection model can learn the maximum predicted value in the lane detection heatmap, as well as the quantization error when the heatmap is restored to its original size.

[0118] In another optional implementation, step S305, which involves adjusting the parameters of the lane detection model using the lane detection coordinate values, quantization error values, and the first lane coordinate label, includes:

[0119] Step 1: Calculate the compensated lane line detection coordinates based on the lane line detection coordinates and quantization error values;

[0120] Step 2: Restore the compensated lane line detection coordinate values ​​to the original lane line detection coordinate values;

[0121] Step 3: Calculate the fourth loss value based on the original lane line detection coordinates and the first lane line coordinate label;

[0122] Step 4: Adjust the parameters of the lane detection model based on the fourth loss value.

[0123] This embodiment provides a lane line detection method, which can be used in computer devices such as terminals or servers. Figure 6 This is a flowchart of a lane line detection method according to an embodiment of the present invention, such as... Figure 6 As shown, the process includes the following steps:

[0124] Step S601: Obtain the image of the lane line to be detected;

[0125] Step S602: Reduce the lane line image to be detected to a preset size to obtain a lane line image of the preset size;

[0126] Step S603: Input the lane line image of the preset size into the trained lane line detection model, output the detection result, and determine the lane line detection coordinate value based on the detection result; wherein the lane line detection coordinate value is the coordinate value of the lane line element (specifically, a pixel) determined based on the detection result.

[0127] Step S604: Obtain the quantization error value based on the lane line detection coordinates and detection results;

[0128] Step S605: Determine the coordinates of the lane line based on the lane line detection coordinates and the quantization error value.

[0129] The lane detection method provided in this embodiment solves the problem of coordinate quantization error in the lane detection method based on key points (heatmaps) by quantization error compensation, thereby improving the accuracy of lane detection. Moreover, this lane detection method can achieve high-precision lane detection under different lighting, weather and road conditions, and has strong robustness.

[0130] The lane detection model used in this embodiment of the invention can be the lane detection model trained by the above-described lane detection model training method embodiment.

[0131] Specifically, for the calculation method of the quantization error value, please refer to the above-described example of the training method for the lane detection model. For instance, the lane detection coordinate values ​​determined based on the detection results are... Figure 7 The coordinates of element p0 in the model indicate that the lane detection model considers element p0 to be a lane line element (the first element). Please refer to [link to relevant documentation]. Figure 8 In the left-hand graph, select 10 elements around p0 with a radius of 5 as the second elements h0 to h10 (green dots), and then calculate the quantization error using the following formula:

[0132]

[0133]

[0134] in, Here, T represents the quantization error value, and T is a temperature coefficient used to control the output value of the Softmax function. By choosing an appropriate temperature coefficient, the calculation of the quantization error value can focus more on the second element near the first element; for example, it can be 10. j Let be the predicted values ​​of the first and second elements, j be the sequence number starting from 0 obtained by sorting the first and second elements according to their coordinate positions, and r be the selection radius of the second element mentioned above.

[0135] Step S505 above, namely, determining the coordinates of the lane line based on the lane line detection coordinates and the quantization error value, can be as follows:

[0136] The lane detection coordinates are calculated after error compensation based on the lane detection coordinates and the quantization error value; specifically, the lane detection coordinates and the quantization error value are summed. in, For lane line detection coordinate values;

[0137] The compensated lane line detection coordinate values ​​are restored to the original lane line detection coordinate values. Specifically, this can be done as follows: S represents the scaling factor when the lane line image to be detected is reduced to a preset size. Please refer to [link / reference]. Figure 8 In the example shown, the size of the lane line image to be detected is 256*256, and the size of the heatmap output by the lane line detection model is 64*64. Figure 8 As shown in the image on the left, where the red dots represent the pixels corresponding to the lane detection coordinates of each row, one row has lane detection coordinates of (16, y). ′ The quantization error value is calculated using the method described above. The final x-axis value in the original lane line detection coordinates is (Rounded). It can be seen that the original lane line detection coordinates obtained here compensate for the quantization error when the coordinates are restored to the original size, resulting in more accurate lane line horizontal coordinates.

[0138] This embodiment also provides a training device for a lane detection model, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0139] This embodiment provides a training device for a lane line detection model, such as... Figure 9 As shown, it includes:

[0140] The first annotation module 901 is used to annotate lane lines on the original size training sample image to obtain the first lane line coordinate label;

[0141] The first image size adjustment module 902 is used to reduce the original size training sample image to a preset size to obtain a training sample image of the preset size.

[0142] The training module 903 is used to train the lane line detection model using training sample images of a preset size, output the detection results, and determine the lane line detection coordinate values ​​based on the detection results.

[0143] The first quantization error acquisition module 904 is used to acquire quantization error values ​​based on lane line detection coordinate values ​​and detection results.

[0144] The model parameter adjustment module 905 is used to adjust the parameters of the lane line detection model using the lane line detection coordinate values, quantization error values, and the first lane line coordinate label.

[0145] In some optional implementations, the model parameter adjustment module 905 includes:

[0146] The first label acquisition unit is used to acquire the second lane line coordinate label corresponding to the training sample image of the preset size;

[0147] The first loss value calculation unit is used to obtain the first loss value based on the lane line detection coordinate value and the second lane line coordinate label.

[0148] The second label acquisition unit is used to acquire the quantization error label based on the first lane line coordinate label and the second lane line coordinate label;

[0149] The second loss value calculation unit is used to obtain the second loss value based on the quantization error value and the quantization error label;

[0150] The adjustment unit is used to adjust the parameters of the lane detection model based on the first loss value and the second loss value.

[0151] In some optional implementations, the first quantization error acquisition module 904 includes:

[0152] The prediction value acquisition unit is used to obtain the prediction values ​​of the first element corresponding to the lane line detection coordinate value and multiple adjacent second elements from the detection results. The prediction value is the value output by the lane line detection model, which indicates the probability that the element is a lane line.

[0153] The first weight acquisition unit is used to determine the first weight of the first element and the second element based on the predicted values ​​of the first element and the second element.

[0154] The quantization error value determination unit is used to determine the quantization error value based on the first weight of the first element and the second element, and the relative position between the first element and the second element.

[0155] In some alternative implementations, the selection radius of the second element is a preset value, which is determined based on the radius during Gaussian processing, which is performed when obtaining the coordinate labels of the second lane line.

[0156] In some optional implementations, the adjustment unit includes:

[0157] The second weight acquisition sub-unit is used to acquire the second weights corresponding to the first loss value and the second loss value, respectively.

[0158] The third loss value calculation subunit is used to weight and sum the first loss value and the second loss value according to the second weight to obtain the third loss value;

[0159] The parameter adjustment subunit is used to adjust the parameters of the lane detection model based on the third loss value.

[0160] In some optional implementations, the model parameter adjustment module 905 includes:

[0161] The compensation unit is used to calculate the compensated lane line detection coordinates based on the lane line detection coordinates and the quantization error value.

[0162] The restoration unit is used to restore the compensated lane line detection coordinate values ​​to the original size lane line detection coordinate values.

[0163] The fourth loss value calculation unit is used to calculate the fourth loss value based on the original size lane line detection coordinate value and the first lane line coordinate label;

[0164] The parameter adjustment unit is used to adjust the parameters of the lane detection model based on the fourth loss value.

[0165] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0166] This embodiment also provides a training device for a lane detection model, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0167] This embodiment provides a lane line detection device, such as Figure 10 As shown, it includes:

[0168] Image acquisition module 1001 is used to acquire images of lane lines to be detected;

[0169] The second image size adjustment module 1002 is used to reduce the lane line image to be detected to a preset size to obtain a lane line image of the preset size.

[0170] The prediction module 1003 is used to input a lane line image of a preset size into a trained lane line detection model, output the detection result, and determine the lane line detection coordinate value based on the detection result.

[0171] The second quantization error acquisition module 1004 is used to acquire quantization error values ​​based on lane line detection coordinate values ​​and detection results.

[0172] The lane line determination module 1005 is used to determine the coordinates of the lane line based on the lane line detection coordinates and quantization error values.

[0173] Further functional descriptions of the above modules are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0174] In this embodiment, the training device and the lane detection device of the lane detection model are presented in the form of functional units. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0175] This invention also provides a computer device having the above-described features. Figure 9 The training device for the lane detection model shown, or the above Figure 10 The lane line detection device shown.

[0176] Please see Figure 11 , Figure 11 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of the present invention, such as... Figure 11As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 11 Take a processor 10 as an example.

[0177] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0178] The memory 20 stores instructions executable by at least one processor 10 to cause at least one processor 10 to perform the method shown in the above embodiments.

[0179] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0180] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0181] The computer device also includes an input device 30 and an output device 40. The processor 10, memory 20, input device 30, and output device 40 can be connected via a bus or other means. Figure 11 Taking the example of a connection between China and Israel via a bus.

[0182] Input device 30 can receive input numerical or character information, and generate key signal inputs related to user settings and function control of the computer device, such as a touchscreen, keypad, mouse, trackpad, touchpad, joystick, one or more mouse buttons, trackball, joystick, etc. Output device 40 may include display devices, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibration motors). The aforementioned display devices include, but are not limited to, liquid crystal displays, light-emitting diodes, displays, and plasma displays. In some alternative embodiments, the display device may be a touchscreen.

[0183] The computer device also includes a communication interface for communicating with other devices or communication networks.

[0184] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0185] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for training a lane line detection model, characterized in that, The method includes: Lane line annotations are performed on the original-size training sample images to obtain the first lane line coordinate labels; The original-size training sample image is reduced to a preset size to obtain a training sample image of the preset size; The lane line detection model is trained using the preset-size training sample images, the detection results are output, and the lane line detection coordinate values ​​are determined based on the detection results. A quantization error value is obtained based on the lane line detection coordinates and the detection result; this includes: obtaining the predicted values ​​of a first element corresponding to the lane line detection coordinates and a plurality of adjacent second elements from the detection result, wherein the predicted values ​​are values ​​output by the lane line detection model that indicate the probability that an element is a lane line; determining a first weight for the first element and the second element based on the predicted values ​​of the first element and the second element; and determining the quantization error value based on the first weights of the first element and the second element, and the relative position between the first element and the second element. The lane line detection model is adjusted using the lane line detection coordinates, the quantization error value, and the first lane line coordinate label.

2. The method of claim 1, wherein, The step of adjusting the parameters of the lane detection model using the lane detection coordinate values, the quantization error values, and the first lane coordinate labels includes: Obtain the coordinate labels of the second lane line corresponding to the training sample image of the preset size; The first loss value is obtained based on the lane line detection coordinate value and the second lane line coordinate label; Based on the first lane line coordinate label and the second lane line coordinate label, obtain the quantization error label; Based on the quantization error value and the quantization error label, obtain the second loss value; The parameters of the lane detection model are adjusted based on the first loss value and the second loss value.

3. The method of claim 2, wherein, The selection radius of the second element is a preset value, which is determined based on the radius during Gaussian processing, which is the processing performed when obtaining the coordinate labels of the second lane line.

4. The method of claim 2, wherein, The step of adjusting the parameters of the lane detection model based on the first loss value and the second loss value includes: Obtain the second weights corresponding to the first loss value and the second loss value, respectively; The first loss value and the second loss value are weighted and summed according to the second weight to obtain the third loss value; The parameters of the lane detection model are adjusted based on the third loss value.

5. The method of claim 1, wherein, The step of adjusting the parameters of the lane detection model using the lane detection coordinate values, the quantization error values, and the first lane coordinate labels includes: Calculate the compensated lane line detection coordinates based on the lane line detection coordinates and the quantization error value; The compensated lane line detection coordinate values ​​are restored to the original size lane line detection coordinate values. The fourth loss value is calculated based on the original lane line detection coordinate values ​​and the first lane line coordinate label; The parameters of the lane detection model are adjusted based on the fourth loss value.

6. A lane line detection method, characterized in that, The method includes: Acquire the image of the lane line to be detected; The lane line image to be detected is reduced to a preset size to obtain a lane line image of the preset size; The preset-size lane line image is input into the trained lane line detection model, the detection result is output, and the lane line detection coordinate value is determined based on the detection result. A quantization error value is obtained based on the lane line detection coordinates and the detection result; this includes: obtaining the predicted values ​​of a first element corresponding to the lane line detection coordinates and a plurality of adjacent second elements from the detection result, wherein the predicted values ​​are values ​​output by the lane line detection model that indicate the probability that an element is a lane line; determining a first weight for the first element and the second element based on the predicted values ​​of the first element and the second element; and determining the quantization error value based on the first weights of the first element and the second element, and the relative position between the first element and the second element. The coordinates of the lane line are determined based on the lane line detection coordinates and the quantization error value.

7. The method of claim 6, wherein, The lane detection model is a model trained using the training method for lane detection models according to any one of claims 1-5.

8. A training device for a lane line detection model, characterized in that, The device includes: The first annotation module is used to annotate lane lines on the original-size training sample image to obtain the first lane line coordinate labels; The first image size adjustment module is used to reduce the original size training sample image to a preset size to obtain a preset size training sample image; The training module is used to train the lane line detection model using the preset-size training sample images, output the detection results, and determine the lane line detection coordinate values ​​based on the detection results. The first quantization error acquisition module is used to acquire a quantization error value based on the lane line detection coordinate value and the detection result; The model parameter adjustment module is used to adjust the parameters of the lane line detection model using the lane line detection coordinate values, the quantization error values, and the first lane line coordinate labels. The first quantization error acquisition module includes: a prediction value acquisition unit, configured to acquire prediction values ​​of a first element corresponding to the lane line detection coordinate value and a plurality of adjacent second elements from the detection result, wherein the prediction value is a value output by the lane line detection model to indicate the probability that an element is a lane line; a first weight acquisition unit, configured to determine a first weight of the first element and the second element based on the prediction values ​​of the first element and the second element; and a quantization error value determination unit, configured to determine the quantization error value based on the first weight of the first element and the second element and the relative position between the first element and the second element.

9. A lane line detection device characterized by comprising: The device includes: The image acquisition module is used to acquire images of the lane lines to be detected. The second image size adjustment module is used to reduce the lane line image to be detected to a preset size to obtain a lane line image of the preset size. The prediction module is used to input the lane line image of the preset size into the trained lane line detection model, output the detection result, and determine the lane line detection coordinate value based on the detection result. The second quantization error acquisition module is used to acquire a quantization error value based on the lane line detection coordinate value and the detection result; specifically, it is used to acquire the predicted values ​​of the first element corresponding to the lane line detection coordinate value and a plurality of adjacent second elements from the detection result, wherein the predicted values ​​are values ​​output by the lane line detection model to indicate the probability that an element is a lane line; determine the first weight of the first element and the second element based on the predicted values ​​of the first element and the second element; and determine the quantization error value based on the first weight of the first element and the second element and the relative position between the first element and the second element. The lane line determination module is used to determine the coordinates of the lane line based on the lane line detection coordinates and the quantization error value.

10. A computer device, comprising: include: The system includes a memory and a processor, which are interconnected. The memory stores computer instructions, and the processor executes the computer instructions to perform the training method of the lane detection model according to any one of claims 1 to 5 or the lane detection method according to claim 6 or 7.

11. A computer readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a computer to perform the training method of the lane detection model according to any one of claims 1 to 5 or the lane detection method according to claim 6 or 7.

Images