Lane line detection method, electronic device, and storage medium

Abstract

The application discloses a lane line detection method, an electronic device and a storage medium. The lane line detection method comprises the following steps: acquiring target point cloud information of a lane line in a current frame image; determining an initial fitting lane line according to the target point cloud information of the lane line; and adjusting the initial fitting lane line according to attribute information of the initial fitting lane line until a target lane line meeting a requirement is obtained. The above scheme can improve the accuracy of the detected target lane line.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automobiles, in particular to a lane line detection method, an electronic device and a storage medium. BACKGROUND

[0002] In intelligent driving, the lane line on the road can be detected through the vehicle camera and sensor to realize intelligent simulation of the actual road lane line. In the application process, the image obtained is usually input into a perception model to obtain the simulated lane line output by the perception model. However, the perception model may cause the number of lane line points in the directly output simulated lane line to be insufficient due to reasons such as occlusion, light, and road marking wear, thereby causing the output simulated lane line to be inaccurate.

[0003] Therefore, there is an urgent need for a lane line detection method. SUMMARY

[0004] The present application at least provides a lane line detection method, an electronic device and a storage medium.

[0005] The present application provides a lane line detection method, comprising: obtaining target point cloud information of a lane line in a current frame image; determining an initial fitted lane line according to the target point cloud information of the lane line; and adjusting the initial fitted lane line according to attribute information of the initial fitted lane line until a target lane line meeting a requirement is obtained.

[0006] The present application provides a lane line detection device, comprising: an obtaining module, a determining module and an adjusting module; the obtaining module is configured to obtain target point cloud information of a lane line in a current frame image; the determining module is configured to determine an initial fitted lane line according to the target point cloud information of the lane line; and the adjusting module is configured to adjust the initial fitted lane line according to attribute information of the initial fitted lane line until a target lane line meeting a requirement is obtained.

[0007] The present application provides an electronic device, comprising a memory and a processor, wherein the processor is configured to execute program instructions stored in the memory to implement the lane line detection method described above.

[0008] The present application provides a computer-readable storage medium having program instructions stored thereon, wherein the program instructions are executed by a processor to implement the lane line detection method described above.

[0009] Compared with directly using the simulated lane line output by the perception model, the present application determines an initial fitted lane line according to the target point cloud information of the lane line in the current frame image obtained, adjusts the initial fitted lane line according to the attribute information of the initial fitted lane line, and obtains a target lane line meeting a requirement, thereby improving the accuracy of the target lane line detected.

[0010] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and are not restrictive of the application. BRIEF DESCRIPTION OF DRAWINGS

[0011] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and serve to explain the technical solutions of the application together with the specification.

[0012] Figure 1 is a flowchart of an exemplary embodiment of the lane line detection method of the application;

[0013] Figure 2 is Figure 1 is a subflowchart of step S11 in

[0014] Figure 3 is a framework diagram of an exemplary embodiment of the lane line detection method of the application;

[0015] Figure 4a is a first diagram of the preset lane line in an exemplary embodiment of the lane line detection method of the application;

[0016] Figure 4b is a second diagram of the preset lane line in an exemplary embodiment of the lane line detection method of the application;

[0017] Figure 4c is a third diagram of the preset lane line in an exemplary embodiment of the lane line detection method of the application;

[0018] Figure 4d is a fourth diagram of the preset lane line in an exemplary embodiment of the lane line detection method of the application;

[0019] Figure 5 is Figure 1 is a subflowchart of step S13 in

[0020] Figure 6 is Figure 1 is another subflowchart of step S13 in

[0021] Figure 7 is a structural diagram of an embodiment of the lane line detection device of the application;

[0022] Figure 8 is a structural diagram of an embodiment of the electronic device of the application;

[0023] Figure 9 is a structural diagram of an embodiment of the computer-readable storage medium of the application. DETAILED DESCRIPTION

[0024] The scheme of the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0025] In the following description, for the purpose of explanation and not limitation, specific details are set forth, such as particular system configurations, interfaces, techniques, in order to provide a thorough understanding of the present application.

[0026] The term "and / or" herein is merely an associated relationship between the associated objects, which means that there can be three relationships, for example, A and / or B can represent the following three cases: A exists alone, A and B exist together, and B exists alone. In addition, the character " / " herein generally represents that the front and rear associated objects are in an "or" relationship. In addition, "multiple" herein means two or more than two. In addition, the term "at least one" herein means any one of multiple or any combination of at least two of multiple, for example, including at least one of A, B and C can mean including any one or more elements selected from the set consisting of A, B and C.

[0027] The present application provides some lane line detection methods and lane line detection devices. The application scenarios of the lane line detection methods include but are not limited to lane line detection. The execution subject of the lane line detection method can be a lane line detection device or a vehicle, for example, the lane line detection device can be arranged in a terminal device or a server or other processing device or a vehicle, wherein the user equipment (User Equipment, UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (Personal Digital Assistant, PDA), handheld device, computing device, vehicle-mounted device, etc. In some possible implementation manners, the lane line detection method can be realized by a processor calling computer readable instructions stored in a memory.

[0028] Please refer to Figure 1 , Figure 1 is a flowchart of an exemplary embodiment of the lane line detection method of the present application. Specifically, the lane line detection method can include the following steps:

[0029] Step S11: Obtain target point cloud information of the lane line in the current frame image.

[0030] The current frame image is used to represent an image collected by an image collection device on a lane. The image collection device can be arranged on a vehicle that needs to detect a lane. For example, the image collection device is a camera on the vehicle, such as a front-view camera. In some application scenarios, after an image set corresponding to a plurality of frames of images is obtained, the plurality of frames of images can be images collected on the lane at different time points within a preset time period, and the current frame image can be an image being processed at present. In another application scenario, the current frame image is used to represent an image collected in real time on the lane, and the collection time point of the current frame image can be the current time point. In this application, it is taken as an example that the current frame image is an image collected by the image collection device on the vehicle on the lane at the current time point. The lane is a lane to be detected in the current frame image. The lane represents a path capable of separating a driving channel in a traffic scene. The driving channel can include but is not limited to a motor vehicle channel, a non-motor vehicle channel, etc. In this application, it is taken as an example that the driving channel is a path that can be driven in during the driving of the vehicle. The number of lanes can be one or more. In the case where the number of lanes is more than one, the same detection processing is performed on each lane. In another application scenario, the lane can be a single lane or a double lane. For example, the lane includes but is not limited to a current lane in which the vehicle is driving, a driving lane having the same direction as the current lane, and a driving lane having a different direction from the current lane. In this application, it is taken as an example that the lane is the current lane in which the vehicle is driving. The target point cloud information represents position information of a plurality of points on the lane in the current frame image after a preset prediction. The plurality of points can be at least two points. The preset prediction can be that the current frame image is input into a perception model to obtain the target point cloud information. The perception model can include a preset feature extraction module. The preset feature extraction module is used to perform the preset prediction on the current frame image to obtain the target point cloud information. The preset feature extraction module can be provided with a preset feature extraction network. Specifically, the preset feature extraction network can be a convolutional neural network, a recurrent neural network, a long short-term memory network, a gated recurrent unit, a BiGRU neural network, an attention mechanism, etc. For example, the preset feature extraction network in the perception model can be a long short-term memory (LSTM) in a recurrent neural network. The target point cloud information includes the number of points on the lane in the current frame image and initial coordinate information of each point.

[0031] In some application scenarios, the above step S11 can be that the current frame image is directly input into the above preset feature extraction module for feature extraction to obtain the target point cloud information corresponding to the current frame image.

[0032] Step S12: determining an initial fitting lane according to the target point cloud information of the lane.

[0033] The initial fitting lane line is a fitting lane line fitted based on target point cloud information. Specifically, the initial fitting lane line is a fitting lane line fitted based on initial coordinate information of each point in the target point cloud information. The several points are at least one point on the to-be-detected lane line. The initial coordinate information of each point in the target point cloud information includes coordinate information of each point on at least one number axis in a coordinate system in which a current frame image is located. Two number axes in the coordinate system in which the current frame image is located include a first number axis and a second number axis passing through the origin of the coordinate system. The present application takes the initial coordinate information of each point in the target point cloud information as an example, which includes coordinate information of each point on the first number axis and the second number axis, respectively. In some application scenarios, the first number axis and the second number axis can be the X axis and the Y axis in the coordinate system in which the current frame image is located, respectively. In other application scenarios, the first number axis and the second number axis can be the Y axis and the X axis in the coordinate system in which the current frame image is located, respectively. It can be understood that the present application does not limit the specific number axes of the first number axis and the second number axis in the coordinate system in which the current frame image is located, and the present application takes the first number axis as the X axis and the second number axis as the Y axis as an example. It can be understood that for any one point, the initial coordinate information in the target point cloud information can be described as: may represent the coordinate information of the point on the first number axis, may represent the coordinate information of the point on the second number axis.

[0034] In some application scenarios, the above step S12 can be to obtain a pre-established parameter model. The coordinate information of at least part of the points in the target point cloud information on any one number axis is input into the parameter model to obtain the initial fitting lane line. The parameter model is a polynomial parameter model used for modeling the lane line. The parameter model includes preset lane line coefficients, coordinate information of the first target number axis in the target point cloud information, and coordinate information of the second target number axis fitted. The preset lane line coefficients are default values or unknown values. The preset lane line coefficients include a first preset coefficient, a second preset coefficient, a third preset coefficient, and a fourth preset coefficient. The initial fitting lane line includes a fitted left lane line and / or a fitted right lane line. The first preset coefficient represents a distance between a center of a rear axle of a vehicle and the fitted left lane line and / or the fitted right lane line in the initial fitting lane line. The second preset coefficient represents an inverse tangent angle of an orientation angle of the fitted left lane line and / or the fitted right lane line, i.e., a slope of the fitted left lane line and / or the fitted right lane line. The third preset coefficient represents a curvature of the fitted left lane line and / or the fitted right lane line. The fourth preset coefficient represents a rate of change of the curvature of the fitted left lane line and / or the fitted right lane line. The first target number axis can be any one number axis in the coordinate system in which the current frame image is located. The second target number axis can be any one number axis other than the first target number axis. Specifically, the establishment process of the parameter model can refer to the following formula (1):

[0035] Formula (1);

[0036] in, , , and These can be used to represent the first preset coefficient, the second preset coefficient, the third preset coefficient, and the fourth preset coefficient in the aforementioned preset lane line coefficients, respectively. Used to represent the first in the target point cloud information The coordinate information of each point on the first target number axis. Used to indicate the first The coordinate information on the second target number axis obtained by fitting points The value of is The fitted function value of the point corresponding to the fitted lane line in the initial fitting. This application uses the first target axis as the X-axis and the second target axis as the Y-axis as an example. Wherein, and Between , Used to indicate transpose. For example, Used to indicate the first A column vector containing the coordinates of points on the X-axis. It satisfies the above-mentioned preset lane line coefficient. , Used to indicate transpose. For example, This is used to represent the column vectors corresponding to the first preset coefficient, the second preset coefficient, the third preset coefficient, and the fourth preset coefficient.

[0037] In some application scenarios, step S12 can involve inputting the coordinate information of all points in the target point cloud information on any number axis into the parameter model to obtain the initial fitted lane line. In other application scenarios, step S12 can involve filtering all points in the target point cloud information according to a preset filtering rule to obtain a preset number of points to be fitted. The preset filtering rule can be to perform equidistant sampling or random sampling on any number axis for the line segments / curves corresponding to all points in the target point cloud information to obtain a preset number of points to be fitted. The coordinate information of each point to be fitted in the target point cloud information on any number axis is then input into the parameter model to obtain the initial fitted lane line.

[0038] Step S13: Adjust the initial fitted lane line according to the attribute information of the initial fitted lane line until the target lane line that meets the requirements is obtained.

[0039] The attribute information of the initial fitted lane line represents the associated information of the fitted lane line determined based on the target point cloud information. The attribute information includes at least one of coordinate information associated with the initial fitted lane line, a type, initial lane line coefficients, and an evaluation result. Specifically, the coordinate information in the attribute information represents the coordinate information of any one point in the target point cloud information on the first target axis and the coordinate information of the second target axis fitted by the point. For example, the coordinate information of the second target axis fitted by the point is the fitted function value of the fitted function corresponding to the point in the initial fitted lane line. The type associated with the initial fitted lane line in the attribute information can be a double-lane line type or a single-lane line type. The initial lane line coefficients associated with the initial fitted lane line in the attribute information include at least one of the first, second, third, and fourth preset coefficients. For example, the initial lane line coefficients include left lane line coefficients and / or right lane line coefficients. The evaluation result associated with the initial fitted lane line in the attribute information represents the evaluation accuracy of the attribute information. The higher the evaluation accuracy of the attribute information, the higher the credibility of the attribute information. The lower the evaluation accuracy of the attribute information, the lower the credibility of the attribute information. For example, when the type associated with the initial fitted lane line in the attribute information is a double-lane line type, the initial fitted lane line includes a fitted left lane line and a fitted right lane line. The initial lane line coefficients include left lane line coefficients and right lane line coefficients. The evaluation result includes a first evaluation result of the fitted left lane line and a second evaluation result of the fitted right lane line. The target lane line is fitted by the adjusted initial lane line coefficients and the coordinate information of each point in the target point cloud on the first target axis.

[0040] The adjustment process can be adjusting the initial lane line coefficients of the initial fitted lane line, and using the adjusted initial lane line coefficients as the lane line coefficients of the target lane line to make the target lane line meet the requirements. The requirement can be that the fitted loss value determined based on the lane line coefficients of the target lane line meets the preset constraint condition. The target lane line can be the fitted lane line corresponding to the current frame image finally output by the lane line detection process.

[0041] In some application scenarios, the step S13 can be based on at least one of the coordinate information associated with the initial fitted lane line, the type, the initial lane line coefficient, and the evaluation result to determine a fitting loss value. In response to the fitting loss value satisfying a preset constraint condition, the initial fitted lane line is subjected to a convergence process until a target lane line meeting the requirement is obtained. For example, in the case where the fitting loss value is minimum, the initial lane line coefficient is solved, and the solved value is taken as the target lane line coefficient respectively. The target lane line coefficient is the adjusted initial lane line coefficient. The target lane line coefficient is taken as the lane line coefficient of the target lane line, and the fitting loss value corresponding to the obtained target lane line meets the preset constraint condition. Specifically, the target lane line can be obtained by replacing the values of the first preset coefficient, the second preset coefficient, the third preset coefficient, and the fourth preset coefficient in the initial lane line coefficient in the pre-established parameter model with the respective coefficients in the target lane line coefficient.

[0042] Compared with directly using the simulated lane line output by the perception model, the scheme can determine the initial fitted lane line according to the target point cloud information of the lane line in the current frame image, adjust the initial fitted lane line according to the attribute information of the initial fitted lane line, and obtain the target lane line meeting the requirement, thereby improving the accuracy of the detected target lane line.

[0043] Please refer to Figure 2 , Figure 2 is Figure 1 a sub-flowchart of the step S11 in

[0044] In some embodiments, the step S11 can include the following steps:

[0045] Step S21: Obtain initial point cloud information corresponding to a current frame image and historical point cloud information corresponding to a historical frame image.

[0046] The initial point cloud information is extracted from the current frame image. The initial point cloud information only contains feature information of lane lines in the current frame image. The historical frame image is an image of lane lines collected at a historical time. The historical time is a collection time before the current time. For example, the historical time can be a previous time before the current time, or a time two hours before the current time. In this application, the historical time is taken as an example of a previous time before the current time. In some application scenarios, the historical point cloud information can be historical initial point cloud information extracted from the historical frame image. The historical initial point cloud information only contains feature information of lane lines in the historical frame image. In other application scenarios, the historical point cloud information can be point cloud information obtained by fusing point cloud information corresponding to an image before the historical frame image and the historical initial point cloud information. The extraction method of the initial point cloud information obtained by extracting the current frame image and the extraction method of the historical initial point cloud information obtained by extracting the historical frame image can be the same extraction method.

[0047] Before the above step S21, the step of extracting the initial point cloud information from the current frame image includes: inputting the current frame image into the above-mentioned preset feature extraction module to obtain lane line information output by the preset feature extraction module. The lane line information output by the preset feature extraction module is discretized to generate initial point cloud information in a coordinate system in which the current frame image is located.

[0048] Step S22: Project the historical point cloud information into a coordinate system in which the current frame image is located to obtain new historical point cloud information.

[0049] The new historical point cloud information represents point cloud information obtained by coordinate system conversion of the historical point cloud information. The new historical point cloud information and the current frame image are in the same coordinate system, and the new historical point cloud information and the historical point cloud information are in different coordinate systems. The projection method can be coordinate system conversion of the historical point cloud information according to the relative motion relationship between the historical frame image and the current frame image.

[0050] It can be considered that, compared with the initial point cloud information, the new historical point cloud information obtained by projecting the historical point cloud information into the coordinate system in which the current frame image is located has less noise, so as to improve the accuracy of the target point cloud information obtained by subsequent fusion.

[0051] Step S23: Fuse the initial point cloud information and the new historical point cloud information to obtain target point cloud information.

[0052] In some application scenarios, the fusion manner in step S23 can be inputting the initial point cloud information and the new historical point cloud information into a preset fusion module to obtain data output by the preset fusion module, and directly taking the data output by the preset fusion module as the target point cloud information. The preset fusion module can be a preset decoder, a convolutional neural network, etc. The preset decoder includes at least one preset decoding layer, and each preset decoding layer has the same structure but can have different parameters. At least one attention mechanism can be arranged in each preset decoding layer. In other application scenarios, the fusion manner in step S23 can be inputting the initial point cloud information and the new historical point cloud information into a preset fusion module to obtain data output by the preset fusion module. The data output by the preset fusion module is subjected to a removal of data beyond a sliding window to obtain processed data, and the processed data is taken as the target point cloud information. The sliding window data is point cloud information in the preset range from the vehicle. The value of the preset range is determined according to the accuracy of the lane line detection.

[0053] It can be considered that the target point cloud information of the current frame image determined based on the initial point cloud information and the historical point cloud information can improve the smoothness of the target point cloud information in the time dimension, so that the target lane line determined based on the target point cloud information has high consistency and smoothness in the time dimension, and thus the overfitting phenomenon is inhibited.

[0054] In some embodiments, before step S13, a step of determining attribute information of the initial fitted lane is performed. In some application scenarios, the step of determining attribute information of the initial fitted lane includes: determining a type and / or an evaluation result associated with the initial fitted lane based on the target point cloud information. Specifically, the target point cloud information is analyzed to obtain analysis results of a plurality of points in the target point cloud information, and the analysis results of the plurality of points in the target point cloud information are taken as the type and / or the evaluation result in the attribute information of the initial fitted lane. The analysis results of the plurality of points include prior information of lane lines corresponding to the plurality of points and accuracy of the prior information. The prior information includes but is not limited to a type of lane lines corresponding to the plurality of points (such as a double-lane line or a single-lane line), whether the lane lines corresponding to the plurality of points are parallel, a farthest distance at which lane widths of the double-lane line corresponding to the plurality of points remain equal, and whether the lane lines corresponding to the plurality of points are one of a plurality of preset lane lines. The plurality of preset lane lines include but are not limited to a completely parallel double-lane line, a partially parallel double-lane line, a completely non-parallel double-lane line, and a single-lane line. Any one of the single-lane line or the double-lane line can be a straight line, a partial straight line, or a curve. For example, in a case where each lane line in the double-lane line is a curve and the type of the preset lane line is a completely parallel double-lane line, the extension lines of each lane line in the double-lane line do not intersect. The accuracy of the prior information ensures the credibility of the prior information. The accuracy of the prior information is taken as the evaluation result in the attribute information of the initial fitted lane.

[0055] In other application scenarios, the step of determining attribute information of the initial fitted lane includes: determining coordinate information associated with the initial fitted lane based on the target point cloud information and the parametric model. The coordinate information associated with the initial fitted lane includes first coordinate information and second coordinate information of each point. Specifically, the following steps are performed for each point: coordinate information of the point in the target point cloud information on a target axis is directly taken as the first coordinate information of the point in the attribute information. For example, the target axis can be any one of the axes in the coordinate system in which the current frame image is located, specifically the X axis in the coordinate system in which the current frame image is located. The coordinate information of the point in the target point cloud information on the target axis and the first coordinate information of the point in the attribute information can both be represented as Then, the fitting function value of the fitting function corresponding to the point in the initial fitting lane line is taken as the second coordinate information of the point in the attribute information. Specifically, the first coordinate information of the point in the attribute information is substituted into the parametric model to obtain the fitting function corresponding to the point in the initial fitting lane line. The number axis of the fitting function corresponding to the point in the initial fitting lane line and the target number axis are different number axes in the coordinate system of the current frame image. The fitting function corresponding to the point in the initial fitting lane line and the value of the fitting function are taken as the second coordinate information of the point in the attribute information. For example, the number axis of the fitting function corresponding to the point in the initial fitting lane line is the Y axis, and the value of the fitting function corresponding to the point in the initial fitting lane line can be expressed as .

[0056] In some other application scenarios, the step of determining the attribute information of the initial fitting lane line comprises: taking the preset lane line coefficients in the fitting function corresponding to the initial fitting lane line as the initial lane line coefficients associated with the initial fitting lane line in the attribute information. The initial lane line coefficients comprise left lane line coefficients and / or right lane line coefficients. Specifically, in the case where the initial fitting lane line comprises a fitted left lane line and / or a fitted right lane line, the preset lane line coefficients in the fitting function corresponding to the fitted left lane line and / or the preset lane line coefficients in the fitting function corresponding to the fitted right lane line are taken as the left lane line coefficients and / or the right lane line coefficients in the attribute information.

[0057] It can be considered that the target point cloud information obtained by fusing the initial point cloud information and the historical point cloud information is point cloud information with higher smoothness.

[0058] In some embodiments, the attribute information comprises an evaluation result of the attribute information, and the step S13 can comprise: a first step of determining a loss determination strategy matched with the evaluation result based on the evaluation result of the attribute information; a second step of determining a fitting loss value corresponding to the loss determination strategy based on the attribute information and the loss determination strategy; and a third step of performing convergence processing on the initial fitting lane line according to the fitting loss value until a target lane line meeting the requirement is obtained.

[0059] The fitting loss value represents a matching degree between a curve of the corresponding fitting function of the initial fitting lane line and the expected lane point cloud data. In some application scenarios, the expected lane point cloud data can be preset point cloud data or point cloud data in the target point cloud information. The loss determination strategy is used to indicate that at least part of the attribute information is used to construct the fitting loss value of the initial fitting lane line. Different evaluation results of the attribute information correspond to different loss determination strategies. The loss determination strategy includes an initial determination strategy and an advanced determination strategy. In some application scenarios, the advanced determination strategy includes the initial determination strategy and other determination strategies. In other application scenarios, the advanced determination strategy only includes other determination strategies except the initial determination strategy. Different attribute information used in different loss determination strategies is different. The evaluation accuracy in the evaluation result of the attribute information represents the credibility of the attribute information. The preset accuracy is a preset value, which can be dynamically set according to the lane line detection accuracy.

[0060] Specifically, the first step can be: in response to the evaluation accuracy in the evaluation result of the attribute information being less than the preset accuracy, confirming that the loss determination strategy is the initial determination strategy. In response to the evaluation accuracy in the evaluation result of the attribute information being greater than or equal to the preset accuracy, confirming that the loss determination strategy is the advanced determination strategy.

[0061] Specifically, the convergence processing can be adjusting the initial lane line coefficients until the fitting loss value of the initial fitting lane line obtained by the adjusted initial lane line coefficients meets the requirement. The third step can be: in the case that the fitting loss value is the smallest, solving the initial lane line coefficients, and taking the solved values as the target lane line coefficients respectively. The target lane line coefficients are the adjusted initial lane line coefficients. Taking the target lane line coefficients as the lane line coefficients of the target lane line, the fitting loss value of the target lane line obtained at this time meets the preset constraint condition. Specifically, the way to obtain the target lane line can be replacing each coefficient in the target lane line coefficients with the values in the first preset coefficient, the second preset coefficient, the third preset coefficient, and the fourth preset coefficient in each initial lane line coefficient in the pre-established parameter model respectively.

[0062] It can be considered that, by determining the loss determination strategy based on the evaluation result of the attribute information in advance, the accuracy of the fitting loss value obtained in the case that the fitting lane line is of different types is higher, so that the accuracy of the target lane line obtained by adjusting the initial fitting lane line through the fitting loss value with higher accuracy is higher.

[0063] In some embodiments, the attribute information comprises evaluation results of the attribute information and fitting functions corresponding to each point in the target point cloud information in the initial fitted lane line. The step S13 can comprise the following steps: in response to the evaluation results representing that the evaluation accuracy is less than the preset accuracy, determining a fitting loss value of the initial fitted lane line according to coordinate information of each point in the target point cloud information and fitting function values of the fitting functions corresponding to the points in the initial fitted lane line. The initial fitted lane line is converged according to the fitting loss value until a target lane line meeting the requirements is obtained.

[0064] For each point, the coordinate information of the point in the target point cloud information represents a value of the point on the first number axis and / or the second number axis in the coordinate system of the current frame image. The coordinate information of each point in the target point cloud information and the coordinate number axis on which the corresponding point in the initial fitted lane line corresponds to the fitting function are the same number axis.

[0065] The first step can be: in response to the evaluation results of the attribute information representing that the evaluation accuracy is less than the preset accuracy, confirming that the loss determination strategy is the initial determination strategy. The initial determination strategy can be set to determine the fitting loss value of the initial fitted lane line according to the coordinate information of each point in the target point cloud information and the fitting function values of the fitting functions corresponding to the points in the initial fitted lane line. The initial determination strategy comprises a first initial sub-strategy and / or a second initial sub-strategy. It can be understood that, in the case where the evaluation results represent that the evaluation accuracy is less than the preset accuracy, after the fitting loss value is obtained by using the initial determination strategy, the third step is executed. Please refer to the above description, which will not be repeated here.

[0066] In some application scenarios, the first initial sub-strategy can include: in the case that the coordinate information of the point in the target point cloud information represents the value of the point on the first number axis in the coordinate system in which the current frame image is located, the coordinate number axis in which the point corresponding to the fitting function in the initial fitting lane line is the first number axis. The fitting function corresponding to the point in the initial fitting lane line is determined based on the coordinate information of the point on the second number axis in the target point cloud information. Specifically, the following steps are performed for each point: the difference between the value of the point on the first number axis in the target point cloud information and the fitting function value of the fitting function corresponding to the point in the initial fitting lane line is taken as the first coordinate difference value corresponding to the point. The first coordinate difference values of the points are weighted and fused to obtain the loss value corresponding to the first initial sub-strategy. Then, the loss value corresponding to the first initial sub-strategy is directly taken as the fitting loss value of the initial fitting lane line. In the process of weighted fusion, the fusion weight of each point can be a preset value, or can be determined according to a preset weight rule. For example, in the case that the fusion weight of each point is a preset value 1, the first coordinate difference value of each point is directly taken as the loss value corresponding to each point in the first initial sub-strategy. In some other application scenarios, in the case that the coordinate information of the point in the target point cloud information represents the value of the point on the second number axis in the coordinate system in which the current frame image is located, the coordinate number axis in which the point corresponding to the fitting function in the initial fitting lane line is the second number axis. The fitting function corresponding to the point in the initial fitting lane line is determined based on the coordinate information of the point on the first number axis in the target point cloud information. Specifically, the following steps are performed for each point: the difference between the value of the point on the second number axis in the target point cloud information and the fitting function value of the fitting function corresponding to the point in the initial fitting lane line is taken as the second coordinate difference value corresponding to the point. The second coordinate difference values of the points are weighted and fused to obtain the loss value corresponding to the second initial sub-strategy. Then, the loss value corresponding to the second initial sub-strategy is directly taken as the fitting loss value of the initial fitting lane line. In the process of weighted fusion, the fusion weight of each point can be a preset value, or can be determined according to a preset weight rule. For example, in the case that the fusion weight of each point is a preset value 1, the second coordinate difference value of each point is directly taken as the loss value corresponding to each point in the second initial sub-strategy.

[0067] In some other application scenarios, in the case that the initial determination strategy is the first initial sub-strategy and the second initial sub-strategy, the loss value corresponding to the first initial sub-strategy and the loss value corresponding to the second initial sub-strategy are weighted and averaged to obtain a processing result, and the processing result is taken as the loss value corresponding to the initial determination strategy. Then, the loss value corresponding to the initial determination strategy is directly taken as the fitting loss value of the initial fitting lane line.

[0068] It can be understood that the first number axis is the X axis, and the second number axis is the Y axis. The only difference between the first initial sub-strategy and the second initial sub-strategy is the number axis in the corresponding coordinate system. The rest of the way to construct the fitting loss value is the same. In the case where the evaluation result represents that the evaluation accuracy is less than the preset accuracy, the application does not limit the specific strategy of using the first initial sub-strategy and / or the second initial sub-strategy in the initial determination strategy. The application takes the second initial sub-strategy as an example of the initial determination strategy, and the subsequent description is omitted.

[0069] It can be considered that using the first initial sub-strategy and / or the second initial sub-strategy in the initial determination strategy determined according to the evaluation result to determine the fitting loss of the initial lane line can combine the coordinate information of each point in the target point cloud information and the coordinate information corresponding to the fitting function obtained by subsequent fitting to determine the fitting loss value, which can improve the accuracy of determining the fitting loss value.

[0070] In some embodiments, the step of determining the fitting loss value of the initial fitting lane line according to the coordinate information of each point in the target point cloud information and the fitting function value of the corresponding fitting function of the corresponding point in the initial fitting lane line can include the following steps: first, the difference between the coordinate information of each point in the target point cloud information and the fitting function value of the corresponding fitting function of the corresponding point in the initial fitting lane line is determined as the coordinate difference. Then, based on the fitting weight of the coordinate information of each point in the target point cloud information and the coordinate difference, the fitting loss value is determined.

[0071] The difference between the coordinate information of each point in the target point cloud information and the fitting function value of the corresponding fitting function of the corresponding point in the initial fitting lane line can be the first coordinate difference value and / or the second coordinate difference value. The coordinate difference represents the degree of deviation between the coordinate information of each point in the target point cloud information and the fitting function value of the corresponding fitting function of the corresponding point in the initial fitting lane line on the same number axis. The coordinate difference represents the coordinate difference between the coordinate information of each point in the target point cloud information and the fitting function value of the corresponding fitting function of the corresponding point in the initial fitting lane line on the same number axis. For example, the coordinate difference can be the loss value corresponding to each point in the first initial sub-strategy, the loss value corresponding to each point in the second initial sub-strategy, and the loss value corresponding to each point in the initial determination strategy.

[0072] The fitting weight of the coordinate information of each point in the target point cloud information represents the importance of the corresponding point in the fitting result obtained based on the preset lane line fitting method.

[0073] The preset weight rule in the weighted fusion of the first initial sub-strategy and / or the second initial sub-strategy can be to determine the fusion weight of the corresponding point according to the distance of each point from the vehicle. The preset weight rule can also be to determine the fusion weight of the corresponding point according to the fitting weight of the coordinate information of each point. In some application scenarios, the closer a point is to the vehicle, the greater the fusion weight of the point, and the fusion weight of each point is directly used as the fitting weight of each point. In other application scenarios, before the step of determining the fitting loss value based on the fitting weight of the coordinate information of each point in the target point cloud information and the coordinate difference, the lane line detection method further includes the following steps: fitting each point in the target point cloud information according to a preset lane line fitting method to obtain the fitting error of each point, and using the fitting error of each point as the fitting weight of the corresponding point. The preset lane line fitting method is a quadratic polynomial fitting method. When a point deviates significantly from the fitting result of the preset lane line fitting method, the point is an abnormal point and will have a smaller fitting weight.

[0074] In some application scenarios, the step of determining the fitting loss value based on the fitting weight of the coordinate information of each point in the target point cloud information and the coordinate difference can include: weighting and fusing the loss values of each point in the coordinate difference according to the fitting weight of each point to obtain the fitting loss value.

[0075] It can be considered that, on the basis of using the coordinate information in the target point cloud information and the fitting function in the initial fitting lane line on the same number axis, the fitting loss is also determined using the fitting weight of each point, which can improve the accuracy of the determined fitting loss.

[0076] In some embodiments, the attribute information includes an evaluation result of the attribute information and an initial lane line coefficient in the initial fitting lane line. The step S13 can include the following steps: in response to the evaluation result representing that the evaluation accuracy is greater than or equal to a preset accuracy, obtaining a historical lane line coefficient corresponding to a historical lane line in a historical frame image. Based on the difference between the historical lane line coefficient and the initial lane line coefficient, a fitting loss value of the initial fitting lane line is determined. The initial fitting lane line is subjected to convergence processing according to the fitting loss value until a target lane line that meets the requirements is obtained.

[0077] The acquisition time of the historical frame image is earlier than the acquisition time of the current frame image. The current frame image and the historical frame image are images collected for the same lane. The lane line in the current frame image is the same lane line as the historical lane line. The historical lane line coefficient is a lane line coefficient in a historical target lane line corresponding to the historical frame image. The historical target lane line includes a historical fitted left lane line and / or a historical fitted right lane line. The historical lane line coefficient includes at least one of a first historical coefficient, a second historical coefficient, a third historical coefficient, and a fourth historical coefficient. Specifically, the first historical coefficient corresponds to a first preset coefficient in the initial lane line coefficient, and the first historical coefficient represents a distance between a rear axle center of the vehicle and the historical fitted left lane line and / or the historical fitted right lane line in the historical target lane line. The second historical coefficient corresponds to a second preset coefficient in the initial lane line coefficient, and the second historical coefficient represents an inverse tangent angle of the historical fitted left lane line and / or the historical fitted right lane line, i.e., a slope of the historical fitted left lane line and / or the historical fitted right lane line. The third historical coefficient corresponds to a third preset coefficient in the initial lane line coefficient, and the third historical coefficient represents a curvature of the historical fitted left lane line and / or the historical fitted right lane line. The fourth historical coefficient corresponds to a fourth preset coefficient in the initial lane line coefficient, and the fourth historical coefficient represents a rate of change of the curvature of the historical fitted left lane line and / or the historical fitted right lane line.

[0078] The first step can be: in response to the evaluation result of the attribute information, if the evaluation accuracy is greater than or equal to the preset accuracy, it is determined that the loss determination strategy is an advanced determination strategy. The advanced determination strategy includes a first advanced strategy. The first advanced strategy can be set to determine the fitting loss value of the initial fitted lane line based on the difference between the historical lane line coefficient and the initial lane line coefficient. It can be understood that, in the case where the evaluation result represents that the evaluation accuracy is greater than or equal to the preset accuracy, after the fitting loss value is obtained by using the first advanced strategy, the third step is executed. Please refer to the above description, which will not be repeated here.

[0079] The historical lane line coefficients include any one of a first historical coefficient, a second historical coefficient, a third historical coefficient, and a fourth historical coefficient. In some application scenarios, the first advanced determination strategy can include: determining a difference between the first preset coefficient and the first historical coefficient as a first coefficient difference. Directly taking the first coefficient difference as a loss value corresponding to the first advanced strategy. Or, determining a product between the first coefficient difference and a fitting weight corresponding to the first preset coefficient as a first coefficient product, and taking the first coefficient product as the loss value corresponding to the first advanced strategy. In other application scenarios, the first advanced determination strategy can include: determining a difference between the second preset coefficient and the second historical coefficient as a second coefficient difference. Directly taking the second coefficient difference as a loss value corresponding to the first advanced strategy. Or, determining a product between the second coefficient difference and a fitting weight corresponding to the second preset coefficient as a second coefficient product, and taking the second coefficient product as the loss value corresponding to the first advanced strategy. In other application scenarios, the first advanced determination strategy can include: determining a difference between the third preset coefficient and the third historical coefficient as a third coefficient difference. Directly taking the third coefficient difference as a loss value corresponding to the first advanced strategy. Or, determining a product between the third coefficient difference and a fitting weight corresponding to the third preset coefficient as a third coefficient product, and taking the third coefficient product as the loss value corresponding to the first advanced strategy. In other application scenarios, the first advanced determination strategy can include: determining a difference between the fourth preset coefficient and the fourth historical coefficient as a fourth coefficient difference. Directly taking the fourth coefficient difference as a loss value corresponding to the first advanced strategy. Or, determining a product between the fourth coefficient difference and a fitting weight corresponding to the fourth preset coefficient as a fourth coefficient product, and taking the fourth coefficient product as the loss value corresponding to the first advanced strategy. Then, taking the loss value corresponding to the first advanced strategy as the fitting loss value of the initial fitted lane line. The fitting weight corresponding to the first preset coefficient, the fitting weight corresponding to the second preset coefficient, the fitting weight corresponding to the third preset coefficient, and the fitting weight corresponding to the fourth preset coefficient can be preset values, and can be dynamically set according to the accuracy of lane line detection.

[0080] In other application scenarios, the historical lane line coefficients include at least two of the first historical coefficient, the second historical coefficient, the third historical coefficient, and the fourth historical coefficient. Based on the difference between the initial lane line coefficient and the corresponding at least two of the first historical coefficient, the second historical coefficient, the third historical coefficient, and the fourth historical coefficient, the fitting loss value of the initial fitted lane line is determined. Specifically, at least two of the first coefficient difference or the first coefficient product, the second coefficient difference or the second coefficient product, the third coefficient difference or the third coefficient product, and the fourth coefficient difference or the fourth coefficient product are weighted and fused to obtain a new coefficient difference, and the new coefficient difference is taken as the loss value corresponding to the first advanced strategy. Directly taking the loss value corresponding to the first advanced strategy as the fitting loss value of the initial fitted lane line.

[0081] In some embodiments, the historical lane line coefficients include a third historical coefficient and a fourth historical coefficient, the initial lane line coefficients include a third preset coefficient, and before the step of determining the fitting loss value of the initial fitted lane line based on a difference between the historical lane line coefficients and the initial lane line coefficients, the lane line detection method includes: obtaining a sampling time difference between the current frame image and the historical frame image and a speed of the vehicle corresponding to the current frame image. The sampling time difference represents a difference between a sampling time of the current frame image and a sampling time of the historical frame image. The speed of the vehicle represents a real-time speed of the vehicle at the sampling time of the current frame image. The speed of the vehicle can be obtained by a wheel speedometer. Based on the third historical coefficient, the fourth historical coefficient, the sampling time difference, and the speed, a new third historical coefficient is constructed, including: taking a product of a preset value, the fourth historical coefficient, the sampling time difference, and the speed as a construction product. The preset value can be set to 3. Taking a sum value between the third historical coefficient and the construction product as a construction sum. The construction sum is taken as the new third historical coefficient. In the step of determining the fitting loss value of the initial fitted lane line based on the difference between the historical lane line coefficients and the initial lane line coefficients, the third coefficient difference can be determined by: taking a difference between the third preset coefficient and the new third historical coefficient as the third coefficient difference.

[0082] It can be considered that the fitting loss value obtained by the difference between the historical lane line coefficients and the initial lane line coefficients can satisfy the linear change of the lane line between different image frames, so that the accuracy of the fitting loss value is higher, and the accuracy of the target lane line obtained based on the fitting loss value is higher.

[0083] In some embodiments, the initial fitted lane line includes a fitted left lane line and a fitted right lane line, and the attribute information includes an evaluation result of the attribute information, a left lane line coefficient in the fitted left lane line, and a right lane line coefficient in the fitted right lane line. The step S13 can include the following steps: in response to the evaluation result representing that the evaluation accuracy is greater than or equal to a preset accuracy, determining the fitting loss value of the initial fitted lane line based on a difference between the left lane line coefficient and the right lane line coefficient. The initial fitted lane line is subjected to a convergence process according to the fitting loss value until a target lane line meeting the requirements is obtained.

[0084] The first step can be: in response to the evaluation result of the attribute information, if the evaluation accuracy is greater than or equal to the preset accuracy, confirming that the loss determination strategy is an advanced determination strategy. The advanced determination strategy includes a second advanced strategy. The second advanced strategy can be set to determine the fitting loss value of the initial fitting lane based on the difference between the left lane coefficient and the right lane coefficient. The attribute information includes the type of the initial fitting lane and the category of the preset lane line to which the initial fitting lane belongs. In response to the attribute information being a completely parallel double lane line or a partially parallel double lane line, the step of obtaining the fitting loss value of the initial fitting lane using the second advanced strategy is performed. It can be understood that, in the case where the evaluation result represents that the evaluation accuracy is greater than or equal to the preset accuracy, after the fitting loss value is obtained using the second advanced determination strategy, the third step is performed. Please refer to the above description, which will not be repeated here.

[0085] The left lane coefficients in the fitting left lane line include at least one of a first left preset coefficient, a second left preset coefficient, a third left preset coefficient, and a fourth left preset coefficient. The first left preset coefficient represents the distance between the rear axle center of the vehicle and the fitting left lane line. The second left preset coefficient represents the inverse tangent of the orientation angle of the fitting left lane line, i.e., the slope of the fitting left lane line. The third left preset coefficient represents the curvature of the fitting left lane line. The fourth left preset coefficient represents the rate of change of the curvature of the fitting left lane line. The right lane coefficients in the fitting right lane line include at least one of a first right preset coefficient, a second right preset coefficient, a third right preset coefficient, and a fourth right preset coefficient. The first right preset coefficient represents the distance between the rear axle center of the vehicle and the fitting right lane line. The second right preset coefficient represents the inverse tangent of the orientation angle of the fitting right lane line, i.e., the slope of the fitting right lane line. The third right preset coefficient represents the curvature of the fitting right lane line. The fourth right preset coefficient represents the rate of change of the curvature of the fitting right lane line.

[0086] The left lane line coefficients in the fitted left lane line include any one of the first left preset coefficient, the second left preset coefficient, the third left preset coefficient, and the fourth left preset coefficient, and the right lane line coefficients in the fitted right lane line include any one of the first right preset coefficient, the second right preset coefficient, the third right preset coefficient, and the fourth right preset coefficient. In some application scenarios, the second advanced determination strategy can include: determining the difference between the first left preset coefficient and the first right preset coefficient as a first preset coefficient difference. The first preset coefficient difference is directly taken as the loss value corresponding to the second advanced strategy. Or, the product between the first preset coefficient difference and the fitting weight corresponding to the first preset coefficient is determined as a first preset coefficient product, and the first preset coefficient product is taken as the loss value corresponding to the second advanced strategy. In other application scenarios, the second advanced determination strategy can include: determining the difference between the second left preset coefficient and the second right preset coefficient as a second preset coefficient difference. The second preset coefficient difference is directly taken as the loss value corresponding to the second advanced strategy. Or, the product between the second preset coefficient difference and the fitting weight corresponding to the second preset coefficient is determined as a second preset coefficient product, and the second preset coefficient product is taken as the loss value corresponding to the second advanced strategy. In other application scenarios, the second advanced determination strategy can include: determining the difference between the third left preset coefficient and the third right preset coefficient as a third preset coefficient difference. The third preset coefficient difference is directly taken as the loss value corresponding to the second advanced strategy. Or, the product between the third preset coefficient difference and the fitting weight corresponding to the third preset coefficient is determined as a third preset coefficient product, and the third preset coefficient product is taken as the loss value corresponding to the second advanced strategy. In other application scenarios, the second advanced determination strategy can include: determining the difference between the fourth left preset coefficient and the fourth right preset coefficient as a fourth preset coefficient difference. The fourth preset coefficient difference is directly taken as the loss value corresponding to the second advanced strategy. Or, the product between the fourth preset coefficient difference and the fitting weight corresponding to the fourth preset coefficient is determined as a fourth preset coefficient product, and the fourth preset coefficient product is taken as the loss value corresponding to the second advanced strategy. Then, the loss value corresponding to the second advanced strategy is taken as the fitting loss value of the initial fitted lane line. The fitting weight corresponding to the first preset coefficient, the fitting weight corresponding to the second preset coefficient, the fitting weight corresponding to the third preset coefficient, and the fitting weight corresponding to the fourth preset coefficient can be preset values, which can be dynamically set according to the accuracy of lane line detection. It can be understood that the first preset coefficient includes the first left preset coefficient and the first right preset coefficient. The second preset coefficient includes the second left preset coefficient and the second right preset coefficient. The third preset coefficient includes the third left preset coefficient and the third right preset coefficient. The fourth preset coefficient includes the fourth left preset coefficient and the fourth right preset coefficient.

[0087] In some embodiments, the fitting left lane line includes a first left preset coefficient, a second left preset coefficient, a third left preset coefficient, and a fourth left preset coefficient, and the fitting right lane line includes a first right preset coefficient, a second right preset coefficient, a third right preset coefficient, and a fourth right preset coefficient. The first preset coefficient difference or the first preset coefficient product, the second preset coefficient difference or the second preset coefficient product, the third preset coefficient difference or the third preset coefficient product, and the fourth preset coefficient difference or the fourth preset coefficient product are fused to obtain a new preset coefficient difference, and the new preset coefficient difference is taken as a loss value corresponding to the second advanced strategy. The loss value corresponding to the second advanced strategy is directly taken as the fitting loss value of the initial fitting lane line.

[0088] It can be considered that the fitting loss value obtained by the change of the lane line coefficients of the left and right lane lines can reflect the parallel constraint in the double lane line, and the fitting loss value corresponding to the double lane line in this type is more accurate, and the accuracy of the target lane line obtained based on the fitting loss value is higher.

[0089] In some embodiments, the initial fitting lane line includes a fitting left lane line and a fitting right lane line, the fitting left lane line includes a first point and a second point, the fitting right lane line includes a third point and a fourth point, the first point and the third point in the target point cloud information have the same value on a preset number axis, and the second point and the fourth point have the same value on the preset number axis, the attribute information includes an evaluation result of the attribute information and a fitting function corresponding to each point in the initial fitting lane line in the target point cloud information. The step S13 can include the following steps: in response to the evaluation result representing that the evaluation accuracy is greater than or equal to a preset accuracy, taking a difference between a fitting function value of the fitting function corresponding to the first point in the fitting left lane line and a fitting function value of the fitting function corresponding to the third point in the fitting right lane line as a first distance. Taking a difference between a fitting function value of the fitting function corresponding to the second point in the fitting left lane line and a fitting function value of the fitting function corresponding to the fourth point in the fitting right lane line as a second distance. Based on the first distance and the second distance, determining a fitting loss value of the initial fitting lane line. Converging the initial fitting lane line according to the fitting loss value until a target lane line meeting a requirement is obtained.

[0090] The left lane line fitting includes the first point and the second point, and the right lane line fitting includes the third point and the fourth point. The first point and the third point in the target point cloud information have the same value on a preset number axis, and the second point and the fourth point have the same value on the preset number axis. The preset number axis represents any one number axis of the target point cloud information of the input parameter model of the first point and the third point, and the second point and the fourth point in the process of obtaining the initial fitting lane line. For example, the preset number axis is the X axis of the coordinate system of the current frame image. The first point and the second point are different points on the fitted left lane line. The third point and the fourth point are different points on the fitted right lane line.

[0091] The first step can be: in response to the evaluation result of the attribute information, if the evaluation accuracy is greater than or equal to the preset accuracy, the loss determination strategy is determined as the advanced determination strategy. The advanced determination strategy includes a third advanced strategy. The third advanced strategy can be set to determine the fitting loss value of the initial fitting lane line based on the first distance and the second distance. The attribute information includes the type of the initial fitting lane line and the category of the preset lane line to which the initial fitting lane line belongs. In response to the attribute information being a completely parallel double lane line or a partially parallel double lane line, the third advanced strategy is executed to obtain the fitting loss value of the initial fitting lane line. It can be understood that in the case where the evaluation result represents that the evaluation accuracy is greater than or equal to the preset accuracy, after the fitting loss value is obtained by using the third advanced determination strategy, the third step is executed. Please refer to the above content, which will not be repeated here.

[0092] The step of determining the first distance and the step of determining the second distance can be executed in series or in parallel, and the application does not limit the order of execution in the case of series execution. The value of the first distance is the difference between the fitting function value of the corresponding fitting function of the first point in the fitted left lane line and the fitting function value of the corresponding fitting function of the third point in the fitted right lane line. The value of the second distance is the difference between the fitting function value of the corresponding fitting function of the second point in the fitted left lane line and the fitting function value of the corresponding fitting function of the fourth point in the fitted right lane line.

[0093] In some application scenarios, the third advanced strategy can be executed in the following manner: the difference between the first distance and the second distance is determined as an initial distance difference, and the initial distance difference is directly used as the loss value corresponding to the third advanced strategy.

[0094] In some embodiments, the third advanced strategy is executed in the following manner: according to the distances between the vehicle and the first point and the third point on the preset number axis, a weight of the first distance is determined; according to the distances between the vehicle and the second point and the fourth point on the preset number axis, a weight of the second distance is determined; the initial distance difference is adjusted according to the weight of the first distance and the weight of the second distance to obtain a target distance difference, and the target distance difference is taken as the loss value corresponding to the third advanced strategy. Then, the loss value corresponding to the third advanced strategy is taken as the fitting loss value of the initial fitted lane line.

[0095] It can be considered that, in the case of a double-lane line, the fitting loss value obtained by the difference between the lane widths corresponding to any two points on the same lane line can reflect the parallel constraint in the double-lane line, and the accuracy of the fitting loss value corresponding to the double-lane line under this type is higher, and thus the accuracy of the target lane line obtained based on the fitting loss value adjustment is higher.

[0096] In some embodiments, the fifth point is used to fit the left lane line, and the sixth point is used to fit the right lane line. The fifth point and the sixth point in the target point cloud information have the same value on the preset number axis. The first point, the second point, and the fifth point are adjacent to each other on the fitted left lane line. The third point, the fourth point, and the sixth point are adjacent to each other on the fitted right lane line. Based on the first distance and the second distance, the step of determining the fitting loss value of the initial fitted lane line includes: taking the difference between the fitting function value of the fifth point in the fitted left lane line and the fitting function value of the sixth point in the fitted right lane line as a third distance. Taking the difference between the first distance and the second distance as a first distance difference. Taking the difference between the second distance and the third distance as a second distance difference. Based on the difference between the first distance difference and the second distance difference, the fitting loss value of the initial fitted lane line is determined.

[0097] The first point and the second point and the fifth point are used to fit the left lane line. The third point and the fourth point and the sixth point are used to fit the right lane line. The first point and the third point in the target point cloud information have the same value on the preset number axis, the second point and the fourth point have the same value on the preset number axis, and the fifth point and the sixth point have the same value on the preset number axis. The preset number axis represents any one number axis in the target point cloud information of the parameter model input by the first point and the third point, the second point and the fourth point, and the fifth point and the sixth point in the process of obtaining the initial fitted lane line. For example, the preset number axis is taken as the X-axis of the coordinate system of the current frame image. The first point, the second point, and the fifth point are different points on the fitted left lane line. The third point, the fourth point, and the sixth point are different points on the fitted right lane line. The second point is the middle point between the first point and the fifth point. The fourth point is the middle point between the third point and the sixth point.

[0098] The first step can be: in response to the evaluation accuracy in the evaluation result of the attribute information being greater than or equal to the preset accuracy, confirming that the loss determination strategy is an advanced determination strategy. The advanced determination strategy includes a third advanced strategy. The third advanced determination strategy can be set to determine the fitting loss value of the initial fitted lane line based on the difference between the first distance difference and the second distance difference. The attribute information includes the type of the initial fitted lane line and the category of the preset lane line to which the initial fitted lane line belongs. In response to the attribute information being a completely parallel double-lane line or a partially parallel double-lane line, the step of obtaining the fitting loss value of the initial fitted lane line using the third advanced strategy is performed. It can be understood that, in the case where the evaluation result represents that the evaluation accuracy is greater than or equal to the preset accuracy, after the fitting loss value is obtained using the third advanced determination strategy, the third step is performed. Please refer to the above description, which will not be repeated here.

[0099] The step of determining the first distance, the step of determining the second distance, and the step of determining the third distance can be executed in series or in parallel. In the case of serial execution, the application does not limit the order of execution. The value of the third distance is the difference between the fitting function value of the fitting function corresponding to the fifth point in the fitted left lane line and the fitting function value of the fitting function corresponding to the sixth point in the fitted right lane line.

[0100] In some application scenarios, the third advanced strategy can be executed in the following manner: determining the difference between the first distance difference and the second distance difference as an initial difference value, and directly taking the initial difference value as the loss value corresponding to the third advanced strategy. In other application scenarios, the third advanced strategy can be executed in the following manner: determining the weight of the first distance difference according to the distance between the first point and the second point on the fitted left lane line and / or the distance between the third point and the fourth point on the fitted right lane line. The weight of the second distance difference is determined according to the distance between the second point and the fifth point on the fitted left lane line and / or the distance between the fourth point and the sixth point on the fitted right lane line. It can be understood that, according to the accuracy of lane line detection, the weight of the distance difference can be dynamically set, for example, if the distance between the first point and the second point on the fitted left lane line is farther, the weight of the first distance difference value is greater or smaller. The weight of the second distance is determined according to the distance between the second point and the fourth point on the preset number axis from the vehicle. The initial difference value is adjusted according to the weight of the first distance difference and the weight of the second distance difference to obtain a target difference value, and the target difference value is taken as the loss value corresponding to the third advanced strategy. Then, the loss value corresponding to the third advanced strategy is taken as the fitting loss value of the initial fitted lane line.

[0101] It can be considered that, in the case of double-lane lines, the fitting loss value of the initial fitted lane line determined based on the difference between the first distance difference and the second distance difference can reflect the parallel constraint in the double-lane lines, and the accuracy of the fitting loss value corresponding to the double-lane lines in this type is higher, and thus the accuracy of the target lane line obtained by adjusting based on the fitting loss value is higher.

[0102] It can be understood that the advanced determination strategy includes the initial determination strategy and other determination strategies. The other determination strategies are at least one of the first advanced strategy, the second advanced strategy, and the third advanced strategy. In some application scenarios, in the case that the other determination strategies are only any one of the first advanced strategy, the second advanced strategy, and the third advanced strategy, after the loss value corresponding to the initial determination strategy and the loss value corresponding to the advanced determination strategy are determined, the loss value corresponding to the other determination strategy is directly added or weightedly fused with the loss value corresponding to the initial determination strategy, and the fusion result is taken as the loss value corresponding to the advanced determination strategy. The loss value corresponding to the advanced determination strategy is taken as the fitting loss value of the initial fitted lane line. In other application scenarios, in the case that the other determination strategies are at least two of the first advanced strategy, the second advanced strategy, and the third advanced strategy, the loss values determined by the at least two are first added or weightedly fused to obtain a new loss value. The new loss value is added or weightedly fused with the loss value corresponding to the initial determination strategy to obtain a final loss value, and the final loss value is taken as the loss value corresponding to the advanced determination strategy. The loss value corresponding to the advanced determination strategy is taken as the fitting loss value of the initial fitted lane line.

[0103] Please refer to Figure 3 , Figure 3 is a schematic diagram of the framework of an exemplary embodiment of the lane line detection method of the present application.

[0104] The lane line detection method is sequentially executed by the lane line perception module, the lane line fusion module, the lane line analysis module, the lane line fitting module, and the lane line output module. The input of the lane line perception module is the current frame image. The lane line perception module is configured to execute the step S21 to obtain the initial point cloud information corresponding to the current frame image. The lane line fusion module is configured to execute the steps S22-S23 to obtain the target point cloud information after obtaining the historical point cloud information of the historical frame image. The lane line fusion module is further configured to execute the step of fitting each point in the target point cloud information according to the preset lane line fitting method, obtaining the fitting error of each point, and taking the fitting error of each point as the fitting weight of the corresponding point after obtaining the target point cloud information. The lane line analysis module is configured to execute the step of analyzing the target point cloud information to obtain the analysis result of the plurality of points in the target point cloud information, and taking the analysis result of the plurality of points in the target point cloud information as the type and / or evaluation result in the attribute information of the initial fitting lane line. The lane line fitting module is configured to execute the steps S12-S13. The result output module is configured to output the target lane line obtained in the step S13 to the downstream task of the vehicle.

[0105] Exemplarily, the vehicle-mounted camera (such as a front-view camera) of the vehicle acquires a current frame image containing a lane line, and inputs the current frame image to the lane line perception module. The lane line perception module extracts lane line information by using a neural network, and discretizes the lane line information to generate the initial point cloud information in the coordinate system of the current frame image the initial point cloud information in the coordinate system .

[0106] The lane line fusion module is responsible for projecting the previous frame historical frame image of the current frame image the historical point cloud information in the coordinate system according to the relative motion relationship to obtain new historical point cloud information in the coordinate system of the current frame image . For example, the relative motion relationship between the previous frame and the current frame can be estimated by using an inertial measurement unit. The initial point cloud information and the new historical point cloud information are fused and removed beyond the sliding window data processing to obtain the target point cloud information in the coordinate system of the current frame image . Then, the target point cloud information is fitted by using a preset lane line fitting method (such as a quadratic polynomial lane line fitting method) to obtain the fitting error of each point in the target point cloud information, and the fitting weight corresponding to each point is assigned based on the fitting error of each point . Specifically, the process of determining the new historical point cloud information can refer to the following formula (2), and the process of determining the fitting weight of any point in the target point cloud information can refer to the following formula (3):

[0107] formula (2);

[0108] Equation (3);

[0109] wherein, represents the point cloud information of the i-th point in the historical point cloud information. represents the point cloud information of the i-th point in the new historical point cloud information in the coordinate system of the current frame image. represents the relative motion between adjacent frames (the current frame image and the historical frame image). represents the fitting weight of the i-th point in the target point cloud information. represents the fitting error of the i-th point in the preset lane line fitting method. represents the scale parameter, which can be dynamically set to a preset value according to the lane line detection accuracy, and can be dynamically set to 0.1 in the present application.

[0110] It can be considered that, in order to save computing resources, the present application uses a sliding window to manage the lane line point cloud set, and only when the lane line point cloud is within a certain range of the vehicle will it be retained to obtain the final target point cloud information. In the lane line fitting process of the current frame image, the historical point cloud information is integrated, which can effectively suppress the overfitting phenomenon caused by occlusion, light, and road marking wear, and improve the consistency and robustness of the fitting result in the time dimension.

[0111] The lane line analysis module analyzes the target point cloud information to determine the analysis results of a plurality of points in the target point cloud information. The analysis results are used as the type in the attribute information of the initial fitting lane line and / or the evaluation result of the vehicle. The analysis results of the plurality of points include prior information of the lane line corresponding to the plurality of points and accuracy of the prior information. The prior information includes but is not limited to the type of the lane line corresponding to the plurality of points (such as a double lane line or a single lane line), whether the lane line corresponding to the plurality of points is parallel, the farthest distance at which the lane width of the double lane line corresponding to the plurality of points remains equal, and whether the lane line corresponding to the plurality of points is one of a plurality of preset lane lines. The plurality of preset lane lines can be the preset lane lines shown in Figure 4a , Figure 4b , Figure 4c and Figure 4d . The prior information includes but is not limited to a completely parallel double lane line (such as Figure 4a or Figure 4d ), a partially parallel double lane line (such as Figure 4b ), a completely non-parallel double lane line (such as Figure 4c ), and a single lane line in the plurality of preset lane lines. For example, as shown in Figure 4a ​​​​​​​or Figure 4b At least part of the left and right lane lines are parallel, at this time, the width threshold is set A certain fixed value. Exemplarily, considering the actual needs of lane line application . Figure 4b In the middle, due to the existence of a broken line, only a certain range of lane lines is parallel, by comparing the lane width from near to far, the lane line analysis module gives the distance of the lane line break point from the vehicle . Figure 4c In the middle, the left and right lane lines do not have a parallel relationship, and the lane line fitting module will discard the lane line parallel constraint (i.e., the second and third advanced determination strategies).

[0112] In some application scenarios, the lane line fitting module can pre-establish a parameter model. The parameter model takes the coordinate system in which the current frame image is located as the coordinate system corresponding to the parameter model. Among them, the first number axis and the second number axis in the coordinate system corresponding to the parameter model can be the X axis and the Y axis in the coordinate system in which the current frame image is located. The input of the lane line fitting module can include the point cloud information of all points in the target point cloud set, wherein it is assumed that the target point cloud has points, and the point cloud information of the i-th point can be represented as , is the fitting weight of the i-th point.

[0113] The lane line fitting module models the lane line using a polynomial parameter model and obtains the initial lane line coefficient optimal solution by analytical solution. In the modeling process, the road geometric characteristics are fully integrated to improve the lane line fitting accuracy and robustness, including lane line curvature linear variation and left and right lane lines approximately parallel. The establishment process of the parameter model can refer to the above formula (1). There are In some application scenarios, in response to the evaluation accuracy being less than the preset accuracy in the evaluation result of the attribute information, the loss determination strategy is determined to be the initial determination strategy. When the road geometric characteristics cannot be determined, the initial determination strategy can be executed to obtain the fitting loss value.

[0114] Specifically, the process of calculating the loss value corresponding to the initial determination strategy can be referred to formula (4):

[0115] formula (4);

[0116] wherein, represents the loss value corresponding to the initial determination strategy. is used to represent the lane line point cloud fitting error. The lane line point cloud fitting error represents the difference between the coordinate information of each point in the target point cloud information and the fitting function value of the corresponding point in the initial fitting lane line. The lane line point cloud fitting error satisfies​ ,in, , Characterizing all points in the above formula (1) The column vectors. T is the transpose sign. The diagonal matrix is... Characterizing the fitting weights for all points, It is the first The fitting weights for each point. Q1 and Q2 are preset values. For example, Q1 is set to 1 and Q2 is set to 2. N is the total number of points in the target point cloud information. For the other parameters in formula (4), please refer to the description of formula (1), which will not be repeated here.

[0117] When the loss value corresponding to the initially determined strategy is used as the fitting loss value, the optimal solution corresponding to the initial lane line coefficients when the fitting loss value is minimized can be expressed as the following formula (5):

[0118] Formula (5);

[0119] For the parameters in formula (5), please refer to the descriptions in formulas (1) and (4), which will not be repeated here. It is understandable that the parameters in formula (5)... With formula (1) All represent the column vectors of the coefficients contained in the initial lane line coefficients. Combining formulas (1) and (4), we obtain the solution equation for the initial lane line coefficients in formula (5). When the loss value corresponding to the initial determined strategy is used as the fitting loss value, the optimal solution corresponding to the initial lane line coefficients is used as the output of the lane line fitting module when the fitting loss value is minimized.

[0120] In other application scenarios, the calculation process of the loss value corresponding to the first advanced strategy can refer to the following formulas (6) to (7):

[0121] Formula (6);

[0122] Formula (7);

[0123] Among them, according to the linear change law of lane curvature, the relationship between the fitting coefficients of the previous and next frames satisfies the above formula (6). For example, the specific process of constructing a new third historical coefficient based on the third historical coefficient and the fourth historical coefficient, the sampling time difference and the speed can be the above formula (6). The third historical coefficient is used to represent the historical lane line coefficients corresponding to the historical frame image. This is used to represent the fourth historical coefficient in the historical lane line coefficients corresponding to the historical frame image. Q3 is a preset value. For example, the value of Q3 can be 3. Used to indicate the speed of a vehicle. is used to represent the above-mentioned collection time difference. is used to represent the above-mentioned new third historical coefficient.

[0124] On the basis of formula (6), the setting of formula (7) takes the following into account: in order to ensure driving comfort and safety, the curvature of a single lane in the initial fitted lane line needs to comply with the linear change rule, whether it is a single lane line or a double lane line. The linear change rule can be expressed as , wherein is the curvature at the starting point of the curve, is the rate of change of the curvature, is the arc length of the lane line, is the curvature corresponding to the arc length . It is assumed that the historical lane line coefficient of the historical frame image is , wherein , , , and can represent the first historical coefficient, the second historical coefficient, the third historical coefficient, and the fourth historical coefficient in the historical lane line coefficient, respectively. is used to represent the column vector of each historical coefficient in the historical lane line coefficient. The current frame lane line fitting coefficient is , the time interval between the previous frame image and the current frame image is , and the vehicle speed is , wherein , , , and can represent the first preset coefficient, the second preset coefficient, the third preset coefficient, and the fourth preset coefficient in the initial lane line coefficient, respectively. is used to represent the column vector of each coefficient in the initial lane line coefficient corresponding to the current frame image. Considering that the time interval (for example, the above-mentioned collection time difference ) between the previous frame and the current frame is short, the historical lane line coefficient and the initial lane line coefficient change slowly, and in order to ensure the stability of the fitting coefficient, the prior information corresponding to the initial lane line coefficient of the current frame image can be set as is used to represent the prior information corresponding to the initial lane line coefficient, , wherein , , , and respectively represent the first historical coefficient, the second historical coefficient, a new third historical coefficient constructed by the above formula (6), and the fourth historical coefficient. The new third historical coefficient obtained by using the linear change rule and other historical coefficients are integrated into the calculation of the loss value corresponding to the first advanced strategy as prior information to obtain a target error term. The target error term represents the deviation between each preset coefficient in the initial lane line coefficient and the corresponding historical coefficient in the prior. is used to represent the above target error term, and satisfies In addition, the fitting weight corresponding to each preset coefficient of the initial lane line coefficient is , , and . Among them, , , and respectively represent the fitting weight corresponding to the first preset coefficient, the fitting weight corresponding to the second preset coefficient, the fitting weight corresponding to the third preset coefficient, and the fitting weight corresponding to the fourth preset coefficient. is used to represent the loss function corresponding to the lane line curvature linear change prior constraint, that is, the loss value corresponding to the first advanced strategy. Among them, is used to represent the diagonal matrix corresponding to the fitting weight corresponding to each preset coefficient of the initial lane line coefficient. For example, the diagonal matrix In actual application, the fitting weights of different preset coefficients can be selectively set according to the actual situation of lane line detection accuracy.

[0125] It can be understood that, in combination with formula (6) and formula (7), when the loss value corresponding to the first advanced strategy is taken as the fitting loss value, the optimal solution of the lane line coefficient is solved when the fitting loss value is the smallest, and the obtained optimal solution is taken as the output of the lane line fitting module.

[0126] In some other application scenarios, the calculation process of the loss value corresponding to the second advanced strategy can refer to the following formula (8):

[0127] Formula (8);

[0128] In the case of attribute information being a completely parallel double-lane line or a partially parallel double-lane line, the step of obtaining a fitting loss value of an initial fitting lane line by using the second advanced strategy is performed. In consideration of the lane line parallel constraint between the fitted left lane line and the fitted right lane line in the initial fitting lane line, the following is considered for the setting of formula (8): according to the output result of the lane line analysis module, when the left and right lane lines are parallel to each other within a range D from the vehicle, the slope, curvature and curvature rate of the left and right lane lines in the vehicle coordinate system should be approximately equal, that is, the second left preset coefficient, the third left preset coefficient and the fourth left preset coefficient in the left lane line coefficient are approximately equal to the second right preset coefficient, the third right preset coefficient and the fourth right preset coefficient in the right lane line coefficient. Among them, is used to represent the loss value corresponding to the second advanced strategy. is used to represent that the left lane line coefficient in the fitted left lane line includes at least one of the second left preset coefficient, the third left preset coefficient and the fourth left preset coefficient. is used to represent that the right lane line coefficient in the fitted right lane line includes at least one of the second right preset coefficient, the third right preset coefficient and the fourth right preset coefficient. is used to represent the deviation degree between each coefficient in the left lane line coefficient and each coefficient in the right lane line coefficient, which is referred to as a preset coefficient difference or an error function. The preset coefficient difference satisfies In addition, the fitting weight corresponding to each preset coefficient of the initial lane line coefficient is , , and . Among them, , , and respectively represent the fitting weight corresponding to the first preset coefficient, the fitting weight corresponding to the second preset coefficient, the fitting weight corresponding to the third preset coefficient and the fitting weight corresponding to the fourth preset coefficient. Among them, is used to represent the diagonal matrix corresponding to the fitting weight corresponding to each preset coefficient of the initial lane line coefficient. For example, the diagonal matrix In actual application, the fitting weights of different preset coefficients can be selectively set according to the actual situation of lane line detection accuracy. Exemplarily, since the first left preset coefficient in the left lane line coefficient is not equal to the first right preset coefficient in the right lane line coefficient, the fitting weight coefficient corresponding to the first preset coefficient is . , and The specific values of the above can be set according to actual situations.

[0129] In some other application scenarios, the calculation process of the loss value corresponding to the third advanced strategy can refer to formula (9) as follows:

[0130] Equation (9);

[0131] In the case of attribute information being a completely parallel double-lane line or a partially parallel double-lane line, a step of obtaining a fitting loss value of an initial fitting lane line by using a third advanced strategy is performed. In this case, considering that the left and right lane lines are parallel in a certain part, the fitting left lane line and the fitting right lane line satisfy the condition that the distance between the fitting left lane line and the vehicle is equal to the distance between the fitting right lane line and the vehicle. For the setting of Equation (9), the following is considered:

[0132] The part where the left and right lane lines are parallel satisfies the condition that the lane width in the range D from the vehicle in the initial fitting lane line is equal in the driving direction of the vehicle. Three points in the range D are randomly selected on the left and right lane lines, respectively. Specifically, the first point, the second point, and the fifth point are obtained by equidistant sampling or random sampling on the above-mentioned left fitting lane line, wherein the first point, the second point, and the fifth point are represented as , , respectively. The third point, the fourth point, and the sixth point are obtained by equidistant sampling or random sampling on the above-mentioned right fitting lane line, wherein the third point, the fourth point, and the sixth point are represented as , , respectively. The coordinate information of the first point and the third point on the X-axis of the coordinate system of the current frame image has the same value. The coordinate information of the second point and the fourth point on the X-axis of the coordinate system of the current frame image has the same value. The coordinate information of the fifth point and the sixth point on the X-axis of the coordinate system of the current frame image has the same value. The second point is the middle point between the first point and the fifth point on the left fitting lane line. The fourth point is the middle point between the third point and the sixth point on the right fitting lane line. Based on this, the lane width at the coordinate , , on the X-axis of the coordinate system is , and respectively. , and respectively represent the above-mentioned first distance, the above-mentioned second distance, and the above-mentioned third distance. Specifically, the calculation process of the above-mentioned first distance, the above-mentioned second distance, and the above-mentioned third distance can be represented as , and . The first distance difference can be represented as . The first distance difference represents the error of the lane width at the coordinate , . Specifically, the calculation process of the first distance difference can satisfy The second distance difference can be represented as The second distance difference represents the error of the lane width of the coordinate , in the initial fitted lane line. Specifically, the calculation process of the second distance difference can satisfy Wherein, Wherein, . , and The calculation can refer to formula (1), specifically is used to represent the column vector of the coordinate information of the point in the X axis in the initial fitted lane line at the coordinate is used to represent the column vector of the coordinate information of the point in the X axis in the initial fitted lane line at the coordinate . is used to represent the column vector of the coordinate information of the point in the X axis in the initial fitted lane line at the coordinate The difference between the first distance difference and the second distance difference obtained within the D range of the initial fitted lane line can be represented as The difference between the first distance difference and the second distance difference satisfies . . is used to represent the loss value corresponding to the third advanced strategy. In addition, the weight of the first distance difference and the weight of the second distance difference can be represented as , Wherein, is used to represent the diagonal matrix composed of the weight of the first distance difference and the weight of the second distance difference, and the weight of the first distance difference and the weight of the second distance difference satisfy The value of , may be adjusted according to actual conditions, for example, the weight of the distance difference obtained by the ego vehicle coordinate closer to the vehicle in the coordinate system X axis coordinate , , is set to a relatively large value.

[0133] It can be understood that the advanced determination strategy includes the initial determination strategy and other determination strategies. The other determination strategies are at least one of the first advanced strategy, the second advanced strategy, and the third advanced strategy. In some application scenarios, when the other determination strategies are only any one of the first advanced strategy, the second advanced strategy, and the third advanced strategy, after the loss value corresponding to the initial determination strategy and the loss value corresponding to the advanced determination strategy are determined, the loss value corresponding to the other determination strategy is directly added to or weightedly fused with the loss value corresponding to the initial determination strategy to obtain a fusion result as the loss value corresponding to the advanced determination strategy. The loss value corresponding to the advanced determination strategy is taken as the fitting loss value of the initial fitted lane. In other application scenarios, when the other determination strategies are at least two of the first advanced strategy, the second advanced strategy, and the third advanced strategy, the loss values determined by the at least two are first added or weightedly fused to obtain a new loss value. The new loss value is added to or weightedly fused with the loss value corresponding to the initial determination strategy to obtain a final loss value, and the final loss value is taken as the loss value corresponding to the advanced determination strategy. The loss value corresponding to the advanced determination strategy is taken as the fitting loss value of the initial fitted lane.

[0134] The manner of constructing the fitting loss value of the initial fitted lane is different for different scene types in which the current frame image is located. For example, according to different types of the initial fitted lane, the process of performing step S13 in the lane line detection method can be to sequentially perform steps as shown in Figure 5 or as shown in Figure 6 .

[0135] In some application scenarios, as shown in Figure 5As shown, steps S51 to S55 are sequentially executed. Step S51: in response to the type of the initial fitted lane line being a single lane line or the first sub-lane line in the initial fitted lane line, the initial determination strategy and the first advanced strategy are taken as the advanced determination strategy. In some application scenarios, the first sub-lane line represents any one of the line segments corresponding to the regions in the non-parallel segment between the left fitted lane line and the right fitted lane line in the case where the type of the initial fitted lane line is a partially parallel double lane line. In other application scenarios, the first sub-lane line represents any one of the left fitted lane line and the right fitted lane line in the initial fitted lane line in the case where the type of the initial fitted lane line is a completely non-parallel double lane line. Step S52: the loss value corresponding to the initial determination strategy and the loss value corresponding to the first advanced strategy are respectively obtained. Step S53: the sum value between the loss value corresponding to the initial determination strategy and the loss value corresponding to the first advanced strategy is determined as the loss value corresponding to the advanced determination strategy. Step S54: the loss value corresponding to the advanced determination strategy is taken as the fitting loss value of the initial fitted lane line. Step S55: the initial fitted lane line is subjected to a convergence process according to the fitting loss value until a target lane line meeting the requirements is obtained. Specifically, the process of determining the fitting loss value of the initial fitted lane line can refer to formula (10):

[0136] Formula (10);

[0137] wherein, is used to represent the fitting loss value of the initial fitted lane line. is used to represent the loss value corresponding to the initial determination strategy, and the calculation process can refer to formula (4) above. is used to represent the loss value corresponding to the advanced determination strategy, and the calculation process can refer to formula (6) and formula (7) above.

[0138] In the above step S55, when the fitting loss value is the smallest, the optimal solution corresponding to the initial lane line coefficient can be represented as formula (11) below:

[0139] Formula (11);

[0140] wherein, the parameters in formula (11) can refer to the descriptions of formula (1), formula (4), formula (6) and formula (7) in the above related formulas, which will not be repeated here. As shown in formula (11), the solving equation of the initial lane line coefficient, when the loss value corresponding to the initial determination strategy is taken as the fitting loss value, the optimal solution corresponding to the initial lane line coefficient as the output of the lane line fitting module when the fitting loss value is the smallest.

[0141] In other application scenarios, as Figure 6As shown, steps S61 to S65 are sequentially executed. Step S61: in response to the type of the initial fitting lane line being a fully parallel double lane line or a second sub-lane line in the initial fitting lane line, the initial determination strategy, the first advanced strategy, the second advanced strategy and the third advanced strategy are taken as the advanced determination strategy. In some application scenarios, the second sub-lane line represents two line segment regions corresponding to the parallel line segment part between the left fitting lane line and the right fitting lane line in the case where the type of the initial fitting lane line is a partially parallel double lane line. Step S62: loss values corresponding to the initial determination strategy, the first advanced strategy, the second advanced strategy and the third advanced strategy are respectively obtained. Step S63: a sum value between the loss values corresponding to the initial determination strategy, the first advanced strategy, the second advanced strategy and the third advanced strategy is determined as a loss value corresponding to the advanced determination strategy. Step S64: the loss value corresponding to the advanced determination strategy is taken as a fitting loss value of the initial fitting lane line. Step S65: the initial fitting lane line is subjected to a convergence process according to the fitting loss value until a target lane line meeting the requirements is obtained.

[0142] As shown, Figure 6 considering that the left fitting lane line and the right fitting lane line are determined, when the left and right lane lines are parallel within a distance D from the vehicle, the loss value corresponding to the initial determination strategy obtained in the lane line fitting module includes a loss value corresponding to the initial determination strategy of the left fitting lane line and a loss value corresponding to the initial determination strategy of the right fitting lane line. The loss value corresponding to the first advanced strategy obtained in the lane line fitting module includes a loss value corresponding to the first advanced strategy of the left fitting lane line and a loss value corresponding to the first advanced strategy of the right fitting lane line. Specifically, the process of determining the fitting loss value of the initial fitting lane line can refer to formula (12), and when the fitting loss value is the smallest in the above step S65, the optimal solution corresponding to the initial lane line coefficient can be expressed as the following formulas (13) to (16):

[0143] Formula (12);

[0144] Formula (13);

[0145] Formula (14);

[0146] Formula (15);

[0147] Formula (16);

[0148] wherein, is used to represent the fitting loss value of the initial fitting lane line. Loss value corresponding to the initial determination strategy for representing the loss value of the initial determination strategy. . Loss value corresponding to the initial determination strategy for representing the left fitted lane line. Loss value corresponding to the initial determination strategy for representing the right fitted lane line. Loss value corresponding to the first advanced strategy. . Wherein, . . . . Total number of point clouds corresponding to the left lane line in the target point cloud information. Total number of point clouds corresponding to the right lane line in the target point cloud information. , Coordinate information of the point cloud corresponding to the left lane line in the target point cloud information, respectively, X-axis corresponding coordinate information and Y-axis corresponding coordinate information. , Coordinate information of the point cloud corresponding to the right lane line in the target point cloud information, respectively, X-axis corresponding coordinate information and Y-axis corresponding coordinate information. Diagonal matrix in the left fitted lane line . Diagonal matrix in the left fitted lane line . Diagonal matrix in the right fitted lane line . , The prior information corresponding to the left lane line coefficient in the left fitted lane line and the prior information corresponding to the right lane line coefficient in the right fitted lane line, respectively. The parameters in formulas (12) to (16) are described in the above related formulas (1), (4), (6), (7), (8) and (9) and will not be repeated here. As shown in formula (13), the initial lane line coefficient solving equation, when the loss value corresponding to the initial determination strategy is taken as the fitting loss value, the optimal solution of the initial lane line coefficient corresponding to the minimum fitting loss value is taken as the output of the lane line fitting module.

[0149] ​​​​In some application scenarios, the lane line fitting module transmits the initial lane line coefficients output by the lane line fitting module to the lane line output module. The initial lane line coefficients correspond to a suboptimal solution. The lane line output module is responsible for transmitting the received initial lane line coefficients to the downstream task. Alternatively, the lane line output module integrates the received initial lane line coefficients with the coordinate information in the target point cloud information to obtain a target lane line, and transmits the target lane line to the downstream task.

[0150] It can be considered that the target point cloud information is obtained by fusing the historical point cloud information and the initial point cloud information, and the subsequent target lane line detection is performed by using the target point cloud information, which can better suppress the overfitting phenomenon when the number of point clouds in the initial point cloud information corresponding to the current frame image is small. In addition, the parameter model of the present application is a cubic polynomial, which is suitable for both straight lane lines and curved lane lines when modeling the initial fitting lane line. It can be considered that the lane line detection method proposed in the present application can improve the stability and consistency of the target lane line in the fitting result in the time dimension. Compared with the fitting result of the lane line obtained by only extracting the initial point cloud information from the current frame image, since the relevant information of the historical frame image is not integrated during fitting, the consistency and stability of the final fitting result in the time dimension is poor. In the lane line modeling process of the present application, the prior information corresponding to the initial lane line coefficient is established according to the linear variation law of the lane line curvature and the historical frame fitting coefficient, and is used as a constraint to improve the stability and consistency of the obtained fitting loss value in the time dimension. In addition, the historical point cloud information is projected into the coordinate system of the current frame image through coordinate transformation and merged with the initial point cloud information to participate in lane line fitting, which can improve the stability and consistency of the obtained fitting loss value in the time dimension. It can be considered that when the type of the initial fitting lane line is related to the parallelism of the lane line, compared with fitting the lane line of the left and right lane lines respectively, the final fitting result cannot guarantee that the left and right lane lines remain in the original parallel state in space. When modeling the lane line, the present application introduces a constraint condition that the lane width within a certain range is equal, and adaptively sets the fitting weight of the width error at different distances to ensure that the obtained fitting loss value remains in the original parallel state in space, thereby improving the accuracy of the obtained fitting loss value. In addition, when modeling the lane line, the present application introduces a constraint that the slope, curvature and curvature change rate of the left and right lane lines are approximately equal, and adaptively adjusts the fitting weight between each other according to the actual situation to ensure that the obtained fitting loss value remains in the original parallel state in space, thereby improving the accuracy of the obtained fitting loss value. Then, the target lane line coefficient obtained by optimizing the fitting loss value can guarantee the parallel state of the lane line. It can be considered that the lane line detection method of the present application can adapt to different lane line scenes, and the present application can determine whether the left and right lane lines are parallel or in what range the left and right lane lines remain in the parallel state through the evaluation result. Furthermore, different loss determination strategies are used to construct the fitting loss value, so that the fitting loss value has high accuracy in different situations.It can be considered that the historical frame lane line point cloud is projected into the current frame coordinate system by coordinate conversion in the present application. When the initial point cloud information detection of the current frame image fails or is not complete enough, the projected lane line point cloud in the current frame coordinate system is directly fitted or the initial point cloud information is completed to complete the lane line detection task and output the target lane line. The lane line detection method of the present application can complete the lane line detection which fails or is not complete for a short time. In addition, the present application can also improve the lane line fitting effect in the intersection scene. There may be a missing phenomenon of lane line in the intersection scene. Compared with using only the current frame image for lane line detection / or fitting, the target lane line coefficient may be determined to be suddenly changed due to the missing of the lane line in the current frame image. The historical point cloud information is projected into the coordinate system of the current frame image by coordinate conversion and fused with the initial point cloud information to obtain target point cloud information. The lane line detection task is completed based on the target point cloud information, which can suppress the sudden change of the target lane line coefficient. It can be considered that the lane line fitting accuracy and robustness of the target lane line obtained by adjusting the initial lane line through attribute information are high. In addition, in the case of double lane lines, the present application introduces the lane line parallel constraint into the lane line modeling process to ensure that the fitted lane line maintains the original parallel state in space and suppresses the overfitting phenomenon. In addition, the present application introduces the lane line curvature linear change constraint into the modeling process of a single lane line to improve the consistency and smoothness of the lane line fitting result (i.e. target lane line coefficient or target lane line) in the time dimension, while suppressing the overfitting phenomenon. In addition, the historical point cloud information corresponding to the historical frame image is introduced as prior information into the lane line modeling process to improve the consistency and smoothness of the lane line fitting result in the time dimension, while suppressing the overfitting phenomenon. In addition, the historical point cloud information is fused in the lane line fitting process of the current frame image to improve the consistency and smoothness of the lane line fitting result in the time dimension, while suppressing the overfitting phenomenon. In addition, each to-be-fitted lane line sample point is adaptively assigned a fitting weight (i.e. fitting weight of the coordinate information of each point in the target point cloud information). The fitting loss value determined based on the fitting weight of the coordinate information of each point in the target point cloud information and the coordinate difference can reduce the influence of abnormal points on the fitting result, improve the accuracy and robustness of the lane line fitting result. In addition, by evaluating the accuracy of the result and the different types of scenarios where the current frame image is located, the fitting loss value of the initial fitted lane line is constructed in different ways. The present application can adaptively select different road geometric characteristics for lane line modeling to obtain the fitting loss value. The present application can improve the application range of the lane line detection method.

[0151] Please refer to Figure 7 , Figure 7is a structural schematic diagram of an embodiment of a lane line detection device of the present application. The lane line detection device 70 comprises an acquisition module 71, a determination module 72, and an adjustment module 73; the acquisition module 71 is configured to acquire target point cloud information of a lane line in a current frame image; the determination module 72 is configured to determine an initial fitting lane line according to the target point cloud information of the lane line; and the adjustment module 73 is configured to perform adjustment processing on the initial fitting lane line according to attribute information of the initial fitting lane line until a target lane line meeting a requirement is obtained.

[0152] Compared with directly using the simulated lane line output by the perception model, the present application determines an initial fitting lane line according to the target point cloud information of the lane line in the current frame image, performs adjustment processing on the initial fitting lane line according to attribute information of the initial fitting lane line until a target lane line meeting a requirement is obtained, and thus the accuracy of the detected target lane line can be improved.

[0153] The functions performed by each module are described in the lane line detection method, which will not be repeated here.

[0154] Please refer to Figure 8 , Figure 8 is a structural schematic diagram of an embodiment of an electronic device of the present application. The electronic device 80 comprises a memory 81 and a processor 82, and the processor 82 is configured to execute program instructions stored in the memory 81 to implement the steps in the above-mentioned embodiment of the lane line detection method. In a specific implementation scenario, the electronic device 80 can include but is not limited to a microcomputer and a server. In addition, the electronic device 80 can also include a notebook computer, a tablet computer, and other mobile devices, which are not limited herein.

[0155] Specifically, the processor 82 is configured to control itself and the memory 81 to implement the steps in the above-mentioned embodiment of the lane line detection method. The processor 82 can also be referred to as a CPU (Central Processing Unit). The processor 82 can be an integrated circuit chip with a signal processing capability. The processor 82 can also be a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. In addition, the processor 82 can be implemented by an integrated circuit chip together.

[0156] Compared to directly using the simulated lane lines output by the perception model, the above-mentioned scheme determines the initial fitted lane line based on the target point cloud information of the lane line in the current frame image, and adjusts the initial fitted lane line according to the attribute information of the initial fitted lane line until the target lane line that meets the requirements is obtained, which can improve the accuracy of the detected target lane line.

[0157] Please see Figure 9 , Figure 9 This is a schematic diagram of a computer-readable storage medium according to an embodiment of the present application. The computer-readable storage medium 90 stores program instructions 901 thereon, which, when executed by a processor, implement the steps in any of the lane line detection method embodiments described above.

[0158] Compared to directly using the simulated lane lines output by the perception model, the above-mentioned scheme determines the initial fitted lane line based on the target point cloud information of the lane line in the current frame image, and adjusts the initial fitted lane line according to the attribute information of the initial fitted lane line until the target lane line that meets the requirements is obtained, which can improve the accuracy of the detected target lane line.

[0159] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0160] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0161] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0162] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0163] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application, essentially or in part, or all or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform all or part of the steps of the methods in the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk, and various other media that can store program codes.

[0164] If the technical solutions of the present application involve personal information, the product applying the technical solutions of the present application has been informed of the personal information processing rules before processing the personal information, and has obtained the personal independent consent. If the technical solutions of the present application involve sensitive personal information, the product applying the technical solutions of the present application has obtained the personal independent consent before processing the sensitive personal information, and at the same time meets the requirement of "explicit consent". For example, at the personal information collection device such as camera, a clear and prominent mark is set to inform that the personal information collection range has been entered, and the personal information will be collected. If the individual voluntarily enters the collection range, it is regarded as agreeing to collect the personal information. Or, on the device for processing personal information, through the pop-up information or by uploading the personal information by the individual, the personal authorization is obtained under the condition that the obvious mark / information informs the personal information processing rules. The personal information processing rules can include the personal information processor, the processing purpose, the processing method, and the type of personal information processed, etc.

Claims

1. A lane line detection method characterized by, The method comprises: acquiring target point cloud information of a lane line in a current frame image; determining an initial fitted lane line according to the target point cloud information of the lane line; adjusting the initial fitted lane line according to attribute information of the initial fitted lane line until a target lane line meeting requirements is obtained; wherein the adjusting of the initial fitted lane line according to the attribute information of the initial fitted lane line until the target lane line meeting requirements is obtained comprises: determining a fitting loss value based on the attribute information and an evaluation result of the attribute information; performing convergence processing on the initial fitted lane line according to the fitting loss value until the target lane line meeting requirements is obtained; the evaluation result represents an evaluation accuracy of the attribute information, and the higher the evaluation accuracy of the attribute information, the higher the credibility of the attribute information; wherein the attribute information comprises a fitting function corresponding to each point in the target point cloud information in the initial fitted lane line; the determining of the fitting loss value based on the attribute information and the evaluation result of the attribute information comprises: in response to the evaluation result representing that the evaluation accuracy is less than a preset accuracy, determining the fitting loss value of the initial fitted lane line according to coordinate information of each point in the target point cloud information and a fitting function value of the fitting function corresponding to the corresponding point in the initial fitted lane line; wherein the initial fitted lane line comprises a fitted left lane line and a fitted right lane line, the fitted left lane line comprises a first point and a second point, the fitted right lane line comprises a third point and a fourth point, the first point and the third point in the target point cloud information have the same value on a preset number axis, and the second point and the fourth point have the same value on a preset number axis; the determining of the fitting loss value based on the attribute information and the evaluation result of the attribute information comprises: in response to the evaluation result representing that the evaluation accuracy is greater than or equal to a preset accuracy, taking a difference between a fitting function value of the fitting function corresponding to the first point in the fitted left lane line and a fitting function value of the fitting function corresponding to the third point in the fitted right lane line as a first distance; taking a difference between a fitting function value of the fitting function corresponding to the second point in the fitted left lane line and a fitting function value of the fitting function corresponding to the fourth point in the fitted right lane line as a second distance; and determining the fitting loss value of the initial fitted lane line based on the first distance and the second distance.

2. The method of claim 1, wherein, The attribute information comprises initial lane line coefficients in the initial fitted lane line; the determining of the fitting loss value based on the attribute information and the evaluation result of the attribute information comprises: in response to the evaluation result representing that the evaluation accuracy is greater than or equal to a preset accuracy, acquiring historical lane line coefficients corresponding to historical lane lines in historical frame images, the acquisition time of the historical frame images being earlier than the acquisition time of the current frame image; determining the fitting loss value of the initial fitted lane line based on the difference between the historical lane line coefficients and the initial lane line coefficients.

3. The method of claim 1, wherein, The initial fitted lane line comprises a fitted left lane line and a fitted right lane line, and the attribute information comprises left lane line coefficients in the fitted left lane line and right lane line coefficients in the fitted right lane line; The determining of the fitting loss value of the initial fitted lane line based on the attribute information and the evaluation result of the attribute information comprises: In response to the evaluation result representing that the evaluation accuracy is greater than or equal to a preset accuracy, the fitting loss value of the initial fitted lane line is determined based on a difference between the left lane line coefficients and the right lane line coefficients.

4. The method of claim 1, wherein, The step of determining the fitting loss value of the initial fitted lane line according to the coordinate information of each point in the target point cloud information and the fitting function value of the corresponding fitting function of the corresponding point in the initial fitted lane line comprises: The difference between the coordinate information of each point in the target point cloud information and the fitting function value of the corresponding fitting function of the corresponding point in the initial fitted lane line is determined as a coordinate difference; The fitting loss value is determined based on the fitting weight of the coordinate information of each point in the target point cloud information and the coordinate difference.

5. The method of claim 1, wherein, The fitted left lane line further comprises a fifth point, and the fitted right lane line comprises a sixth point, the fifth point and the sixth point in the target point cloud information have the same value on a preset number axis, the first point, the second point and the fifth point are adjacent to each other on the fitted left lane line, and the third point, the fourth point and the sixth point are adjacent to each other on the fitted right lane line, and the step of determining the fitting loss value of the initial fitted lane line based on the first distance and the second distance comprises: The difference between the fitting function value of the corresponding fitting function of the fifth point in the fitted left lane line and the fitting function value of the corresponding fitting function of the sixth point in the fitted right lane line is taken as a third distance; The difference between the first distance and the second distance is taken as a first distance difference; The difference between the second distance and the third distance is taken as a second distance difference; The fitting loss value of the initial fitted lane line is determined based on the difference between the first distance difference and the second distance difference.

6. The method of claim 1, wherein, The step of obtaining the target point cloud information of the lane line in the current frame image comprises: Obtaining initial point cloud information corresponding to the current frame image and historical point cloud information corresponding to a historical frame image, the initial point cloud information being obtained by extracting the current frame image; Projecting the historical point cloud information to a coordinate system in which the current frame image is located to obtain new historical point cloud information; Fusing the initial point cloud information and the new historical point cloud information to obtain the target point cloud information.

7. An electronic device, comprising: It comprises: A memory and a processor, wherein the memory stores program instructions, and the processor retrieves the program instructions from the memory to execute the method of any one of claims 1-6.

8. A computer-readable storage medium having stored thereon program instructions, wherein, The program instructions are executed by the processor to implement the method of any one of claims 1-6.

Images