Method, device, electronic device and storage medium for determining virtual lane line

Abstract

The present application relates to a method, device, electronic device and storage medium for determining a virtual lane line, the method comprising: obtaining road perception information in the direction of vehicle travel, and based on the road perception information, identifying the outline of lane edge markers on at least one side of the road; performing a first segmentation process according to the outline of the lane edge marker to obtain a first sampling point and a first segment; sorting and connecting the first segment according to the curvature at the first sampling point to obtain a reconstructed curve; on the reconstructed curve, performing a second segmentation process according to the curvature at the first sampling point to obtain a second sampling point, connecting the second sampling point, and determining a virtual lane line. In this way, the final virtual lane line can be more consistent with the actual direction of the lane, and the lane line ahead can be accurately obtained even in a curved road scenario, providing more accurate reference information for vehicle positioning and path planning.

Description

Technical Field

[0001] The present application relates to the field of intelligent driving technology, and in particular to a method, device, electronic device and storage medium for determining a virtual lane line. Background Art

[0002] During intelligent driving, vehicles need to use sensors such as cameras or radars to sense lane lines in order to achieve vehicle positioning and path planning. However, in some curved road scenarios (such as on and off ramps), the road view in front of the vehicle is easily blocked by lane edge markers on both sides, resulting in the vehicle being unable to perceive the lane lines ahead, which limits the vehicle positioning and path planning performance and affects driving safety. Therefore, how to accurately obtain the lane lines ahead in curved road scenarios has become a technical problem that needs to be solved urgently. Summary of the invention

[0003] The present application provides a method, device, electronic device and storage medium for determining a virtual lane line to solve the problem in the prior art that a vehicle cannot perceive the lane line ahead in some curved road scenarios.

[0004] In a first aspect, the present application provides a method for determining a virtual lane line, the method comprising:

[0005] Acquiring road perception information in the direction of vehicle travel, and identifying the outline of lane edge markers of at least one side of the road based on the road perception information;

[0006] Performing a first segmentation process according to the lane edge marker contour to obtain a first sampling point and a first segment;

[0007] Sort and connect the first segments according to the curvature at the first sampling points to obtain a reconstructed curve;

[0008] On the reconstructed curve, a second segmentation process is performed according to the curvature at the first sampling point to obtain a second sampling point, and the second sampling points are connected to determine a virtual lane line.

[0009] Optionally, connecting the second sampling points to determine a virtual lane line includes:

[0010] Connecting the second sampling points to obtain a lane edge marker curve, wherein the lane edge marker curve is a continuous and smooth curve;

[0011] The virtual lane line is determined according to the lane edge marker curve.

[0012] Optionally, the connecting the second sampling points to obtain a lane edge marker curve includes:

[0013] Performing curve fitting on the connecting lines between two adjacent sampling points in the second sampling points respectively to obtain each fitting curve, wherein the conditions that each fitting curve needs to satisfy include: the equations corresponding to each fitting curve are all cubic equations, the points fitted based on the cubic equations all fall on the corresponding fitting curves, the first-order derivative and the second-order derivative of the cubic equation corresponding to the connecting points between any two adjacent fitting curves are continuous, and the first-order derivative and the second-order derivative of the cubic equation corresponding to the two end points of the lane edge marker contour are both zero;

[0014] The fitting curves are spliced ​​to obtain the lane edge marker curve.

[0015] Optionally, determining the virtual lane line according to the lane edge marker curve includes: identifying a visible lane line based on the road perception information, and determining a lane line splicing starting point from the visible lane line;

[0016] Determine a lane line curve based on the lane edge marker curve, wherein the lane line curve is used to represent the direction of the lane line in the direction of vehicle travel;

[0017] Based on the lane line curve, lane line splicing is performed from the lane line splicing starting point to obtain the virtual lane line.

[0018] Optionally, the lane edge marker curve includes a first guardrail curve on the left side of the road and a second guardrail curve on the right side of the road;

[0019] The determining of a lane line curve based on the lane edge marker curve includes:

[0020] Obtain vehicle steering information;

[0021] When the vehicle turning information indicates that the vehicle is in a right turn state, determining the lane line curve based on the first guardrail curve;

[0022] When the vehicle turning information indicates that the vehicle is in a left turn state, determining the lane line curve based on the second guardrail curve;

[0023] When the vehicle turning information indicates that the vehicle is in a straight-moving state, the lane line curve is determined based on the first guardrail curve and / or the second guardrail curve.

[0024] Optionally, the first segmentation processing method includes:

[0025] The lane edge marker contour is evenly segmented according to a preset interval distance.

[0026] Optionally, the sorting and connecting the first segments according to the curvature at the first sampling points to obtain the reconstructed curve includes:

[0027] sorting the first segments according to the curvature at the first sampling points;

[0028] According to the sorting result, a sampling point with the largest curvature is selected from the first sampling points as a third sampling point, and a first segment between the third sampling point and a target number of sampling points closest to the third sampling point is connected to reconstruct a curve segment, wherein the target number is a preset fixed value or a variable value determined based on a curvature difference with the third sampling point;

[0029] Deleting the third sampling point and a target number of sampling points closest to the third sampling point from the first sampling points to obtain remaining sampling points;

[0030] A sampling point with the largest curvature is selected from the remaining sampling points as a new third sampling point, and a first segment between the new third sampling point and a target number of sampling points closest to the new third sampling point is connected to reconstruct a new curve segment, and the process is repeated until all sampling points in the first sampling point are deleted to obtain the reconstructed curve, wherein the reconstructed curve includes a plurality of curve segments.

[0031] Optionally, the second segment processing method includes:

[0032] Determining a target curvature range according to the maximum curvature and the minimum curvature of each curve segment in the reconstructed curve;

[0033] Dividing the target curvature range into a preset number of range intervals in descending order of curvature, wherein different range intervals correspond to different numbers of segments, and the number of segments corresponding to each range interval is proportional to the curvature of the curve segment corresponding to each range interval;

[0034] According to the number of segments corresponding to each range interval, the curve segments belonging to each range interval are segmented.

[0035] In a second aspect, the present application further provides a device for determining a virtual lane line, the device comprising:

[0036] A first recognition module is used to obtain road perception information in the direction of vehicle travel, and based on the road perception information, recognize the outline of lane edge markers on at least one side of the road;

[0037] A first segmentation processing module, used for performing a first segmentation processing according to the lane edge marker contour to obtain a first sampling point and a first segment;

[0038] a sorting and connecting module, configured to sort and connect the first segments according to the curvature at the first sampling points to obtain a reconstructed curve;

[0039] The second segmentation processing module is used to perform a second segmentation processing on the reconstructed curve according to the curvature at the first sampling point, obtain the second sampling point, connect the second sampling point, and determine the virtual lane line.

[0040] In a third aspect, the present application further provides an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0041] Memory, used to store computer programs;

[0042] The processor is used to implement the method for determining the virtual lane line described in any one of the first aspects when executing the program stored in the memory.

[0043] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method for determining a virtual lane line described in any one of the first aspects is implemented.

[0044] The above technical solution provided by the present application has the following advantages over the prior art: the method provided by the embodiment of the present application obtains the road perception information in the direction of vehicle travel, and identifies the lane edge marker contour of at least one side of the road based on the road perception information; performs a first segmentation process according to the lane edge marker contour to obtain a first sampling point and a first segment; sorts and connects the first segment according to the curvature at the first sampling point to obtain a reconstructed curve; on the reconstructed curve, performs a second segmentation process according to the curvature at the first sampling point to obtain a second sampling point, connects the second sampling point, and determines a virtual lane line. In the above manner, the lane edge marker contour can be first segmented, and then the first segment is reconstructed based on the curvature at the first sampling point, and then the reconstructed curve is second segmented according to the curvature at the first sampling point, so that the final virtual lane line is more in line with the actual direction of the lane, so that the lane line ahead can be accurately obtained even in a curved scene, providing more accurate reference information for vehicle positioning and path planning. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0047] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0048] Figure 1 A flowchart of a method for determining a virtual lane line provided in an embodiment of the present application;

[0049] Figure 2 A schematic diagram of the distribution of a second sampling point provided in an embodiment of the present application;

[0050] Figure 3 A schematic diagram of a lane edge marker curve fitting process provided in an embodiment of the present application;

[0051] Figure 4 A schematic diagram of the structure of a device for determining a virtual lane line provided in an embodiment of the present application;

[0052] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0054] The disclosure below provides many different embodiments or examples to implement different structures of the present invention. In order to simplify the disclosure of the present invention, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present invention. In addition, the present invention can repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed.

[0055] In order to solve the problem in the prior art that the vehicle cannot perceive the lane line ahead in a curved scenario, the present application provides a method, device, electronic device and storage medium for determining a virtual lane line, which can accurately obtain the lane line ahead in a curved scenario.

[0056] See also Figure 1 , Figure 1 A flowchart of a method for determining a virtual lane line provided in an embodiment of the present application is shown below. Figure 1 As shown, the method for determining the virtual lane line may include the following steps:

[0057] Step 101: Acquire road perception information in the direction of vehicle travel, and identify the outline of lane edge markers on at least one side of the road based on the road perception information.

[0058] Specifically, the road perception information may be based on image data perceived by a forward-looking camera, point cloud data perceived by lasers, radars, etc., or data perceived by a fusion of multiple sensors such as forward-looking cameras, lasers, radars, etc., which is not specifically limited in the embodiments of the present application. The lane edge markers may include but are not limited to guardrails, traffic cones (ice cream cones), and other roadblocks or fences arranged according to lane lines. When there are lane edge markers on both sides of the road, the outline of the lane edge marker can be identified based on the lane edge markers on both sides or one side; when there are lane edge markers on only one side of the road, the outline of the lane edge marker can be identified based on the lane edge markers on that side.

[0059] After obtaining the road perception information in the direction of vehicle travel (i.e., in front of the vehicle), the outline of the lane edge marker on at least one side of the road can be identified based on the road perception information. Specifically, if the road perception information is image data, the edge information in the image can be obtained by using an edge detection algorithm, and then the outline of the lane edge marker can be identified according to the outline of each edge information. If the road perception information is point cloud data, the distribution characteristics of the point cloud can be used to identify the outline of the lane edge marker from the point cloud data. If the road perception information is fused data of multiple sensors, the outline of the lane edge marker can be obtained in multiple dimensions using the above method, and verified and improved according to the acquisition results of each sensor. As an optional implementation, the color image obtained by the front-view camera can be used as the road perception information. In this way, when identifying the outline of the lane edge marker, the color image can be gray-processed first to obtain the gray-scale image corresponding to the color image, so as to simplify the processing process. Then, the filtering technology is applied to filter the gray-scale image to obtain a denoised image to eliminate or reduce the interference of noise, while retaining the edge information of the image as much as possible. Then, an edge detection algorithm is used to detect the edge of the denoised image, and the contour of the lane edge marker is identified based on the edge detection result. The edge detection algorithm here can be a first-order derivative operator and a second-order derivative operator. As an optional implementation, a Canny edge detector can be used for edge detection. By calculating the gradient intensity and using techniques such as double threshold processing and non-maximum suppression, the edges in the image can be detected more accurately. It should be noted that after the edge detection is completed, the detection results can also be optimized by post-processing. First, edge refinement is performed, that is, the detected edge width is reduced to a single pixel to improve the clarity of the edge; secondly, edge connection is performed, and the broken edges are connected through a certain strategy to form a continuous contour; then, false edges are removed, that is, those edges that are incorrectly detected due to noise or algorithm errors are removed; finally, morphological operations (such as expansion, corrosion, opening operation, and closing operation) are used to further improve the shape and continuity of the edge.

[0060] Step 102: Perform a first segmentation process according to the lane edge marker contour to obtain a first sampling point and a first segment.

[0061] In this step, after the lane edge marker contour is identified, the first segmentation processing can be performed according to the lane edge marker contour to obtain the first sampling point and the first segment. It should be noted that the first segmentation processing here can be uniform segmentation processing based on the length of the contour according to a preset interval, such as setting a segment every 1 meter, etc.; it can also be non-uniform segmentation processing based on the curvature of the contour, such as setting more segments for contour positions with large curvature, and setting fewer segments for contour positions with large curvature, etc.

[0062] Step 103: Sort and connect the first segments according to the curvature at the first sampling point to obtain a reconstructed curve.

[0063] After obtaining the first sampling point and the first segment, the curvature at the first sampling point can be calculated, and then the first segment can be sorted and connected according to the curvature at the first sampling point to obtain a reconstructed curve. Specifically, the first segments can be sorted and connected in order from high to low according to the curvature at the first sampling point to obtain the reconstructed curve; the first segments can also be sorted and connected in order from low to high according to the curvature at the first sampling point to obtain the reconstructed curve, which is not specifically limited in this application.

[0064] Step 104: On the reconstructed curve, perform a second segmentation process according to the curvature at the first sampling point to obtain a second sampling point, connect the second sampling points, and determine a virtual lane line.

[0065] After obtaining the reconstructed curve, a second segmentation process can be performed on the reconstructed curve according to the curvature at the first sampling point. Specifically, more segments can be set for locations with larger curvature on the reconstructed curve, and fewer segments can be set for locations with smaller curvature. In this way, most of the computing power can be concentrated on locations with larger curvature to improve the accuracy of the lane line ahead in the curved scene. After the second segmentation process, the second sampling point can be obtained, and then the second sampling point can be connected to determine the virtual lane line.

[0066] In this embodiment, the lane edge marker contour can be subjected to a first segmentation process, and then the first segment can be reconstructed based on the curvature at the first sampling point, and then the reconstructed curve can be subjected to a second segmentation process based on the curvature at the first sampling point, so that the final virtual lane line is more consistent with the actual direction of the lane, and thus the lane line ahead can be accurately obtained even in a curved road scenario, providing more accurate reference information for vehicle positioning and path planning.

[0067] Furthermore, the above step 104, connecting the second sampling point to determine the virtual lane line, includes:

[0068] Connect the second sampling points to obtain a lane edge marker curve, wherein the lane edge marker curve is a continuous and smooth curve;

[0069] Determine the virtual lane line based on the lane edge marker curve.

[0070] In one embodiment, when determining the virtual lane line based on the second sampling point, a preset curve fitting algorithm may be used to fit the second sampling point to obtain a continuous and smooth curve. The preset curve fitting algorithm may include but is not limited to a polynomial fitting algorithm, a spline fitting algorithm, a nonlinear fitting algorithm, and a machine learning fitting algorithm. Then, the smooth curve is used as a lane edge marker curve, and then the virtual lane line is accurately determined according to the road direction indicated by the lane edge marker curve. Therefore, even in a curved road scenario, the lane line ahead can be accurately obtained, providing more accurate reference information for vehicle positioning and path planning.

[0071] Furthermore, the above step of connecting the second sampling point to obtain a lane edge marker curve includes:

[0072] Performing curve fitting on the connecting lines between two adjacent sampling points in the second sampling points respectively to obtain each fitting curve, wherein the conditions that each fitting curve needs to meet include: the equations corresponding to each fitting curve are all cubic equations, the points based on the cubic equation fitting all fall on the corresponding fitting curve, the first-order derivative and the second-order derivative of the cubic equation corresponding to the connecting points between any two adjacent fitting curves are continuous, and the first-order derivative and the second-order derivative of the cubic equation corresponding to the two end points of the lane edge marker contour are both zero;

[0073] The fitting curves are spliced ​​to obtain the lane edge marker curve.

[0074] In one embodiment, when obtaining the lane edge marker curve, the line between two adjacent sampling points in the second sampling point can be firstly fitted to obtain each fitting curve. For example, assuming that two adjacent sampling points of the kth segment are recorded as , but The segment can be expressed as ,like Figure 2 shown.

[0075] In the process of performing curve fitting on each of the K segments using a preset curve fitting algorithm, a cubic equation and its derivative equation may be constructed first, wherein the constructed cubic equation is as follows:

[0076] ;

[0077] in, represents the solution value of the kth sampling point, represents the kth sampling point, , , and Represents the coefficient corresponding to the kth segment.

[0078] Taking the first-order and second-order derivatives of the above formula, we can get:

[0079] ;

[0080] ;

[0081] From the above formula, we can see that each small segment has four unknowns. , when the total number of segments is When To perform curve fitting, we need to solve this unknown variable. In solving this When there are unknown numbers, the following conditions need to be met:

[0082] 1. The equations corresponding to the fitting curves of each segment are all cubic equations;

[0083] 2. The points fitted based on the cubic equation all fall on the corresponding segments, that is, ,in, is the actual lane edge marker curve;

[0084] 3. The first-order derivative and the second-order derivative of the cubic equation corresponding to the connection point between any two adjacent segments are continuous;

[0085] 4. The first and second derivatives of the cubic equation corresponding to the two endpoints of the lane edge marker contour are both zero. That is, the constraint conditions for setting the two endpoints of the lane edge marker contour based on the natural boundary are: .

[0086] By solving the equation based on the above conditions, 4K unknowns can be obtained, and finally a complete lane edge marker curve can be obtained. Then, the fitting curves are spliced ​​to obtain the lane edge marker curve.

[0087] In this embodiment, a lane edge marker curve close to a real lane edge marker can be obtained through curve fitting, so as to accurately determine the road direction in the vehicle's driving direction.

[0088] Furthermore, the above step of determining the virtual lane line according to the lane edge marker curve includes:

[0089] Based on the road perception information, the visible lane lines are identified, and the starting point of the lane line splicing is determined from the visible lane lines;

[0090] Based on the lane edge marker curve, a lane line curve is determined, wherein the lane line curve is used to represent the direction of the lane line in the direction of vehicle travel;

[0091] Based on the lane line curve, lane line splicing is performed from the lane line splicing starting point to obtain a virtual lane line.

[0092] In one embodiment, when determining a virtual lane line based on a lane edge marker curve, the visible lane line can be first identified based on the acquired road perception information, and the lane line splicing starting point can be determined from the visible lane line. The lane line splicing starting point can be a lane line vanishing point on the visible lane line, or a point on the visible lane line close to the lane line vanishing point, which is not specifically limited in the embodiments of the present application. Furthermore, based on the lane edge marker curve, the lane line curve is determined, and then based on the lane line curve, lane line splicing is performed from the lane line splicing starting point to obtain a virtual lane line. As an optional implementation,

[0093] In order to ensure the stitching effect, the lane line stitching starting point is selected to be a distance before the lane line vanishing point. Then, according to the lane line splicing starting point, the lane edge marker splicing point is obtained by horizontal projection along the lateral direction of the vehicle, which is set as , and the lateral projection error can be obtained Then, the lane line curve equation is obtained according to the lane edge marker curve equation. Assume that the lane edge marker curve equation is , then the lane curve equation can be Therefore, lane line splicing is performed from the lane line splicing starting point to generate a virtual lane line.

[0094] In this embodiment, a lane line curve is obtained by mapping the lane edge marker curve, and then the lane line is spliced ​​at the lane line splicing starting point through the lane line curve, so as to obtain a virtual lane line. In this way, not only can accurate positioning and path planning information be provided to the vehicle when the lane edge marker blocks the road surface and lane line ahead, thereby improving the environmental perception ability of the automatic driving system, but more accurate prediction and control can also be performed based on the virtual lane line, thereby improving the driving stability and safety, and providing the driver with a more comfortable and reliable driving experience; in addition, it can effectively deal with the problems caused by the occlusion of lane edge markers in special scenes such as curves, so that the vehicle can perform well in various road environments and weather conditions, thereby improving the practicality and reliability of automatic driving technology.

[0095] Furthermore, the lane edge marker curve includes a first guardrail curve on the left side of the road and a second guardrail curve on the right side of the road; the above step of determining the lane line curve based on the lane edge marker curve includes:

[0096] Obtain vehicle steering information;

[0097] When the vehicle steering information indicates that the vehicle is in a right turn state, determining a lane exit curve based on the first guardrail curve;

[0098] When the vehicle steering information indicates that the vehicle is in a left turn state, determining a lane exit curve based on the second guardrail curve;

[0099] When the vehicle turning information indicates that the vehicle is in a straight-moving state, a lane-exit curve is determined based on the first guardrail curve and / or the second guardrail curve.

[0100] In one embodiment, when determining the lane-exit line curve based on the lane edge marker curve, the vehicle steering information can be first obtained, and the vehicle steering state can be determined based on the vehicle steering information. When the vehicle steering information indicates that the vehicle is in a right turn state, the lane-exit line curve is determined based on the first guardrail curve on the left side of the road; when the vehicle steering information indicates that the vehicle is in a left turn state, the lane-exit line curve is determined based on the second guardrail curve on the right side of the road; when the vehicle steering information indicates that the vehicle is in a straight state, the lane-exit line curve is determined based on the first guardrail curve on the left side of the road and / or the second guardrail curve on the right side of the road.

[0101] Through the above method, the first guardrail curve on the left side of the road and / or the second guardrail curve on the right side of the road can be flexibly selected according to the vehicle turning state to determine the lane line curve, thereby improving the flexibility of lane line determination and expanding the usage scenarios.

[0102] Furthermore, the first segment processing method includes:

[0103] The lane edge marker contour is evenly segmented according to the preset interval distance.

[0104] In one embodiment, when performing the first segmentation processing according to the lane edge marker contour, the uniform segmentation processing is performed according to the preset interval based on the length of the contour, such as setting a segment every 1 meter. In this way, based on the first segmentation processing, uniformly distributed first sampling points can be obtained, which facilitates the subsequent sorting and connection of the first segments according to the curvature at the first sampling points to obtain the reconstructed curve.

[0105] Furthermore, the above step 103, sorting and connecting the first segments according to the curvature at the first sampling point to obtain the reconstructed curve, includes:

[0106] sorting the first segments according to the curvature at the first sampling point;

[0107] According to the sorting result, a sampling point with the largest curvature is selected from the first sampling points as the third sampling point, and the first segment between the third sampling point and a target number of sampling points closest to the third sampling point is connected to reconstruct a curve segment, wherein the target number is a preset fixed value or a variable value determined based on a curvature difference with the third sampling point;

[0108] The third sampling point and a target number of sampling points closest to the third sampling point are deleted from the first sampling point to obtain remaining sampling points;

[0109] A sampling point with the largest curvature is selected from the remaining sampling points as a new third sampling point, and the first segment between the new third sampling point and a target number of sampling points closest to the new third sampling point is connected to reconstruct a new curve segment, and the process is repeated until all sampling points in the first sampling point are deleted to obtain a reconstructed curve, wherein the reconstructed curve includes multiple curve segments.

[0110] In one embodiment, when the first segments are sorted and connected to obtain a reconstructed curve, the first segments may be sorted according to the curvature at the first sampling point, and then according to the sorting result, the sampling point with the largest curvature is selected from the first sampling point as the third sampling point, and the first segment between the third sampling point and the target number of sampling points closest to the third sampling point is connected to reconstruct a curve segment. Then, the third sampling point and the target number of sampling points closest to the third sampling point are deleted from the first sampling point to obtain the remaining sampling points, and then the sampling point with the largest curvature is selected from the remaining sampling points as a new third sampling point, and the first segment between the new third sampling point and the target number of sampling points closest to the new third sampling point is connected to reconstruct a new curve segment, and the cycle is repeated until all sampling points in the first sampling point are deleted to obtain a reconstructed curve. It should be noted that the target number here can be a preset fixed value, or a variable value determined based on the curvature difference with the third sampling point.

[0111] When the target number is a preset fixed value, the first segments between the sampling points of the number can be directly connected to reconstruct a curve segment. For example, when the target number is 3, the first segment between the third sampling point and the sampling point adjacent to its left and the first segment between the third sampling point and the sampling point adjacent to its right are directly connected to form a first curve segment, and then the three sampling points are deleted, and a new third sampling point with the largest curvature is selected from the remaining sampling points. Then, the first segment between the new third sampling point and the sampling point adjacent to its left and the first segment between the new third sampling point and the sampling point adjacent to its right are directly connected to form a second curve segment. The above operation is performed on all sampling points in descending order according to the curvature until all sampling points in the first sampling point are deleted. It should be noted that, when a third sampling point is included in a previously reconstructed curve, the third sampling point is skipped and the operation is performed on the next sampling point; if one of the two sampling points adjacent to the third sampling point is included in the previously reconstructed curve, the third sampling point and the other sampling point not included form a curve segment; if the third sampling point is adjacent to only one point, the two sampling points form a curve segment.

[0112] When the target number is a change value determined based on the curvature difference with the third sampling point, the curvature of the two sampling points adjacent to the third sampling point can be first subtracted from the curvature of the third sampling point to determine whether the difference is greater than the preset threshold. If the difference is less than the preset threshold, the second sampling point before the third sampling point and the second sampling point after the third sampling point are compared, and the cycle is repeated until the difference is greater than the preset threshold, and all the sampling points that have been cycled are combined to form the first curve segment. Then the sampling points that have been cycled are deleted, and the sampling point with the largest curvature is selected from the remaining sampling points as the new third sampling point. The above operation is performed again to obtain the second curve segment, and the cycle is repeated until all the sampling points are deleted. It should be noted that if the sampling points in a certain cycle are included in the previously reconstructed curve, the sampling points that are not included in the sampling points in the cycle are combined into a curve segment.

[0113] In this embodiment, the first segments may be sorted and connected according to the sorting result of the curvature at the first sampling point to obtain a reconstructed curve, so as to divide the sampling points with different curvatures into different curve segments.

[0114] Furthermore, the second segment processing method includes:

[0115] Determine the target curvature range according to the maximum curvature and the minimum curvature of each curve segment in the reconstructed curve;

[0116] Divide the target curvature range into a preset number of range intervals in descending order of curvature, wherein different range intervals correspond to different numbers of segments, and the number of segments corresponding to each range interval is proportional to the curvature of the curve segment corresponding to each range interval;

[0117] According to the number of segments corresponding to each range interval, the curve segments belonging to each range interval are segmented.

[0118] Specifically, the above preset number can be set according to actual needs, such as 4, 5, 6, etc.

[0119] In one embodiment, when performing the second segmentation processing according to the curvature at the first sampling point, the target curvature range can be determined according to the maximum curvature value and the minimum curvature value of each curve segment in the reconstructed curve, and then the target curvature range is divided into a preset number of range intervals in the order of curvature from high to low, and then the curve segments belonging to each range interval are segmented according to the number of segments corresponding to each range interval. For example, the target curvature range can be divided into 5 range intervals in the order of curvature from high to low, and the number of segments corresponding to these 5 range intervals decreases in sequence, and the number of segments corresponding to these 5 range intervals can be 5, 4, 3, 2, and 1 respectively.

[0120] In this embodiment, secondary segmentation is performed by reconstructing the curvature of each curve segment in the curve, so that the curve segment with larger curvature has more segments, and the curve segment with smaller curvature has fewer segments, so as to improve the fitting accuracy of the large curvature curve and better cope with the curvature change.

[0121] In one embodiment, when determining the virtual lane line, the road perception information in the direction of vehicle travel may be first obtained, and based on the road perception information, the outline of the lane edge marker may be identified, and then a first segmentation process may be performed according to the outline of the lane edge marker to obtain a first sampling point and a first segment, and then the curvature at the first sampling point may be calculated, and the first segments may be sorted and connected according to the curvature at the first sampling point to obtain a reconstructed curve, and on the reconstructed curve, a second segmentation process may be performed according to the curvature at the first sampling point to obtain a second sampling point, and the second sampling points may be connected to obtain a virtual lane line, such as Figure 3 shown.

[0122] Compared with the prior art, the method for determining a virtual lane line provided in the embodiment of the present application has the following beneficial effects:

[0123] 1. The front-view camera identifies the guardrails on both sides of the road, fits the curves, and maps the virtual lane lines. This can provide accurate positioning and path planning information for the vehicle even when the guardrails block the road and lane lines ahead, thereby improving the environmental perception capabilities of the autonomous driving system.

[0124] 2. By providing accurate virtual lane line information, the experience of intelligent driving functions can be effectively improved. The vehicle can make more accurate predictions and controls based on this information, thereby improving driving stability and safety, and providing drivers with a more comfortable and reliable driving experience.

[0125] 3. This method is optimized for special scenarios such as curves, and can effectively deal with problems caused by guardrail occlusion in these scenarios. This enhanced adaptability enables the vehicle to perform well in various road environments and weather conditions, thereby improving the practicality and reliability of autonomous driving technology.

[0126] 4. This solution can be applied to intelligent driving systems equipped with forward-looking cameras, and can be extended to intelligent driving systems with roof-mounted laser radars. For intelligent driving systems equipped with roof-mounted laser radars, laser radars can be used to identify road guardrails, and "vision-laser radar" fusion can also be used to identify guardrails. After identifying the guardrails, according to this solution, virtual lane lines are obtained through curve fitting of lane edge markers and lane line splicing and projection.

[0127] See also Figure 4 , Figure 4 A schematic diagram of a virtual lane line determination device provided in an embodiment of the present application. Figure 4 As shown, the virtual lane line determination device 400 includes:

[0128] The first recognition module 401 is used to obtain road perception information in the direction of vehicle travel, and recognize the outline of lane edge markers of at least one side of the road based on the road perception information;

[0129] A first segmentation processing module 402, configured to perform a first segmentation processing according to the lane edge marker contour to obtain a first sampling point and a first segment;

[0130] A sorting and connecting module 403, configured to sort and connect the first segments according to the curvature at the first sampling point to obtain a reconstructed curve;

[0131] The second segmentation processing module 404 is used to perform a second segmentation processing on the reconstructed curve according to the curvature at the first sampling point, obtain the second sampling point, connect the second sampling point, and determine the virtual lane line.

[0132] Optionally, the second segment processing module 404 includes:

[0133] A connection submodule, used to connect the second sampling points to obtain a lane edge marker curve, wherein the lane edge marker curve is a continuous and smooth curve;

[0134] The determination submodule is used to determine the virtual lane line according to the lane edge marker curve.

[0135] Optionally, the connection submodule includes:

[0136] A fitting unit is used to perform curve fitting on the connecting line between two adjacent sampling points in the second sampling points to obtain each fitting curve, wherein the conditions that each fitting curve needs to meet include: the equations corresponding to each fitting curve are all cubic equations, the points based on the cubic equation fitting all fall on the corresponding fitting curve, the first-order derivative and the second-order derivative of the cubic equation corresponding to the connecting point between any two adjacent fitting curves are continuous, and the first-order derivative and the second-order derivative of the cubic equation corresponding to the two end points of the lane edge marker contour are both zero;

[0137] The first splicing unit is used to splice the fitting curves to obtain a lane edge marker curve.

[0138] Optionally, determining the submodule includes:

[0139] A recognition unit, used to recognize visible lane lines based on road perception information, and determine the lane line splicing starting point from the visible lane lines;

[0140] A first determining unit is used to determine a lane line curve based on a lane edge marker curve, wherein the lane line curve is used to represent the direction of the lane line in the direction of vehicle travel;

[0141] The second splicing unit is used to perform lane line splicing from a lane line splicing starting point based on the lane line curve to obtain a virtual lane line.

[0142] Optionally, the lane edge marker curve includes a first guardrail curve on the left side of the road and a second guardrail curve on the right side of the road; and the determining submodule further includes:

[0143] An acquisition unit, used for acquiring vehicle steering information;

[0144] A second determination unit is used to determine a lane exit curve based on the first guardrail curve when the vehicle steering information indicates that the vehicle is in a right turn state;

[0145] A third determining unit is used to determine a lane exit curve based on the second guardrail curve when the vehicle steering information indicates that the vehicle is in a left turn state;

[0146] The fourth determination unit is used to determine a lane exit curve based on the first guardrail curve and / or the second guardrail curve when the vehicle turning information indicates that the vehicle is in a straight-moving state.

[0147] Optionally, the first segment processing method includes:

[0148] The lane edge marker contour is evenly segmented according to the preset interval distance.

[0149] Optionally, the sorting and connecting module 403 includes:

[0150] a sorting submodule, for sorting the first segments according to the curvature at the first sampling point;

[0151] According to the sorting result, a sampling point with the largest curvature is selected from the first sampling points as the third sampling point, and the first segment between the third sampling point and a target number of sampling points closest to the third sampling point is connected to reconstruct a curve segment, wherein the target number is a preset fixed value or a variable value determined based on a curvature difference with the third sampling point;

[0152] A deletion submodule, used for deleting the third sampling point and a target number of sampling points closest to the third sampling point from the first sampling point to obtain remaining sampling points;

[0153] The loop submodule is used to select a sampling point with the largest curvature from the remaining sampling points as a new third sampling point, and connect the first segment between the new third sampling point and the target number of sampling points closest to the new third sampling point to reconstruct a new curve segment, and loop in sequence until all sampling points in the first sampling point are deleted to obtain a reconstructed curve, wherein the reconstructed curve includes multiple curve segments.

[0154] Optionally, the second segment processing method includes:

[0155] Determine the target curvature range according to the maximum curvature and the minimum curvature of each curve segment in the reconstructed curve;

[0156] Divide the target curvature range into a preset number of range intervals in descending order of curvature, wherein different range intervals correspond to different numbers of segments, and the number of segments corresponding to each range interval is proportional to the curvature of the curve segment corresponding to each range interval;

[0157] According to the number of segments corresponding to each range interval, the curve segments belonging to each range interval are segmented.

[0158] It should be noted that the virtual lane line determination device 400 can implement the virtual lane line determination method provided by any of the aforementioned method embodiments and can achieve the same technical effect, which will not be described one by one here.

[0159] like Figure 5 As shown, the embodiment of the present application further provides an electronic device, including a processor 511, a communication interface 512, a memory 513 and a communication bus 514, wherein the processor 511, the communication interface 512, and the memory 513 communicate with each other through the communication bus 514.

[0160] Memory 513, used for storing computer programs;

[0161] In one embodiment of the present application, the processor 511 is used to implement the method for determining the virtual lane line provided by any one of the aforementioned method embodiments when executing the program stored in the memory 513.

[0162] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for determining a virtual lane line provided by any one of the aforementioned method embodiments is implemented.

[0163] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0164] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution, in essence, or the part that contributes to the relevant technology, can be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiment.

[0165] It should be understood that the terms used herein are only for the purpose of describing specific example embodiments and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "said" as used herein may also be meant to include plural forms. The terms "include", "comprise", "contain", and "have" are inclusive, and therefore specify the existence of stated features, steps, operations, elements and / or parts, but do not exclude the existence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The method steps, processes, and operations described herein are not interpreted as necessarily requiring them to be performed in the specific order described or illustrated, unless the execution order is clearly indicated. It should also be understood that additional or alternative steps may be used.

[0166] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for determining a virtual lane line, characterized in that: The method comprises: Acquiring road perception information in the direction of vehicle travel, and identifying the outline of lane edge markers of at least one side of the road based on the road perception information; Performing a first segmentation process according to the lane edge marker contour to obtain a first sampling point and a first segment; Sort and connect the first segments according to the curvature at the first sampling points to obtain a reconstructed curve; On the reconstructed curve, performing a second segmentation process according to the curvature at the first sampling point to obtain a second sampling point, and connecting the second sampling points to determine a virtual lane line; The step of sorting and connecting the first segments according to the curvature at the first sampling points to obtain a reconstructed curve includes: sorting the first segments according to the curvature at the first sampling points; According to the sorting result, a sampling point with the largest curvature is selected from the first sampling points as a third sampling point, and a first segment between the third sampling point and a target number of sampling points closest to the third sampling point is connected to reconstruct a curve segment, wherein the target number is a preset fixed value; Deleting the third sampling point and a target number of sampling points closest to the third sampling point from the first sampling points to obtain remaining sampling points; A sampling point with the largest curvature is selected from the remaining sampling points as a new third sampling point, and a first segment between the new third sampling point and a target number of sampling points closest to the new third sampling point is connected to reconstruct a new curve segment, and the process is repeated until all sampling points in the first sampling point are deleted to obtain the reconstructed curve, wherein the reconstructed curve includes a plurality of curve segments.

2. The method for determining a virtual lane line according to claim 1, characterized in that: The connecting the second sampling points to determine a virtual lane line includes: Connecting the second sampling points to obtain a lane edge marker curve, wherein the lane edge marker curve is a continuous and smooth curve; The virtual lane line is determined according to the lane edge marker curve.

3. The method for determining a virtual lane line according to claim 2, characterized in that: The connecting the second sampling points to obtain a lane edge marker curve includes: Performing curve fitting on the connecting lines between two adjacent sampling points in the second sampling points respectively to obtain each fitting curve, wherein the conditions that each fitting curve needs to satisfy include: the equations corresponding to each fitting curve are all cubic equations, the points fitted based on the cubic equations all fall on the corresponding fitting curves, the first-order derivative and the second-order derivative of the cubic equation corresponding to the connecting points between any two adjacent fitting curves are continuous, and the first-order derivative and the second-order derivative of the cubic equation corresponding to the two end points of the lane edge marker contour are both zero; The fitting curves are spliced ​​to obtain the lane edge marker curve.

4. The method for determining a virtual lane line according to claim 2, characterized in that: Determining the virtual lane line according to the lane edge marker curve includes: identifying a visible lane line based on the road perception information, and determining a lane line splicing starting point from the visible lane line; Determine a lane line curve based on the lane edge marker curve, wherein the lane line curve is used to represent the direction of the lane line in the direction of vehicle travel; Based on the lane line curve, lane line splicing is performed from the lane line splicing starting point to obtain the virtual lane line.

5. The method for determining a virtual lane line according to claim 4, characterized in that: The lane edge marker curve includes a first guardrail curve on the left side of the road and a second guardrail curve on the right side of the road; The determining of a lane line curve based on the lane edge marker curve includes: Obtain vehicle steering information; When the vehicle turning information indicates that the vehicle is in a right turn state, determining the lane line curve based on the first guardrail curve; When the vehicle turning information indicates that the vehicle is in a left turn state, determining the lane line curve based on the second guardrail curve; When the vehicle turning information indicates that the vehicle is in a straight-moving state, the lane line curve is determined based on the first guardrail curve and / or the second guardrail curve.

6. The method for determining a virtual lane line according to any one of claims 1 to 5, characterized in that: The first segmentation processing method includes: The lane edge marker contour is evenly segmented according to a preset interval distance.

7. The method for determining a virtual lane line according to any one of claims 1 to 5, characterized in that: The step of sorting and connecting the first segments according to the curvature at the first sampling points to obtain a reconstructed curve further includes: sorting the first segments according to the curvature at the first sampling points; According to the sorting result, a sampling point with the largest curvature is selected from the first sampling points as a third sampling point, and a first segment between the third sampling point and a target number of sampling points closest to the third sampling point is connected to reconstruct a curve segment, wherein the target number is a change value determined based on a curvature difference with the third sampling point; Deleting the third sampling point and a target number of sampling points closest to the third sampling point from the first sampling points to obtain remaining sampling points; A sampling point with the largest curvature is selected from the remaining sampling points as a new third sampling point, and a first segment between the new third sampling point and a target number of sampling points closest to the new third sampling point is connected to reconstruct a new curve segment, and the process is repeated until all sampling points in the first sampling point are deleted to obtain the reconstructed curve, wherein the reconstructed curve includes a plurality of curve segments.

8. The method for determining a virtual lane line according to any one of claims 1 to 5, characterized in that: The second segment processing method includes: Determining a target curvature range according to the maximum curvature and the minimum curvature of each curve segment in the reconstructed curve; Dividing the target curvature range into a preset number of range intervals in descending order of curvature, wherein different range intervals correspond to different numbers of segments, and the number of segments corresponding to each range interval is proportional to the curvature of the curve segment corresponding to each range interval; According to the number of segments corresponding to each range interval, the curve segments belonging to each range interval are segmented.

9. A device for determining a virtual lane line, characterized in that: The device comprises: A first recognition module is used to obtain road perception information in the direction of vehicle travel, and based on the road perception information, recognize the outline of lane edge markers on at least one side of the road; A first segmentation processing module, used for performing a first segmentation processing according to the lane edge marker contour to obtain a first sampling point and a first segment; a sorting and connecting module, configured to sort and connect the first segments according to the curvature at the first sampling points to obtain a reconstructed curve; A second segmentation processing module is used to perform a second segmentation processing on the reconstructed curve according to the curvature at the first sampling point, obtain a second sampling point, connect the second sampling point, and determine a virtual lane line; Wherein, the sorting and connection module includes: a sorting submodule, configured to sort the first segments according to the curvature at the first sampling points; a reconstruction submodule, configured to select, according to the sorting result, a sampling point with the largest curvature from the first sampling points as a third sampling point, and connect the first segment between the third sampling point and a target number of sampling points closest to the third sampling point to reconstruct a curve segment, wherein the target number is a preset fixed value; a deletion submodule, configured to delete the third sampling point and a target number of sampling points closest to the third sampling point from the first sampling points to obtain remaining sampling points; A loop submodule is used to select a sampling point with the largest curvature from the remaining sampling points as a new third sampling point, and connect the first segment between the new third sampling point and the target number of sampling points closest to the new third sampling point to reconstruct a new curve segment, and loop in sequence until all sampling points in the first sampling point are deleted to obtain the reconstructed curve, wherein the reconstructed curve includes multiple curve segments.

10. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; The processor is used to implement the method for determining the virtual lane line according to any one of claims 1 to 8 when executing the program stored in the memory.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for determining a virtual lane line according to any one of claims 1 to 8 is implemented.

Images