Virtual lane line generation method and device, vehicle and storage medium

Abstract

The invention relates to a virtual lane line generation method and device, a vehicle and a storage medium. The method comprises the steps of obtaining a road image and vehicle positioning information; determining a lane end point position and an intersection end point position from the road image based on the vehicle positioning information; extending the lane end point position according to the vehicle positioning information to generate a lane end point extension line; extending the intersection endpoint position according to the vehicle positioning information to generate an intersection endpoint extension line; determining a fitting rule according to a geometrical relationship between the lane end point extension line and the intersection end point extension line; and based on the fitting rule, fitting is carried out on the lane end point position and the intersection end point position, and a virtual lane line is generated. According to the method, the virtual lane line can be determined by identifying the image information, so that the virtual lane line can be accurately generated in a scene without lane mark guidance, such as a high-precision map or an intersection, and the driving control stability of a vehicle can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle assisted driving, and in particular to a virtual lane line generation method, a virtual lane line generation device, a vehicle and a computer-readable storage medium. Background Art

[0002] In the field of autonomous driving, lane line attributes are an important part of path positioning and planning control. Lane line attributes can provide lateral positioning constraints for vehicles, and can also be used as a strategy for centerline generation in planning control. However, current lane line attributes are based on high-precision maps, or are obtained by semantic extraction based on data with lane lines, such as images with lane lines. For scenes without high-precision maps or intersections of road driving, it is difficult to generate virtual lane lines, resulting in poor stability of autonomous driving. Summary of the invention

[0003] One of the purposes of the present invention is to provide a method for generating a virtual lane line to solve the problem that it is difficult to generate virtual lane lines in the prior art, resulting in poor stability of autonomous driving; the second purpose is to provide a virtual lane line generating device; the third purpose is to provide a vehicle; and the fourth purpose is to provide a computer-readable storage medium.

[0004] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0005] A method for generating a virtual lane line, comprising:

[0006] Acquire road images and vehicle positioning information;

[0007] Based on the vehicle positioning information, determining a lane endpoint position and an intersection endpoint position from the road image;

[0008] According to the vehicle positioning information, the lane endpoint position is extended to generate a lane endpoint extension line;

[0009] According to the vehicle positioning information, the intersection endpoint position is extended to generate an intersection endpoint extension line;

[0010] Determining a fitting rule according to a geometric relationship between an extension line of the lane endpoint and an extension line of the intersection endpoint;

[0011] Based on the fitting rule, the lane endpoint position and the intersection endpoint position are fitted to generate a virtual lane line.

[0012] Optionally, the step of determining the fitting rule according to the geometric relationship between the lane endpoint extension line and the intersection endpoint extension line comprises:

[0013] Performing vector multiplication of the direction vector of the extension line of the lane endpoint and the direction vector of the extension line of the intersection endpoint to determine a geometric relationship;

[0014] In response to the geometric relationship being a quasi-vertical relationship, determining the fitting rule to be arc fitting;

[0015] In response to the geometric relationship being a quasi-parallel relationship, determining an angle between an extension line of the lane endpoint and an extension line of the route intersection endpoint;

[0016] A fitting rule is determined based on the angle.

[0017] Optionally, the step of determining a fitting rule based on the angle includes:

[0018] Determining whether the angle is greater than a preset threshold;

[0019] In response to the angle being greater than the preset threshold, determining that the fitting rule is Bessel fitting;

[0020] In response to the angle being not greater than the preset threshold, determining that the fitting rule is straight line fitting.

[0021] Optionally, when the fitting rule is Bessel fitting, the step of performing fitting based on the fitting rule for the lane endpoint position and the intersection endpoint position to generate a virtual lane line includes:

[0022] Determining a Bezier curve control point according to the lane endpoint position and the intersection endpoint position;

[0023] Bezier fitting is performed based on the Bezier curve control points, the lane endpoint positions and the intersection endpoint positions to generate a virtual lane line.

[0024] Optionally, the step of determining a Bezier curve control point according to the lane endpoint position and the intersection endpoint position includes:

[0025] The point that divides the distance between the lane endpoint position and the intersection endpoint position into three equal parts is determined as the Bezier curve control point.

[0026] Optionally, when the fitting rule is straight line fitting, the step of performing fitting based on the fitting rule for the lane endpoint position and the intersection endpoint position to generate a virtual lane line includes:

[0027] The lane endpoint position and the intersection endpoint position are linearly fitted to generate a virtual lane line.

[0028] Optionally, when the fitting rule is arc fitting, the step of performing fitting based on the fitting rule for the lane endpoint position and the intersection endpoint position to generate a virtual lane line includes:

[0029] Based on the arc fitting, determining the lane endpoint position and the intersection endpoint position as circle tangent points;

[0030] Perform circumscribed circle fitting based on the circle tangent points to generate a circumscribed circle;

[0031] In the circumscribed circle, the arc between the circle tangent points is determined as a virtual lane line.

[0032] Optionally, the step of determining a lane endpoint position and an intersection endpoint position from the road image based on the vehicle positioning information includes:

[0033] Based on the vehicle positioning information, determining the current lane in the road image;

[0034] Determining a lane endpoint position in the current lane;

[0035] Performing image recognition on the road image to determine an intersection area;

[0036] Lane segmentation is performed in the intersection area to determine the intersection endpoint position.

[0037] Optionally, the step of extending the lane endpoint position according to the vehicle positioning information to generate a lane endpoint extension line includes:

[0038] Determining the direction of the intersection according to the vehicle positioning information;

[0039] Extending the lane endpoint position by a preset distance along the direction of the intersection to generate a lane update position;

[0040] The lane endpoint position is connected to the lane update position to generate a lane endpoint extension line.

[0041] Optionally, the step of extending the intersection endpoint position according to the vehicle positioning information to generate an intersection endpoint extension line includes:

[0042] Determining the direction of the intersection according to the vehicle positioning information;

[0043] Along the direction of the intersection, extending the intersection endpoint position by a preset distance to generate an intersection update position;

[0044] The intersection endpoint position is connected with the intersection update position to generate an intersection endpoint extension line.

[0045] A virtual lane line generating device, comprising:

[0046] An acquisition module, used to acquire road images and vehicle positioning information;

[0047] A position recognition module, used to determine the lane endpoint position and the intersection endpoint position from the road image based on the vehicle positioning information;

[0048] A first extension module, used to extend the lane endpoint position according to the vehicle positioning information to generate a lane endpoint extension line;

[0049] A second extension module is used to extend the intersection endpoint position according to the vehicle positioning information to generate an intersection endpoint extension line;

[0050] A fitting rule determination module, used to determine a fitting rule according to a geometric relationship between an extension line of the lane endpoint and an extension line of the intersection endpoint;

[0051] The fitting module is used to fit the lane endpoint position and the intersection endpoint position based on the fitting rule to generate a virtual lane line.

[0052] A vehicle comprises a processor, a memory and a computer program stored in the memory and capable of running on the processor, wherein the computer program implements the steps of the virtual lane line generating method as described above when executed by the processor.

[0053] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the vehicle steering control method described above are implemented.

[0054] Beneficial effects of the present invention:

[0055] (1) The present invention can determine virtual lane lines by recognizing image information, so that virtual lane lines can be generated in scenarios without high-precision maps or without lane markings such as intersections, and does not rely on neural network models, which is beneficial to improving the driving control stability of the vehicle and reducing the dependence on vehicle computing power;

[0056] (2) The present invention determines the corresponding fitting rules based on the geometric relationship between different lane endpoint extension lines and intersection endpoint extension lines; different fitting rules are used to fit the virtual lane lines, so that the virtual lane lines can be accurately generated in different scenarios, thereby generating the virtual lane lines with high accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 A flowchart of a method for generating a virtual lane line according to an embodiment of the present invention;

[0058] Figure 2 A flowchart of another embodiment of a method for generating a virtual lane line according to the present invention;

[0059] Figure 3A schematic diagram of the lane endpoint position and the intersection endpoint position of the present invention;

[0060] Figure 4a A schematic diagram of a virtual lane line of Bezier fitting according to the present invention;

[0061] Figure 4b It is a Bessel fitting schematic diagram of the present invention;

[0062] Figure 5a A schematic diagram of a virtual lane line of arc fitting according to the present invention;

[0063] Figure 5b It is a schematic diagram of arc fitting of the present invention;

[0064] Figure 6 A flowchart of an example of a method for generating a virtual lane line according to the present invention;

[0065] Figure 7 A structural block diagram of an embodiment of a virtual lane line generating device of the present invention;

[0066] Figure 8 A schematic diagram of a processor and a storage medium of a vehicle embodiment of the present invention;

[0067] Fig. 9 A schematic diagram of an embodiment of a computer-readable storage medium of the present invention. DETAILED DESCRIPTION

[0068] The following will describe the embodiments of the present invention with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention, not for limiting the scope of protection of the present invention.

[0069] Reference Figure 1 , shows a flowchart of a method for generating a virtual lane line according to an embodiment of the present invention. The method for generating a virtual lane line may specifically include the following steps:

[0070] Step 101, obtaining road images and vehicle positioning information;

[0071] In the embodiment of the present invention, the image of the road can be collected by a vehicle-mounted visual sensor such as a forward-looking, surround-looking or periscopic sensor. The vehicle positioning information is provided by a navigation system. The road image and the vehicle positioning information can be obtained as raw data for processing.

[0072] Step 102, based on the vehicle positioning information, determining a lane endpoint position and an intersection endpoint position from the road image;

[0073] Based on the vehicle positioning information, the vehicle's position, driving status, etc. are determined, and the lane endpoint position and intersection endpoint position are determined from the road image. Among them, the lane endpoint position is the centerline endpoint of the lane where the vehicle is currently located or the position of the center point of the vehicle in the lane. The intersection endpoint position is the centerline endpoint of the lane of the next intersection that the vehicle is about to drive to. There can be multiple intersection endpoint positions, that is, the same processing can be performed on multiple intersection endpoint positions at the same time to generate virtual lane lines between lane endpoint positions and different intersection endpoint positions.

[0074] Step 103, extending the lane endpoint position according to the vehicle positioning information to generate a lane endpoint extension line;

[0075] Based on the vehicle positioning information, the lane endpoint position is extended to form a virtual line segment, which is the lane endpoint extension line.

[0076] Step 104, extending the intersection endpoint position according to the vehicle positioning information to generate an intersection endpoint extension line;

[0077] Similarly, based on the vehicle positioning information, the endpoint position of the intersection is extended to form a virtual line segment, which is the intersection endpoint extension line.

[0078] Step 105, determining a fitting rule according to the geometric relationship between the lane endpoint extension line and the intersection endpoint extension line;

[0079] The geometric relationship between the lane endpoint extension line and the intersection endpoint extension line is determined, and different driving modes of the lane of the vehicle traveling to the next intersection are determined based on the geometric relationship, and then different fitting rules are determined.

[0080] Step 106: Based on the fitting rule, fit the lane endpoint position and the intersection endpoint position to generate a virtual lane line.

[0081] Based on the fitting rules, the lane endpoint is used as the starting point and the intersection endpoint is used as the end point to perform line fitting and generate a virtual lane line. The virtual lane line guides the vehicle from the current lane to the lane of the next intersection.

[0082] The embodiment of the present invention obtains a road image and vehicle positioning information; determines the lane endpoint position and the intersection endpoint position from the road image based on the vehicle positioning information; extends the lane endpoint position according to the vehicle positioning information to generate a lane endpoint extension line; extends the intersection endpoint position according to the vehicle positioning information to generate an intersection endpoint extension line; determines a fitting rule based on the geometric relationship between the lane endpoint extension line and the intersection endpoint extension line; and performs fitting on the lane endpoint position and the intersection endpoint position based on the fitting rule to generate a virtual lane line. The virtual lane line can be determined by identifying the image information, so that the virtual lane line can be generated in scenarios without high-precision maps or intersections without lane marking guidance, which is beneficial to improving the driving control stability of the vehicle.

[0083] Reference Figure 2 , shows a flowchart of another embodiment of a virtual lane line generation method of the present invention. The virtual lane line generation method may specifically include the following steps:

[0084] Step 201, calibrating the sensor;

[0085] Calibration can be performed on the visual image sensor, position sensor, etc. on the vehicle to ensure that the collected data meets the usage requirements. For example, after the visual image sensor is calibrated, the output of the lane line reprojection error must meet the actual corresponding size within 0.1 to 0.2 lane widths, and the external parameter stitching error of each camera must not exceed 5 pixels.

[0086] Furthermore, in order to stitch together the image results, it is ensured that the multiple visual image sensors can produce a map result without obvious human visual differences. If the back-end control has a certain error tolerance range, it can be appropriately relaxed, which is determined according to the capability of the back-end control, and the embodiment of the present invention does not specifically limit this.

[0087] Step 202, obtaining road images and vehicle positioning information;

[0088] Road images can be obtained from visual image sensors. Vehicle positioning information can be obtained from the navigation system. Among them, the vehicle positioning information is based on the positioning after fusion positioning and Kalman filtering. The Kalman filtering output sensor of the fusion positioning will not affect the composition of the virtual lane line, making the fitting effect more accurate.

[0089] Step 203, based on the vehicle positioning information, determining the lane endpoint position and the intersection endpoint position from the road image;

[0090] Based on the vehicle positioning information, the vehicle's location is determined, and the lane endpoint position and intersection endpoint position are identified and separated from the road image.

[0091] Specifically, the step of determining the lane endpoint position and the intersection endpoint position from the road image based on the vehicle positioning information includes: determining the current lane in the road image based on the vehicle positioning information; determining the lane endpoint position in the current lane; performing image recognition on the road image to determine the intersection area; performing lane segmentation in the intersection area to determine the intersection endpoint position.

[0092] Based on the vehicle positioning information, the current lane of the vehicle can be determined in the road image, and the position of the vehicle in the current lane can be determined as the lane endpoint. Image recognition can also be performed on the road image to determine the intersection area, that is, the area without lane lines, and lane segmentation can be performed in the intersection area to determine the lanes of the next intersection, and the endpoint positions corresponding to these lanes, that is, the intersection endpoint positions, can be determined. For example, Figure 3 As shown in the figure, the location of point V is the end point of the lane, N i The location of is the endpoint of the intersection.

[0093] Step 204, extending the lane endpoint position according to the vehicle positioning information to generate a lane endpoint extension line;

[0094] The lane endpoint position is extended based on the vehicle positioning information to form a lane endpoint extension line.

[0095] Specifically, the step of extending the lane endpoint position based on the vehicle positioning information to generate a lane endpoint extension line includes: determining the direction of the intersection based on the vehicle positioning information; extending the lane endpoint position by a preset distance along the direction of the intersection to generate a lane update position; connecting the lane endpoint position with the lane update position to generate a lane endpoint extension line.

[0096] First, based on the vehicle positioning information, the vehicle's direction of movement can be determined, the intersection position to be fitted can be determined, and then the direction approaching the intersection can be determined as the intersection direction. The lane endpoint position is extended by a preset distance to generate a lane update position; the lane endpoint position is connected to the lane update position to generate a lane endpoint extension line. Figure 3 For example, the lane endpoint position is V, and it is extended toward the intersection by a preset distance L, generating a lane update position S, and the lane endpoint extension line is the line segment VS.

[0097] Step 205, extending the intersection endpoint position according to the vehicle positioning information to generate an intersection endpoint extension line;

[0098] The intersection endpoint position is extended based on the vehicle positioning information to form an intersection endpoint extension line.

[0099] Specifically, the step of extending the intersection endpoint position based on the vehicle positioning information to generate an intersection endpoint extension line includes: determining the direction of the intersection based on the vehicle positioning information; extending the intersection endpoint position by a preset distance along the intersection direction to generate an intersection update position; connecting the intersection endpoint position with the intersection update position to generate an intersection endpoint extension line.

[0100] Similarly, the intersection position can be determined based on the vehicle positioning information, and then the intersection direction can be determined. Along the intersection direction, the intersection endpoint position is extended by a preset distance to generate the intersection update position; the intersection endpoint position is connected with the intersection update position to generate the intersection endpoint extension line. Figure 3 For example, the intersection endpoint position is N i (i.e. N 1 、N 2 、N 3 ), extend the preset distance L, generate the intersection update position E i (i.e. E 1 、E 2 、E 3 ). Line segment N 1 E 1 , line segment N 2 E 2 and line segment N 3 E 3 All are extension lines of intersection endpoints.

[0101] In addition, the preset distance of extending the lane end position and the preset distance of extending the intersection end position can be the same. The specific value is specifically limited. In a preferred example of the present invention, it is 1.5 to 2 times the width of the pedestrian crossing line.

[0102] Step 206, determining a fitting rule according to the geometric relationship between the lane endpoint extension line and the intersection endpoint extension line;

[0103] Then, based on the geometric relationship between the lane endpoint extension line and the intersection endpoint extension line, corresponding fitting rules can be determined based on different geometric relationships.

[0104] In an optional embodiment of the present invention, the step of determining the fitting rule according to the geometric relationship between the lane endpoint extension line and the intersection endpoint extension line comprises:

[0105] Sub-step S2061, multiplying the direction vector of the lane endpoint extension line with the direction vector of the intersection endpoint extension line to determine a geometric relationship;

[0106] The direction vector of the lane endpoint extension line and the direction vector of the intersection endpoint extension line can be vector multiplied, and the geometric relationship can be determined based on the result of the multiplication. The vector multiplication can be a cross product or a dot product, which is not limited in the embodiment of the present invention. For example, the direction vector of the lane endpoint extension line and the direction vector of the intersection endpoint extension line are cross-multiplied or dot-multiplied, and the geometric relationship is distinguished as a quasi-perpendicular relationship or a quasi-parallel relationship based on whether the result approaches 0. That is, the quasi-parallel relationship is the geometric relationship between the lane endpoint extension line and the intersection endpoint extension line when the angle between the direction vector of the lane endpoint extension line and the direction vector of the intersection endpoint extension line is less than the angle threshold; and the quasi-perpendicular relationship is the geometric relationship between the lane endpoint extension line and the intersection endpoint extension line when the angle between the direction vector of the lane endpoint extension line and the direction vector of the intersection endpoint extension line is greater than or equal to the angle threshold. The size of the angle threshold can be determined according to demand.

[0107] Sub-step S2062, in response to the geometric relationship being a quasi-vertical relationship, determining the fitting rule to be arc fitting;

[0108] The geometric relationship is a quasi-vertical relationship, which means that the current lane is close to the lane corresponding to the next intersection, such as the change of the road under the vehicle turning condition. The quasi-vertical relationship represents the geometric relationship between the previous lane and the lane corresponding to the next intersection that is close to the vertical. At this time, in response to the geometric relationship being a quasi-vertical relationship, the fitting rule can be determined to be arc fitting. Arc fitting is to determine the virtual lane line in the form of fitting arcs.

[0109] Sub-step S2063, in response to the geometric relationship being a quasi-parallel relationship, determining an angle between an extension line of the lane endpoint and an extension line of the road junction endpoint;

[0110] The geometric relationship is a quasi-parallel relationship, which means that the current lane and the lane corresponding to the next intersection are roughly in a straight line, such as the change of the road when the vehicle moves forward or changes lanes. The quasi-parallel relationship represents the geometric relationship that the current lane and the lane corresponding to the next intersection are nearly parallel. At this time, in response to the geometric relationship being a quasi-parallel relationship, the angle between the extension line of the lane endpoint and the extension line of the intersection endpoint can be determined. Figure 3 As shown, the angle between the extended line of the lane endpoint and the extended line of the line intersection endpoint is angle a.

[0111] Sub-step S2064, determining a fitting rule based on the angle.

[0112] The corresponding fitting rule is determined based on the size of the angle.

[0113] Specifically, the step of determining the fitting rule based on the angle includes: judging whether the angle is greater than a preset threshold; in response to the angle being greater than the preset threshold, determining that the fitting rule is Bessel fitting; in response to the angle being not greater than the preset threshold, determining that the fitting rule is straight line fitting.

[0114] It can be determined whether the angle is greater than a preset threshold. When the angle is greater than the preset threshold, it means that the vehicle needs to change lanes when driving to the lane of the next intersection, and the fitting rule can be determined to be Bezier fitting. Bezier fitting is the fitting of Bezier curves based on the lane endpoint position and the intersection endpoint position. When the angle is not greater than the preset threshold, it means that the vehicle can maintain the current state and drive straight to the lane of the next intersection, and the fitting rule can be determined to be straight line fitting. Straight line fitting is the fitting of a straight line based on the lane endpoint position and the intersection endpoint position. Among them, the preset threshold is determined in relation to the current lane conditions, and the embodiment of the present invention does not specifically limit this. In a preferred example of the present invention, on urban roads, the value of the preset threshold Max is 10° to 15°.

[0115] Step 207: Based on the fitting rule, fitting is performed on the lane endpoint position and the intersection endpoint position to generate a virtual lane line.

[0116] The corresponding fitting rules are used to fit the path between the lane endpoint position and the intersection endpoint position to generate a virtual lane line.

[0117] In one example of the present invention, when the fitting rule is straight line fitting, the step of fitting the lane endpoint position and the intersection endpoint position based on the fitting rule to generate a virtual lane line includes: fitting the lane endpoint position and the intersection endpoint position with a straight line to generate a virtual lane line.

[0118] When the fitting rule is straight line fitting, a straight line fitting can be performed between the lane endpoint position and the intersection endpoint position, and the lane endpoint position and the intersection endpoint position are connected into a straight line, and the straight line is used as a virtual lane line.

[0119] In one example of the invention, when the fitting rule is Bezier fitting, the step of fitting the lane endpoint position and the intersection endpoint position based on the fitting rule to generate a virtual lane line includes: determining Bezier curve control points based on the lane endpoint position and the intersection endpoint position; performing Bezier fitting based on the Bezier curve control points, the lane endpoint position and the intersection endpoint position to generate a virtual lane line.

[0120] When Bezier fitting is used, the Bezier curve control points can be first determined based on the lane endpoint position and the intersection endpoint position. The Bezier curve control points are the position points required to generate the Bezier curve, and can be determined according to different Bezier curve forms. For example, when a third-order Bezier curve is used for fitting, the point that divides the distance between the lane endpoint position and the intersection endpoint position into three equal parts can be determined as the Bezier curve control point.

[0121] Based on the Bezier curve control points, lane endpoint positions and intersection endpoint positions as curve fitting control points, Bezier fitting is performed to generate virtual lane lines. Figure 4a and Figure 4b , 4 control points are selected, namely S, E i , P and Q i Among them, the control points P and Q i The total length of line segment SE2 is divided into three equal parts and then obtained by equal distance. This method of obtaining can avoid complex calculations and ensure the smoothness of the generated virtual lane line, thereby ensuring the comfort and safety of the vehicle during operation.

[0122] In one example of the invention, when the fitting rule is arc fitting, the step of fitting the lane endpoint position and the intersection endpoint position based on the fitting rule to generate a virtual lane line includes: based on the arc fitting, determining the lane endpoint position and the intersection endpoint position as circle tangent points; performing circumscribed circle fitting based on the circle tangent points to generate a circumscribed circle; in the circumscribed circle, determining the arc between the circle tangent points as the virtual lane line.

[0123] When performing arc fitting, the lane endpoint position and the intersection endpoint position can be used as the tangent point of the circle, that is, the circle tangent point. Based on the two circle tangent points, the corresponding circumscribed circle is fitted, and the arc between the two circle tangent points in the circumscribed circle is determined as the virtual lane line.

[0124] For example, Figure 5a and Figure 5b , S, E i are the tangent points, at S and E i The center of the circle is selected within the constructed range to determine the tangent arc. At the same time, since the actual entry point of the lane line is extended, the smoothness of the subsequent line segments can be ensured.

[0125] The embodiment of the present invention calibrates the sensor; obtains the road image and the vehicle positioning information; determines the lane endpoint position and the intersection endpoint position from the road image based on the vehicle positioning information; extends the lane endpoint position according to the vehicle positioning information to generate the lane endpoint extension line; extends the intersection endpoint position according to the vehicle positioning information to generate the intersection endpoint extension line; determines the fitting rule according to the geometric relationship between the lane endpoint extension line and the intersection endpoint extension line; and performs fitting for the lane endpoint position and the intersection endpoint position based on the fitting rule to generate a virtual lane line. The virtual lane line can be determined by identifying the image information, so that the virtual lane line can be generated in the scene without high-precision map or intersection without lane marking guidance, which is conducive to improving the driving control stability of the vehicle. Based on the geometric relationship between different lane endpoint extension lines and intersection endpoint extension lines, the corresponding fitting rule is determined; different fitting rules are used to fit the virtual lane line, so that the virtual lane line generation can be accurately fitted in different scenes, so that the accuracy of the virtual lane line can be generated.

[0126] In order to make the implementation process of the embodiments of the present invention clear to those skilled in the art, refer to Figure 6 , the following is an example:

[0127] S1: Calibrate and confirm each sensor device.

[0128] S2: Acquire an image, determine the current running direction of the vehicle, and record the endpoint information of the lane where the vehicle is currently located (i.e., the lane endpoint position), recorded as V;

[0129] S3: Get the image and obtain the endpoint information of the intersection (i.e. the location of the intersection endpoint) based on the lane line segmentation map, recorded as point N i ;

[0130] S4: Based on the vehicle's positioning information, take L as the distance and extend V and N in sequence i The location of the endpoint is recorded as S and E respectively. i ;

[0131] S5: Determine line segments VS and N respectively i E i The geometric relationship is divided into two categories: almost perpendicular (i.e., quasi-perpendicular relationship) and almost parallel (i.e., quasi-parallel relationship).

[0132] S6: For the general vertical class, use and as the tangent points, take the center of the circle on the inside, draw an arc, and form a virtual lane line;

[0133] S7: For almost parallel classes, extend VS and connect SE i , with VS as 0° and counterclockwise as positive, calculate VS and SEi The angle between the two is denoted as α, and the threshold is Max;

[0134] S8: If equation (1) is satisfied, then directly connect to SE i As a virtual lane line. If formula (1) is not satisfied, SE i Divide the image into three equal parts, and extend S and E by the three-division distance respectively. i To P and Q i ;

[0135] |α≤Max (1)

[0136] S9: S, E i , P and Q i As control points, S and E i As the endpoint, make a Bezier curve and construct a virtual lane line.

[0137] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0138] Reference Figure 7 , shows a structural block diagram of an embodiment of a virtual lane line generating device of the present invention, and the virtual lane line generating device may specifically include the following modules:

[0139] An acquisition module 701 is used to acquire road images and vehicle positioning information;

[0140] A position identification module 702, configured to determine a lane endpoint position and an intersection endpoint position from the road image based on the vehicle positioning information;

[0141] The first extension module 703 is used to extend the lane endpoint position according to the vehicle positioning information to generate a lane endpoint extension line;

[0142] The second extension module 704 is used to extend the intersection endpoint position according to the vehicle positioning information to generate an intersection endpoint extension line;

[0143] A fitting rule determination module 705, for determining a fitting rule according to a geometric relationship between an extension line of the lane endpoint and an extension line of the intersection endpoint;

[0144] The fitting module 706 is used to fit the lane endpoint position and the intersection endpoint position based on the fitting rule to generate a virtual lane line.

[0145] In an optional embodiment of the invention, the fitting rule determination module 705 includes:

[0146] A geometric relationship determination submodule, used for performing vector multiplication of the direction vector of the lane endpoint extension line and the direction vector of the intersection endpoint extension line to determine the geometric relationship;

[0147] A first response submodule, configured to determine that the fitting rule is arc fitting in response to the geometric relationship being a quasi-vertical relationship;

[0148] A second response submodule is used for determining the angle between the extension line of the lane endpoint and the extension line of the line intersection endpoint in response to the geometric relationship being a quasi-parallel relationship;

[0149] The fitting rule determination submodule is used to determine the fitting rule based on the angle.

[0150] In an optional embodiment of the invention, the fitting rule determination submodule includes:

[0151] A judging unit, used to judge whether the angle is greater than a preset threshold;

[0152] A first response unit, configured to determine that the fitting rule is Bessel fitting in response to the angle being greater than the preset threshold;

[0153] The second response unit is used to determine that the fitting rule is straight line fitting in response to the angle being not greater than the preset threshold.

[0154] In an optional embodiment of the invention, when the fitting rule is Bessel fitting, the fitting module 706 includes:

[0155] A control point determination submodule, used to determine the Bezier curve control point according to the lane endpoint position and the intersection endpoint position;

[0156] The first fitting submodule is used to perform Bezier fitting based on the Bezier curve control point, the lane endpoint position and the intersection endpoint position to generate a virtual lane line.

[0157] In an optional embodiment of the invention, the control point determination submodule includes:

[0158] The control point determination unit is used to determine the point that divides the distance between the lane endpoint position and the intersection endpoint position into three equal parts as the Bezier curve control point.

[0159] In an optional embodiment of the invention, when the fitting rule is straight line fitting, the fitting module 706 includes:

[0160] The second fitting submodule is used to perform straight line fitting between the lane endpoint position and the intersection endpoint position to generate a virtual lane line.

[0161] In an optional embodiment of the invention, when the fitting rule is arc fitting, the fitting module 706 includes:

[0162] A circle tangent point determination submodule, used for determining the lane endpoint position and the intersection endpoint position as circle tangent points based on the arc fitting;

[0163] A third fitting submodule is used to perform circumscribed circle fitting based on the circle tangent points to generate a circumscribed circle;

[0164] The arc determination submodule is used to determine, in the circumscribed circle, the arc between the circle tangent points as a virtual lane line.

[0165] In an optional embodiment of the invention, the location identification module 702 includes:

[0166] A current lane determination submodule, configured to determine a current lane in the road image based on the vehicle positioning information;

[0167] A lane endpoint position determination submodule, used to determine the lane endpoint position in the current lane;

[0168] A road intersection area determination submodule, used to perform image recognition on the road image to determine the road intersection area;

[0169] The intersection endpoint position determination submodule is used to perform lane segmentation in the intersection area and determine the intersection endpoint position.

[0170] In an optional embodiment of the invention, the first extension module 703 includes:

[0171] A direction determination submodule, used to determine the direction of the intersection based on the vehicle positioning information;

[0172] A first extension submodule is used to extend the lane endpoint position by a preset distance along the direction of the intersection to generate a lane update position;

[0173] The first connection module is used to connect the lane endpoint position with the lane update position to generate a lane endpoint extension line.

[0174] In an optional embodiment of the invention, the second extension module 704 includes:

[0175] A direction determination submodule, used to determine the direction of the intersection based on the vehicle positioning information;

[0176] A second extension submodule is used to extend the intersection endpoint position by a preset distance along the direction of the intersection to generate an updated intersection position;

[0177] The second connection module is used to connect the intersection endpoint position with the intersection update position to generate an intersection endpoint extension line.

[0178] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0179] Reference Figure 8 , an embodiment of the present invention further provides a vehicle, comprising:

[0180] A processor 801 and a storage medium 802, wherein the storage medium 802 stores a computer program executable by the processor 801, and when the vehicle is running, the processor 801 executes the computer program to perform the virtual lane line generation method as described in any one of the embodiments of the present invention. The virtual lane line generation method includes:

[0181] Acquire road images and vehicle positioning information;

[0182] Based on the vehicle positioning information, determining a lane endpoint position and an intersection endpoint position from the road image;

[0183] According to the vehicle positioning information, the lane endpoint position is extended to generate a lane endpoint extension line;

[0184] According to the vehicle positioning information, the intersection endpoint position is extended to generate an intersection endpoint extension line;

[0185] Determining a fitting rule according to a geometric relationship between an extension line of the lane endpoint and an extension line of the intersection endpoint;

[0186] Based on the fitting rule, the lane endpoint position and the intersection endpoint position are fitted to generate a virtual lane line.

[0187] Optionally, the step of determining the fitting rule according to the geometric relationship between the lane endpoint extension line and the intersection endpoint extension line comprises:

[0188] Performing vector multiplication of the direction vector of the extension line of the lane endpoint and the direction vector of the extension line of the intersection endpoint to determine a geometric relationship;

[0189] In response to the geometric relationship being a quasi-vertical relationship, determining the fitting rule to be arc fitting;

[0190] In response to the geometric relationship being a quasi-parallel relationship, determining an angle between an extension line of the lane endpoint and an extension line of the route intersection endpoint;

[0191] A fitting rule is determined based on the angle.

[0192] Optionally, the step of determining a fitting rule based on the angle includes:

[0193] Determining whether the angle is greater than a preset threshold;

[0194] In response to the angle being greater than the preset threshold, determining that the fitting rule is Bessel fitting;

[0195] In response to the angle being not greater than the preset threshold, determining that the fitting rule is straight line fitting.

[0196] Optionally, when the fitting rule is Bessel fitting, the step of performing fitting based on the fitting rule for the lane endpoint position and the intersection endpoint position to generate a virtual lane line includes:

[0197] Determining a Bezier curve control point according to the lane endpoint position and the intersection endpoint position;

[0198] Bezier fitting is performed based on the Bezier curve control points, the lane endpoint positions and the intersection endpoint positions to generate a virtual lane line.

[0199] Optionally, the step of determining a Bezier curve control point according to the lane endpoint position and the intersection endpoint position includes:

[0200] The point that divides the distance between the lane endpoint position and the intersection endpoint position into three equal parts is determined as the Bezier curve control point.

[0201] Optionally, when the fitting rule is straight line fitting, the step of performing fitting based on the fitting rule for the lane endpoint position and the intersection endpoint position to generate a virtual lane line includes:

[0202] The lane endpoint position and the intersection endpoint position are linearly fitted to generate a virtual lane line.

[0203] Optionally, when the fitting rule is arc fitting, the step of performing fitting based on the fitting rule for the lane endpoint position and the intersection endpoint position to generate a virtual lane line includes:

[0204] Based on the arc fitting, determining the lane endpoint position and the intersection endpoint position as circle tangent points;

[0205] Perform circumscribed circle fitting based on the circle tangent points to generate a circumscribed circle;

[0206] In the circumscribed circle, the arc between the circle tangent points is determined as the virtual lane line.

[0207] Optionally, the step of determining a lane endpoint position and an intersection endpoint position from the road image based on the vehicle positioning information includes:

[0208] Based on the vehicle positioning information, determining the current lane in the road image;

[0209] Determining a lane endpoint position in the current lane;

[0210] Performing image recognition on the road image to determine an intersection area;

[0211] Lane segmentation is performed in the intersection area to determine the intersection endpoint position.

[0212] Optionally, the step of extending the lane endpoint position according to the vehicle positioning information to generate a lane endpoint extension line includes:

[0213] Determining the direction of the intersection according to the vehicle positioning information;

[0214] Extending the lane endpoint position by a preset distance along the direction of the intersection to generate a lane update position;

[0215] The lane endpoint position is connected to the lane update position to generate a lane endpoint extension line.

[0216] Optionally, the step of extending the intersection endpoint position according to the vehicle positioning information to generate an intersection endpoint extension line includes:

[0217] Determining the direction of the intersection according to the vehicle positioning information;

[0218] Along the direction of the intersection, extending the intersection endpoint position by a preset distance to generate an intersection update position;

[0219] The intersection endpoint position is connected with the intersection update position to generate an intersection endpoint extension line.

[0220] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the processor.

[0221] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0222] Reference Fig. 9 The embodiment of the present invention further provides a computer-readable storage medium 901, on which a computer program is stored. When the computer program is executed by a processor, the method for generating a virtual lane line as described in any one of the embodiments of the present invention is executed. The method for generating a virtual lane line includes:

[0223] Acquire road images and vehicle positioning information;

[0224] Based on the vehicle positioning information, determining a lane endpoint position and an intersection endpoint position from the road image;

[0225] According to the vehicle positioning information, the lane endpoint position is extended to generate a lane endpoint extension line;

[0226] According to the vehicle positioning information, the intersection endpoint position is extended to generate an intersection endpoint extension line;

[0227] Determining a fitting rule according to a geometric relationship between an extension line of the lane endpoint and an extension line of the intersection endpoint;

[0228] Based on the fitting rule, the lane endpoint position and the intersection endpoint position are fitted to generate a virtual lane line.

[0229] Optionally, the step of determining the fitting rule according to the geometric relationship between the lane endpoint extension line and the intersection endpoint extension line comprises:

[0230] Performing vector multiplication of the direction vector of the extension line of the lane endpoint and the direction vector of the extension line of the intersection endpoint to determine a geometric relationship;

[0231] In response to the geometric relationship being a quasi-vertical relationship, determining the fitting rule to be arc fitting;

[0232] In response to the geometric relationship being a quasi-parallel relationship, determining an angle between an extension line of the lane endpoint and an extension line of the route intersection endpoint;

[0233] A fitting rule is determined based on the angle.

[0234] Optionally, the step of determining a fitting rule based on the angle includes:

[0235] Determining whether the angle is greater than a preset threshold;

[0236] In response to the angle being greater than the preset threshold, determining that the fitting rule is Bessel fitting;

[0237] In response to the angle being not greater than the preset threshold, determining that the fitting rule is straight line fitting.

[0238] Optionally, when the fitting rule is Bessel fitting, the step of performing fitting based on the fitting rule for the lane endpoint position and the intersection endpoint position to generate a virtual lane line includes:

[0239] Determining a Bezier curve control point according to the lane endpoint position and the intersection endpoint position;

[0240] Bezier fitting is performed based on the Bezier curve control points, the lane endpoint positions and the intersection endpoint positions to generate a virtual lane line.

[0241] Optionally, the step of determining a Bezier curve control point according to the lane endpoint position and the intersection endpoint position includes:

[0242] The point that divides the distance between the lane endpoint position and the intersection endpoint position into three equal parts is determined as the Bezier curve control point.

[0243] Optionally, when the fitting rule is straight line fitting, the step of performing fitting based on the fitting rule for the lane endpoint position and the intersection endpoint position to generate a virtual lane line includes:

[0244] The lane endpoint position and the intersection endpoint position are linearly fitted to generate a virtual lane line.

[0245] Optionally, when the fitting rule is arc fitting, the step of performing fitting based on the fitting rule for the lane endpoint position and the intersection endpoint position to generate a virtual lane line includes:

[0246] Based on the arc fitting, determining the lane endpoint position and the intersection endpoint position as circle tangent points;

[0247] Perform circumscribed circle fitting based on the circle tangent points to generate a circumscribed circle;

[0248] In the circumscribed circle, the arc between the circle tangent points is determined as the virtual lane line.

[0249] Optionally, the step of determining a lane endpoint position and an intersection endpoint position from the road image based on the vehicle positioning information includes:

[0250] Based on the vehicle positioning information, determining the current lane in the road image;

[0251] Determining a lane endpoint position in the current lane;

[0252] Performing image recognition on the road image to determine an intersection area;

[0253] Lane segmentation is performed in the intersection area to determine the intersection endpoint position.

[0254] Optionally, the step of extending the lane endpoint position according to the vehicle positioning information to generate a lane endpoint extension line includes:

[0255] Determining the direction of the intersection according to the vehicle positioning information;

[0256] Extending the lane endpoint position by a preset distance along the direction of the intersection to generate a lane update position;

[0257] The lane endpoint position is connected to the lane update position to generate a lane endpoint extension line.

[0258] Optionally, the step of extending the intersection endpoint position according to the vehicle positioning information to generate an intersection endpoint extension line includes:

[0259] Determining the direction of the intersection according to the vehicle positioning information;

[0260] Along the direction of the intersection, extending the intersection endpoint position by a preset distance to generate an intersection update position;

[0261] The intersection endpoint position is connected with the intersection update position to generate an intersection endpoint extension line.

[0262] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0263] It will be appreciated by those skilled in the art that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0264] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0265] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0266] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0267] The above embodiments are only preferred embodiments for fully illustrating the present invention, and the protection scope of the present invention is not limited thereto. Any equivalent substitution or change made by a person skilled in the art based on the present invention is within the protection scope of the present invention.

Claims

1. A method for generating a virtual lane line, characterized in that: include: Acquire road images and vehicle positioning information; Based on the vehicle positioning information, determining a lane endpoint position and an intersection endpoint position from the road image; According to the vehicle positioning information, the lane endpoint position is extended to generate a lane endpoint extension line; According to the vehicle positioning information, the intersection endpoint position is extended to generate an intersection endpoint extension line; Determining a fitting rule according to a geometric relationship between an extension line of the lane endpoint and an extension line of the intersection endpoint; Based on the fitting rule, the lane endpoint position and the intersection endpoint position are fitted to generate a virtual lane line.

2. The method according to claim 1, characterized in that The step of determining the fitting rule according to the geometric relationship between the lane endpoint extension line and the intersection endpoint extension line comprises: Performing vector multiplication of the direction vector of the extension line of the lane endpoint and the direction vector of the extension line of the intersection endpoint to determine a geometric relationship; In response to the geometric relationship being a quasi-vertical relationship, determining the fitting rule to be arc fitting; In response to the geometric relationship being a quasi-parallel relationship, determining an angle between an extension line of the lane endpoint and an extension line of the route intersection endpoint; A fitting rule is determined based on the angle.

3. The method according to claim 2, characterized in that The step of determining the fitting rule based on the angle comprises: Determining whether the angle is greater than a preset threshold; In response to the angle being greater than the preset threshold, determining that the fitting rule is Bessel fitting; In response to the angle being not greater than the preset threshold, determining that the fitting rule is straight line fitting.

4. The method according to claim 3, characterized in that When the fitting rule is Bezier fitting, the step of performing fitting based on the fitting rule for the lane endpoint position and the intersection endpoint position to generate a virtual lane line includes: Determining a Bezier curve control point according to the lane endpoint position and the intersection endpoint position; Bezier fitting is performed based on the Bezier curve control points, the lane endpoint positions and the intersection endpoint positions to generate a virtual lane line.

5. The method according to claim 4, characterized in that The step of determining the Bezier curve control point according to the lane endpoint position and the intersection endpoint position comprises: The point that divides the distance between the lane endpoint position and the intersection endpoint position into three equal parts is determined as the Bezier curve control point.

6. The method according to claim 3, characterized in that: When the fitting rule is straight line fitting, the step of fitting the lane endpoint position and the intersection endpoint position based on the fitting rule to generate a virtual lane line includes: The lane endpoint position and the intersection endpoint position are linearly fitted to generate a virtual lane line.

7. The method according to claim 2, characterized in that When the fitting rule is arc fitting, the step of fitting the lane endpoint position and the intersection endpoint position based on the fitting rule to generate a virtual lane line includes: Based on the arc fitting, determining the lane endpoint position and the intersection endpoint position as circle tangent points; Perform circumscribed circle fitting based on the circle tangent points to generate a circumscribed circle; In the circumscribed circle, the arc between the circle tangent points is determined as the virtual lane line.

8. The method according to claim 1, characterized in that The step of determining the lane endpoint position and the intersection endpoint position from the road image based on the vehicle positioning information comprises: Based on the vehicle positioning information, determining the current lane in the road image; Determining a lane endpoint position in the current lane; Performing image recognition on the road image to determine an intersection area; Lane segmentation is performed in the intersection area to determine the intersection endpoint position.

9. The method according to claim 1, characterized in that: The step of extending the lane endpoint position according to the vehicle positioning information to generate a lane endpoint extension line comprises: Determining the direction of the intersection according to the vehicle positioning information; Extending the lane endpoint position by a preset distance along the direction of the intersection to generate a lane update position; The lane endpoint position is connected to the lane update position to generate a lane endpoint extension line.

10. The method according to claim 1, characterized in that The step of extending the intersection endpoint position according to the vehicle positioning information to generate an intersection endpoint extension line comprises: Determining the direction of the intersection according to the vehicle positioning information; Along the direction of the intersection, extending the intersection endpoint position by a preset distance to generate an intersection update position; The intersection endpoint position is connected with the intersection update position to generate an intersection endpoint extension line.

11. A virtual lane line generating device, characterized in that: include: An acquisition module, used to acquire road images and vehicle positioning information; A position recognition module, used to determine the lane endpoint position and the intersection endpoint position from the road image based on the vehicle positioning information; A first extension module, used to extend the lane endpoint position according to the vehicle positioning information to generate a lane endpoint extension line; A second extension module is used to extend the intersection endpoint position according to the vehicle positioning information to generate an intersection endpoint extension line; A fitting rule determination module, used to determine a fitting rule according to a geometric relationship between an extension line of the lane endpoint and an extension line of the intersection endpoint; The fitting module is used to fit the lane endpoint position and the intersection endpoint position based on the fitting rule to generate a virtual lane line.

12. A vehicle, characterized in that: The method comprises a processor, a memory and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the steps of the method for generating a virtual lane line according to any one of claims 1 to 10 are implemented.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the virtual lane line generation method according to any one of claims 1 to 10 are implemented.

Images