Lane line fitting method, device, equipment and medium

Abstract

The invention relates to the technical field of automatic driving, and discloses a lane line fitting method, device and equipment and a medium, and the method comprises the steps: obtaining a lane image; extracting a lane line graph with a preset length from the lane image; taking points from the lane line graph at equal intervals to obtain a plurality of lane line fitting point positions; fitting and calculating a current lane line equation by using the lane line fitting point; and controlling the vehicle to run based on the current lane line equation, returning to the step of acquiring the lane image when the vehicle runs to a preset position, and fitting the lane line equation of the next lane line with the preset length. On the basis of an original lane line fitting scheme, optimization is carried out only through the principle of distance estimation and equidistant point taking, under the condition that an algorithm and an architecture are not changed, the weight value of fitting lane line data at the far end is particularly ensured, the stability of a finally-calculated curve result is ensured, and the method is suitable for being applied to a vehicle. And the far-end error of the lane line fitting result is reduced.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a lane line fitting method, device, equipment and medium. Background Art

[0002] Lane line information plays an important role in both traditional autonomous driving solutions and end-to-end solutions based on intelligent driving. Lane line information is not only the key to the lateral distance positioning of the positioning system, but also the key to the safe and comfortable control of the vehicle by the predictive control end. At present, the lane line equations fitted by some commonly used lane line fitting algorithms are more accurate at the near end of the road, but there are large errors at the far end of the lane, especially in the curved scenes with small curvature. This method often leads to excessive errors at the far end of the lane line fitting results, which has a great impact on the lateral positioning processing of the vehicle, resulting in serious deviations in the positioning of the vehicle in the middle of the lane. Summary of the invention

[0003] In view of this, the present invention provides a lane line fitting method, device, equipment, medium and product to solve the problem of excessive far-end error of lane line fitting results.

[0004] In a first aspect, the present invention provides a lane line fitting method, the method comprising: acquiring a lane image; extracting a lane line graphic of a preset length from the lane image, the preset length being the length calculated from the current position of the vehicle, and the preset length being less than the actual length of the lane; taking points at equal distances from the lane line graphic to obtain a number of lane line fitting points; using the lane line fitting points to fit and calculate the current lane line equation; controlling vehicle travel based on the current lane line equation, and when the vehicle travels to a preset position, returning to the step of acquiring the lane image, fitting the lane line equation of the next lane line of a preset length, the preset position being a set position close to the end point of the current preset length.

[0005] According to the above technical means, the present invention optimizes the original lane line fitting scheme only through the principles of distance estimation and equidistant point selection, thereby achieving the situation where the algorithm and architecture are not changed, especially ensuring the weight value of the fitted lane line data at the far end, ensuring the stability of the final calculated curve result, and reducing the far-end error of the lane line fitting result.

[0006] In some optional implementations, acquiring the lane image includes: calibrating an image acquisition device with external parameters according to a reprojection error; and acquiring the lane image using the image acquisition device after the external parameter calibration.

[0007] According to the above technical means, the present invention also performs external parameter calibration on the image acquisition device before road image acquisition, thereby reducing the error of the conversion matrix between the image and the real space, and further improving the accuracy of the subsequent fitted lane line conversion to the vehicle coordinate system.

[0008] In some optional embodiments, a lane line graphic of a preset length is extracted from a lane image, including: when the focal length of the image acquisition device is greater than 8 mm, selecting a lane line graphic with a preset length of 40 m; when the focal length of the image acquisition device is less than or equal to 8 mm, selecting a lane line graphic with a preset length not exceeding 30 m; when the focal length of the image acquisition device is less than or equal to 8 mm, and the positioning accuracy needs to be higher than the requirement of 0.2 times the width between two lane lines, selecting a lane line graphic with a preset length not exceeding 20 m.

[0009] According to the above technical means, as the effective focal length of the camera decreases and the accuracy increases, the distant route at a longer distance in the image may be deformed due to the camera's own parameters, resulting in inaccurate fitted lane lines. Therefore, the present invention adaptively reduces the length of each fitted lane line according to the reduction of the effective focal length of the camera and the improvement of the accuracy, so that the lane line fitting process is completed as close to the vehicle as possible, thereby further improving the accuracy of the fitted lane line in each frame.

[0010] In some optional implementations, the current lane line equation is calculated using lane line fitting point fitting, including: determining a basis of a target equation based on the number of curves in the lane line graphic; substituting the graphic coordinates of the lane line fitting point into the target equation to obtain the lane line equation in the image coordinate system; and converting the lane line equation in the image coordinate system based on the conversion relationship between the image coordinate system and the vehicle coordinate system to obtain the current lane line equation in the vehicle coordinate system.

[0011] According to the above technical means, the present invention flexibly adjusts the basis order and basis type of the lane line equation based on the complexity of the lane line curve, so as to improve the fitting accuracy in response to more complex lane lines.

[0012] In some optional embodiments, the basis of the target equation is determined according to the number of curves in the lane line graphic, including: when the number of curves is less than 2, using a 3rd-order basis; when the number of curves is greater than or equal to 2, using a 5th-order basis or a Legendre basis.

[0013] According to the above technical means, the present invention adopts a more complex 5th-order basis or Legendre basis for continuous complex curves with a larger number of curves, and a 3rd-order basis can be used for fitting for single curvature curves, thereby selecting a more suitable formula basis according to the number of curves, taking into account the accuracy and efficiency of lane line fitting.

[0014] In some optional implementations, the lane line equation in the image coordinate system is converted according to the conversion relationship between the image coordinate system and the vehicle coordinate system to obtain the current lane line equation in the vehicle coordinate system, including: converting the lane line equation in the image coordinate system to the vehicle coordinate system according to the conversion relationship between the image coordinate system and the vehicle coordinate system; adjusting the parameters of the converted lane line equation based on the lateral constraint to obtain the adjusted lane line equation, the lateral constraint being used to constrain the first lane line figure generated corresponding to the lane line equation to fall within the lane composed of the actual lane line figure.

[0015] According to the above technical means, the present invention converts the lane line equation in the image coordinate system to the vehicle coordinate system, and also limits the lane line equation based on the lateral constraint condition that it cannot exceed the actual lane line edge, so as to fine-tune the parameters of the lane line equation, thereby further improving the accuracy of the lane line equation.

[0016] In some optional implementations, the method further includes: determining a lane edge position based on a current lane line equation; and determining a lateral distance between the vehicle and the lane edge when the vehicle is traveling using the lane edge position.

[0017] According to the above technical means, the present invention uses the optimized lane line equation to laterally locate the position of the vehicle in the center of the lane, which can significantly improve the accuracy of the vehicle's lateral positioning and solve the problem of too much space in the middle of the lane when the vehicle travels to the far end of the lane.

[0018] In a second aspect, the present invention provides a lane line fitting device, which includes: an image acquisition module for acquiring a lane image; a graphic recognition module for extracting a lane line graphic of a preset length from the lane image, wherein the preset length is the length calculated from the current position of the vehicle, and the preset length is less than the actual length of the lane; a data point acquisition module for taking points at equal distances from the lane line graphic to obtain a number of lane line fitting points; an equation fitting module for fitting and calculating the current lane line equation using the lane line fitting points; a loop control module for controlling vehicle driving based on the current lane line equation, and when the vehicle drives to a preset position, returning to the step of acquiring the lane image, and fitting the lane line equation of the next lane line of a preset length, wherein the preset position is a set position close to the end point of the current preset length.

[0019] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0020] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to cause a computer to execute the method of the first aspect or any corresponding embodiment thereof.

[0021] The technical solution provided by the present invention has the following advantages:

[0022] (1) Based on the above technical means, the present invention optimizes the original lane line fitting scheme only through the principle of distance estimation and equidistant point selection, thereby achieving the goal of ensuring the weight value of the fitted lane line data at the far end without changing the algorithm and architecture, ensuring the stability of the final calculated curve result, and reducing the far-end error of the lane line fitting result.

[0023] (2) Based on the above technical means, the present invention also performs external parameter calibration on the image acquisition device before road image acquisition, thereby reducing the error of the conversion matrix between the image and the real space, and further improving the accuracy of the subsequent fitted lane line conversion to the vehicle coordinate system.

[0024] (3) According to the above technical means, as the effective focal length of the camera decreases and the accuracy increases, the distant route at a longer distance in the image may be deformed due to the camera's own parameters, resulting in inaccurate fitted lane lines. Therefore, the present invention adaptively reduces the length of each fitted lane line based on the reduction of the effective focal length of the camera and the improvement of the accuracy, so that the lane line fitting process is completed as close to the vehicle as possible, thereby further improving the accuracy of the fitted lane line in each frame.

[0025] (4) According to the above technical means, the present invention flexibly adjusts the basis order and basis type of the lane line equation based on the complexity of the lane line curve, thereby improving the fitting accuracy for more complex lane lines.

[0026] (5) According to the above technical means, the present invention adopts a more complex 5th-order basis or Legendre basis for continuous complex curves with a larger number of curves, and a 3rd-order basis can be used for fitting for single curvature curves, thereby selecting a more appropriate formula basis according to the number of curves, taking into account the accuracy and efficiency of lane line fitting.

[0027] (6) According to the above technical means, the present invention converts the lane line equation in the image coordinate system to the vehicle coordinate system, and also limits the lane line equation based on the lateral constraint condition that it cannot exceed the edge of the actual lane line, so as to fine-tune the parameters of the lane line equation, thereby further improving the accuracy of the lane line equation.

[0028] (7) According to the above technical means, the present invention uses the optimized lane line equation to perform lateral positioning of the vehicle in the center of the lane, which can significantly improve the accuracy of the vehicle's lateral positioning and solve the problem of excessive use of the middle position of the lane when the vehicle travels to the far end of the lane. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0030] Figure 1 This is the lane line fitting effect diagram of the related technology;

[0031] Figure 2 is a schematic flow chart of a lane line fitting method according to an embodiment of the present invention;

[0032] Figure 3 It is a schematic diagram of comparison effect of a lane line fitting method according to an embodiment of the present invention and a fitting lane line of a related technology;

[0033] Figure 4 is another schematic diagram of comparison effect of a lane line fitting method and a related technology according to an embodiment of the present invention;

[0034] Figure 5 is another schematic flow chart of a lane line fitting method according to an embodiment of the present invention;

[0035] Figure 6 is a schematic structural diagram of a lane line fitting device according to an embodiment of the present invention;

[0036] Figure 7 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0038] Lane line information plays an important role in both traditional autonomous driving solutions and end-to-end solutions based on intelligent driving. Lane line information is not only the key to the lateral distance positioning of the positioning system, but also the key to the safe and comfortable control of the vehicle by the predictive control end. At present, the lane line equations fitted by some commonly used lane line fitting algorithms are more accurate at the near end of the road, but there are large errors at the far end of the lane, especially in the curved scenes with small curvature. This method often leads to excessive errors at the far end of the lane line fitting results, which has a great impact on the lateral positioning processing of the vehicle, resulting in serious deviations in the positioning of the vehicle in the middle of the lane.

[0039] The main reason for the above problem is that under normal circumstances, the curve is an arc, and its mathematical description is usually as shown in the following formula (1), which means that as the lane travels x distance in the forward direction, the lane changes y in the lateral direction. The traditional cubic fitting expression has a basis as shown in formula (2). Since formula (2) lacks y in the fitting, 2 Therefore, in the y direction, using formula (2) to fit the lane line will inevitably lead to a certain degree of deviation, and according to the data distribution used in the fitting, it will lead to different degrees of difference. The deviation increases with the increase of x, so the error of the lane line is mainly reflected in the far end of the fitting. In addition, since most of the fitting points of the traditional lane line fitting are taken at the near end of the vehicle, and the far end is extended by the lane line equation, combined with the above lane line equation fitting method, the error of the lane line at the far end is more serious. Figure 1 As shown in the example curve, it can also be seen that when the curvature of the curve is smaller, the lane line fitting is less affected, and when the curvature of the curve is larger, the lane line fitting is more affected and the deviation is larger.

[0040] Ax 2 +By 2 +Cx+Dy-E=0 (1)

[0041] Ax 3 +Bx 2 +Cx+D=y (2)

[0042] In the above equations (1) and (2), A, B, C, D, and E are all equation coefficients.

[0043] Based on this, a solution is needed to alleviate the fitting error at the far end of the lane line.

[0044] According to an embodiment of the present invention, a lane line fitting method embodiment is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0045] In this embodiment, a lane line fitting method is provided, which can be used for vehicle-side equipment. Figure 2 : is a flow chart of a lane line fitting method according to an embodiment of the present invention, the process includes the following steps:

[0046] Step S101, acquiring a lane image;

[0047] Step S102, extracting a lane line graphic of a preset length from the lane image, wherein the preset length is the length calculated from the current position of the vehicle and is smaller than the actual length of the lane;

[0048] Step S103, taking points at equal distances from the lane line graph to obtain a number of lane line fitting points;

[0049] Step S104, using the lane line fitting points to fit and calculate the current lane line equation;

[0050] Step S105, controlling the vehicle to travel based on the current lane line equation, and when the vehicle travels to a preset position, returning to the step of acquiring the lane image, fitting the lane line equation of the next lane line of a preset length, wherein the preset position is a set position close to the end point of the current preset length

[0051] Specifically, the embodiment of the present invention first obtains a lane image when fitting lane lines. The lane image refers to the road image taken in front of the vehicle. When the vehicle is driving on the current road, it is necessary to fit the lane lines on both sides of the road, so as to obtain the mathematical equation of the lane line. After acquiring the lane image, the embodiment of the present invention extracts a lane line graphic of a preset length from the lane image through a lane line perception algorithm, so as to obtain a lane line segmentation map. The lane line segmentation map refers to an image that only displays the lane lines without displaying other parts and objects in the scene. The lane line graphic is the specific shape of the lane line. The lane line perception algorithm can use edge detection technology, mainly by detecting the edges in the road image to identify the lane line. Common edge detection technologies include Sobel operator, Prewitt operator, Roberts operator and Canny operator. The lane line perception algorithm can also use a machine learning model, mainly by extracting the semantic features of the image to achieve an end-to-end effect. For example, the SCNN algorithm can be used to extract the lane line. In the embodiment of the present invention, the specific type of the lane line perception algorithm used is not specifically limited, but the lane line perception algorithm should be limited to an accuracy of lane line output higher than 90%, and its mean intersection and union ratio is above 90% (mean intersection and union ratio is a commonly used evaluation indicator in semantic segmentation tasks, which is used to measure the similarity between the model prediction results and the real labels). When the lane lines in pixel units in the image are converted into physical scales in reality according to the calibration relationship, the accuracy of the physical distance of the actual lane lines can be guaranteed, thereby ensuring the accuracy of positioning and vehicle safety.

[0052] It should be noted that the preset length defined in the embodiment of the present invention is the length calculated from the current position of the vehicle, and the lane line graphic of the preset length is intercepted from the lane image, in order to avoid the problem that the equation at the far end of the lane is obtained by extension when the lane line is fitted. The embodiment of the present invention only intercepts the shorter lane line from the current position of the vehicle, so that only the equation of the intercepted preset length is fitted. Whenever the vehicle approaches the end point of the currently intercepted preset length (for example, a preset position 1m, 2m or 3m before the end point of the preset length, this embodiment only gives an example of the preset position and is not limited to the position 1m, 2m or 3m before the end point of the preset length), the lane line equation of the next preset length is fitted again, so that the lane line fitted each time is as close to the near end as possible, and the equation at the far end of the lane line takes effect through re-fitting.

[0053] Afterwards, the embodiment of the present invention takes points at equal intervals from the intercepted lane line graph, for example, taking a point every 3 meters until the end of the lane line graph, thereby obtaining a number of lane line fitting points. Usually, too large an equal distance number N will affect the calculation efficiency. Usually, an equal distance number 10 is selected for point taking. For example, for a lane line of 40 meters in length, 40 / 10=4, a point is taken every 4 meters. Finally, the obtained lane line fitting points are used to obtain the current lane line equation through fitting calculation, ensuring that the equation parameters at each position of the intercepted lane line are obtained through fitting.

[0054] Since most points of traditional lane line fitting are taken at the near end of the vehicle, and the far end is obtained by extending the lane line equation, the present invention optimizes the traditional lane line fitting scheme by only two principles: short-distance estimation to intercept lane lines and equidistant point selection. By equidistantly selecting points at the far end of the lane, the weight value of the fitted lane line data at the far end is ensured without changing the algorithm and architecture, and the stability of the final calculated curve result is guaranteed. The lane line calculated by this method reduces the far-end error of the lane line fitting result, and to a certain extent solves the problem caused by the lack of y in the fitting equation. 2 The fitting error caused by the basis term.

[0055] For example Figure 3 As shown in FIG. 1 , it is a rendering of an embodiment of the present invention. In the case of a curve with the same curvature, the lane line output of the traditional method has a significant curvature change at the far end, which causes a sudden change in the lateral orientation prediction of the positioning system. Figure 4 As shown in FIG. 1 , when the solution provided by the embodiment of the present invention is not used, the lane line is offset at the far end, causing the vehicle to be unable to locate in the middle of the road, and there may be a risk of misalignment or even collision with the guardrail. However, by using the technical solution provided by the embodiment of the present invention, it can be found that even if the road curvature is small, the lane line at the far end of the road still changes smoothly and the change is small, so that the lateral positioning of the entire vehicle is as stable as possible, so as to achieve the desired effect. Figure 4This allows the vehicle to maneuver smoothly through curves without any bumps.

[0056] In some optional implementations, the above step S101 includes:

[0057] Step a1, calibrating the image acquisition device based on the reprojection error;

[0058] Step a2, using the image acquisition device calibrated with external parameters to acquire lane images.

[0059] Specifically, the reprojection error refers to the difference between the projection of the real three-dimensional space point on the image plane (i.e., the pixel point on the image) and the virtual pixel point obtained by calculation. The existence of this difference is due to various reasons that the calculated value is not completely consistent with the actual situation. In order to minimize this difference, it is necessary to minimize the sum of these differences to obtain the optimal camera pose parameters and the coordinates of the three-dimensional space point. The camera calibration is required to restore the object in the space using the image captured by the camera. [Image] = M [actual object], the matrix M is the parameter of the camera, usually, the camera extrinsic parameter M needs to be obtained by calculation, and the process of solving the parameter is called extrinsic parameter calibration. The embodiment of the present invention calculates the camera extrinsic parameters with the minimum reprojection error as the constraint condition of parameter calibration. For example, after the image acquisition device is calibrated, the output of the reprojection error for the lane line needs to meet the actual corresponding size within 0.1 to 0.2 lane widths. This is only an example and is not limited to this. In the above steps, the result of the external parameter calibration constrained by the reprojection error can also improve the accuracy of the physical distance of the actual lane line to ensure the accuracy of subsequent positioning and vehicle safety.

[0060] In particular, if the computing accuracy of the equipment at the rear end of the vehicle is relatively high and there is a certain allowable error range for vehicle control, the aforementioned reprojection error standard and the lane line perception algorithm mean intersection and union ratio standard can be appropriately relaxed, depending on the specific needs and determined according to the capabilities of the rear-end control.

[0061] In some optional implementations, the above step S102 includes:

[0062] Step b1, when the focal length of the image acquisition device is greater than 8 mm, a lane line pattern with a preset length of 40 m is selected;

[0063] Step b2, when the focal length of the image acquisition device is less than or equal to 8 mm, select a lane line graphic with a preset length not exceeding 30 m;

[0064] Step b3, when the focal length of the image acquisition device is less than or equal to 8 mm, and the positioning accuracy needs to be higher than the requirement of 0.2 times the width between two lane lines, select a lane line graphic with a preset length not exceeding 20 m.

[0065] Specifically, the embodiment of the present invention takes into account that the effective shooting distance of a general single camera is within 40m, while the effective distance of a camera with a small focal length should be reduced, and when the accuracy requirement of lane line fitting is higher, a clearer shooting image should be required. Therefore, the embodiment of the present invention also monitors the device parameters of the image acquisition device in real time. When the focal length of the image acquisition device is greater than 8mm, a lane line graphic with a preset length of 40m can be collected. When the focal length of the image acquisition device is less than or equal to 8mm, the camera is a small focal length camera. Only by selecting a lane line graphic with a preset length not exceeding 30m can the collected lane line be clearer and the far-end blurred part with a longer distance be discarded. In addition, when the focal length of the image acquisition device is less than or equal to 8mm, and the positioning accuracy needs to be higher than the requirement of 0.2 times the width between the two lane lines, the accuracy requirement of lane line fitting is higher, and the camera focal length is small. At this time, it is necessary to collect a lane line graphic with a smaller preset length, that is, not more than 20m, to ensure the clarity of the lane line graphic, thereby ensuring the accuracy of subsequent lane line fitting.

[0066] According to the above technical means, as the effective focal length of the camera decreases and the accuracy increases, the distant route at a longer distance in the image may be deformed due to the camera's own parameters, resulting in inaccurate fitted lane lines. Therefore, the present invention adaptively reduces the length of each fitted lane line according to the reduction of the effective focal length of the camera and the improvement of the accuracy, so that the lane line fitting process is completed as close to the vehicle as possible, thereby further improving the accuracy of the fitted lane line in each frame.

[0067] In some optional implementations, the above step S104 includes:

[0068] Step c1, determining the basis of the target equation according to the number of curves in the lane line graph;

[0069] Step c2, substituting the graphic coordinates of the lane line fitting points into the target equation to obtain the lane line equation in the image coordinate system;

[0070] Step c3, according to the conversion relationship between the image coordinate system and the vehicle coordinate system, the lane line equation in the image coordinate system is converted to obtain the current lane line equation in the vehicle coordinate system.

[0071] Specifically, the embodiments of the present invention flexibly adjust the basis order and basis type of the lane line equation based on the complexity of the lane line's curves. When the lane line figure has a small number of curves, simple bases such as the 1st order, 2nd order, and 3rd order can be used. For lane line figures with a large number of curves, the curves are more complex, so a higher order or more complex basis can be used as the equation for fitting, thereby improving the fitting accuracy for more complex lane lines.

[0072] For example, in a specific application embodiment, when the number of bends is less than 2, a 3rd-order basis is used; when the number of bends is greater than or equal to 2, a 5th-order basis or a Legendre basis is used.

[0073] That is, when there is only one curve, for the arc-shaped lane line, the present invention adopts x, x 2 and x 3 The target equation is created by substituting the lane line fitting points into the target equation to obtain the fitted lane line equation. If the number of curves is greater than or equal to 2 and the lane line shape is more complex, the present invention can use a 5th-order basis or Legendre basis to create the corresponding equation, thereby facilitating the fitting of complex curves. For example, the 5th-order basis includes x, x 2 、x 3 、x 4 and x 5 , the Legendre basis is not described in detail. According to the above technical means, the embodiment of the present invention adopts a more complex 5th-order basis or Legendre basis for continuous complex curves with more curves, and a 3rd-order basis can be used for fitting for single curvature curves, so as to select a more suitable formula basis according to the number of curves, which can speed up the fitting efficiency when the basis is simpler, and improve the fitting accuracy of the lane line equation when the basis is more complex, taking into account the accuracy and efficiency of lane line fitting.

[0074] Finally, according to the conversion relationship between the image coordinate system and the vehicle coordinate system, the lane line equation fitted in the image coordinate system is converted, and the coefficients corresponding to the equation are adjusted to the values ​​in the vehicle coordinate system to obtain the lane line equation in the vehicle coordinate system, which can be used for vehicle positioning tasks and other tasks.

[0075] In some optional implementations, the above step c3 includes:

[0076] Step d1, converting the lane line equation in the image coordinate system to the vehicle coordinate system according to the conversion relationship between the image coordinate system and the vehicle coordinate system;

[0077] Step d2, adjusting the parameters of the converted lane line equation based on the lateral constraint to obtain the adjusted lane line equation, the lateral constraint is used to constrain the first lane line figure generated corresponding to the lane line equation to fall within the lane composed of the actual lane line figure.

[0078] Specifically, the embodiment of the present invention also performs lateral constraints on the fitted lane line equation based on the actual lane line. After the lane line equation is calculated, the fitted lane line graphic can be drawn based on the lane line equation, and the fitted lane line graphic is compared with the actual lane line to calculate the difference between the two lines. In this embodiment, the lateral constraint is that the first lane line graphic generated corresponding to the lane line equation cannot exceed the edge of the actual lane line graphic, so that the fitted lane line needs to be within the width of the lane composed of the actual lane lines. Based on this, when the vehicle is laterally positioned laterally, it can be ensured that the vehicle will not touch the edge of the actual lane line, thereby improving the reliability of subsequent vehicle positioning.

[0079] Finally, in some optional embodiments, such as Figure 5 As shown, the edge position of the lane is determined based on the current lane line equation calculated in the previous steps, and then the width of the lane is determined using the lane edge position. When the vehicle is driving, the vehicle is positioned to be centered in the lane, thereby determining the lateral distance between the vehicle and the lane edge when the vehicle drives to various positions in the lane, thereby completing accurate, reliable and safe lateral positioning.

[0080] In this embodiment, a lane line fitting device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and will not be repeated here. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0081] This embodiment provides a lane line fitting device, such as Figure 6 As shown, including:

[0082] An image acquisition module 601 is used to acquire lane images;

[0083] A pattern recognition module 602 is used to extract a lane line pattern of a preset length from the lane image, where the preset length is the length calculated from the current position of the vehicle and is smaller than the actual length of the lane;

[0084] The data point acquisition module 603 is used to obtain points at equal distances from the lane line graph to obtain a number of lane line fitting points;

[0085] An equation fitting module 604 is used to calculate the current lane line equation by using the lane line fitting point fitting;

[0086] The loop control module 605 is used to control the vehicle driving based on the current lane line equation, and when the vehicle drives to a preset position, returns to the step of acquiring the lane image to fit the lane line equation of the next preset length lane line, where the preset position is a set position close to the end point of the current preset length.

[0087] In some optional implementations, the image acquisition module 601 includes:

[0088] An external parameter calibration unit, used for performing external parameter calibration on the image acquisition device according to the reprojection error;

[0089] The image acquisition unit is used to acquire lane images using an image acquisition device calibrated with external parameters.

[0090] In some optional implementations, the graphic recognition module 602 includes:

[0091] A first length setting unit, used for selecting a lane line graphic with a preset length of 40m when the focal length of the image acquisition device is greater than 8mm;

[0092] A second length setting unit, used for selecting a lane line graphic with a preset length not exceeding 30m when the focal length of the image acquisition device is less than or equal to 8mm;

[0093] The third length setting unit is used to select a lane line graphic with a preset length not exceeding 20m when the focal length of the image acquisition device is less than or equal to 8mm and the positioning accuracy needs to be higher than the requirement of 0.2 times the width between two lane lines.

[0094] In some optional implementations, the equation fitting module 604 includes:

[0095] A basis determination unit, used for determining a basis of a target equation according to the number of curves in the lane line graph;

[0096] A calculation unit, used for substituting the graphic coordinates of the lane line fitting points into the target equation to obtain the lane line equation in the image coordinate system;

[0097] The conversion unit is used to convert the lane line equation in the image coordinate system according to the conversion relationship between the image coordinate system and the vehicle coordinate system to obtain the current lane line equation in the vehicle coordinate system.

[0098] In some optional implementations, the base determination unit includes:

[0099] A low-order basis determination unit, for adopting a 3rd-order basis when the number of curves is less than 2;

[0100] The high-order basis determination unit is used to adopt a 5th-order basis or a Legendre basis when the number of curves is greater than or equal to 2.

[0101] In some optional implementations, the conversion unit includes:

[0102] An initial conversion unit, used for converting the lane line equation in the image coordinate system to the vehicle coordinate system according to the conversion relationship between the image coordinate system and the vehicle coordinate system;

[0103] The parameter adjustment unit is used to adjust the parameters of the converted lane line equation based on the lateral constraint to obtain the adjusted current lane line equation. The lateral constraint is used to constrain the first lane line figure generated corresponding to the lane line equation to fall within the lane composed of the actual lane line figure.

[0104] In some optional embodiments, the device further comprises:

[0105] A position determination module, used to determine the lane edge position based on the current lane line equation;

[0106] The lateral positioning module is used to determine the lateral distance between the vehicle and the lane edge when the vehicle is traveling by using the lane edge position.

[0107] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0108] The lane line fitting device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0109] The embodiment of the present invention also provides a computer device having the above Figure 6 The lane line fitting device shown.

[0110] See also Figure 7 , Figure 7 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 7 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 7 A processor 10 is taken as an example.

[0111] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0112] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0113] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0114] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0115] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0116] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0117] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0118] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A lane line fitting method, characterized in that: The method comprises: Get lane image; Extracting a lane line graphic of a preset length from the lane image, wherein the preset length is a length calculated from a current position of the vehicle and is smaller than an actual length of the lane; Taking points at equal distances from the lane line graph to obtain a number of lane line fitting points; Calculate the current lane line equation by using the lane line fitting point fitting; The vehicle is controlled based on the current lane line equation, and when the vehicle reaches a preset position, the step of acquiring the lane image is returned to fit the lane line equation of the next lane line of a preset length, wherein the preset position is a set position close to the end point of the current preset length.

2. The method according to claim 1, characterized in that: The acquiring of the lane image comprises: Perform extrinsic calibration on the image acquisition device based on the reprojection error; The lane image is acquired by using the image acquisition device after extrinsic parameter calibration.

3. The method according to claim 2, characterized in that The step of extracting a lane line graphic of a preset length from the lane image comprises: When the focal length of the image acquisition device is greater than 8 mm, a lane line pattern with a preset length of 40 m is selected; When the focal length of the image acquisition device is less than or equal to 8 mm, a lane line pattern with a preset length not exceeding 30 m is selected; When the focal length of the image acquisition device is less than or equal to 8 mm, and the positioning accuracy needs to be higher than the requirement of 0.2 times the width between two lane lines, a lane line graphic with a preset length not exceeding 20 m is selected.

4. The method according to claim 1, characterized in that The method of calculating the current lane line equation by fitting the lane line fitting points includes: Determining a basis of a target equation according to the number of curves in the lane line graph; Substituting the graphic coordinates of the lane line fitting points into the target equation to obtain the lane line equation in the image coordinate system; According to the conversion relationship between the image coordinate system and the vehicle coordinate system, the lane line equation in the image coordinate system is converted to obtain the current lane line equation in the vehicle coordinate system.

5. The method according to claim 4, characterized in that The step of determining a basis of a target equation according to the number of curves in the lane line graphic comprises: When the number of bends is less than 2, a 3rd-order basis is used; When the number of bends is greater than or equal to 2, a 5th-order basis or a Legendre basis is used.

6. The method according to claim 4, characterized in that The converting the lane line equation in the image coordinate system according to the conversion relationship between the image coordinate system and the vehicle coordinate system to obtain the current lane line equation in the vehicle coordinate system includes: According to the conversion relationship between the image coordinate system and the vehicle coordinate system, the lane line equation in the image coordinate system is converted to the vehicle coordinate system; The converted lane line equation is parameter-adjusted based on the lateral constraint to obtain the adjusted current lane line equation, wherein the lateral constraint is used to constrain the first lane line figure generated corresponding to the lane line equation to fall within the lane composed of the actual lane line figure.

7. The method according to claim 1, characterized in that The method further comprises: Determine the lane edge position based on the current lane line equation; The lane edge position is used to determine the lateral distance between the vehicle and the lane edge when the vehicle is traveling.

8. A lane line fitting device, characterized in that: The device comprises: An image acquisition module, used for acquiring lane images; A pattern recognition module, used to extract a lane line pattern of a preset length from the lane image, wherein the preset length is the length calculated from the current position of the vehicle and is smaller than the actual length of the lane; A data point acquisition module, used to take points at equal distances from the lane line graph to obtain a number of lane line fitting points; An equation fitting module, used for fitting and calculating the current lane line equation using the lane line fitting points; A loop control module is used to control the vehicle driving based on the current lane line equation, and when the vehicle drives to a preset position, returns to the step of acquiring the lane image to fit the lane line equation of the next preset length lane line, wherein the preset position is a set position close to the end point of the current preset length.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 7.

Images