Lane line fitting method, electronic device and automobile

Abstract

The application relates to the technical field of automobiles and discloses a lane line fitting method, an electronic device and an automobile. The lane line fitting method comprises the following steps: acquiring image data; according to the order of point sets, calculating the average change rate of the slope and / or the approximate average curvature of every three adjacent points in the image data; according to the average change rate of the slope and / or the approximate average curvature, dividing the image data into a plurality of point sets to determine a plurality of line segments; performing polynomial fitting on the plurality of line segments to obtain a lane line corresponding to each line segment; after obtaining the lane line corresponding to each line segment, connecting every two adjacent line segments based on a Bezier curve to determine a lane line corresponding to the image data. According to the application, the point sets are divided into a plurality of line segments, the segments are polynomially fitted, and then the adjacent two line segments are connected based on the Bezier curve, so that the accuracy of lane line fitting can be improved, the smoothness of the lane curve can be ensured, and the display effect of the lane line can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of automotive electronics, and in particular to a lane line fitting method, an electronic device and a vehicle. BACKGROUND

[0002] Head-up display (HUD) assisted driving is a new research field in the automotive industry in recent years. HUD has limited popularity at present, and is more commonly used in airplanes or high-end sedans. In China, ordinary vehicles are basically not equipped with HUD. However, as people's interest in future technology gradually increases, augmented reality (AR) technology is gradually entering the public's field of vision. AR-HUD, which is a combination of AR and HUD, has a new market. AR is a technology that superimposes a three-dimensional model on a real scene in real time, and adds auxiliary explanations to the actual scene to make the scene more specific and concrete. AR is commonly used in teaching, medical treatment and display to facilitate the public's understanding and operation. AR-HUD displays the auxiliary explanations of the real world on the HUD, and then transmits them to the human eye. This is augmented reality head-up display (AR-HUD).

[0003] At present, in the field of AR-HUD technology, a polynomial fitting method is mainly used to fit lane lines. The least squares method is used to calculate a fitting formula in a set of points, and then points fitted by the formula are calculated. However, the defects of this technology are that the polynomial fitting method can only solve the one-to-one mapping of the point set with the horizontal coordinate greater than or equal to the vertical coordinate, and for the point set of the guide line with a large change rate, low-order polynomial fitting is insufficient, and high-order polynomial has a shock phenomenon at the end points of the definition interval, such as the Runge phenomenon, resulting in an undesirable display effect of the lane line. SUMMARY

[0004] Embodiments of the present application provide a lane line fitting method, an electronic device and a vehicle to solve the problem that the lane line fitting is insufficient in the prior art, and the fitting at the curve of the lane line is prone to a shock phenomenon, resulting in an undesirable display effect of the lane line.

[0005] To solve the above problems, embodiments of the present application provide the following technical solutions:

[0006] In a first aspect, the present application provides a lane line fitting method, comprising:

[0007] obtaining image data, wherein the image data comprises a plurality of point sets;

[0008] calculating an average change rate of the slope of every adjacent three points in the image data and / or an approximate average curvature of every adjacent three points in the image data according to the order of the point sets;

[0009] According to the average change rate of the slope and / or the approximate average curvature, the image data is divided into a plurality of point sets to determine a plurality of line segments;

[0010] The plurality of line segments are polynomially fitted to obtain a lane line corresponding to each line segment;

[0011] After obtaining the lane line corresponding to each line segment, each adjacent two line segments are connected based on a Bezier curve to determine a lane line corresponding to the image data.

[0012] In some embodiments, the average change rate of the slope of each adjacent three points in the image data is calculated, including:

[0013] The initial coordinate values of each adjacent three points in the image data are obtained, wherein the three points include a first point, a second point and a third point;

[0014] The slopes of two line segments corresponding to adjacent two points among the three points are calculated respectively;

[0015] According to the slopes, the average change rate of the slopes of the two line segments corresponding to the adjacent two points among the three points is calculated.

[0016] In some embodiments, according to the average change rate of the slope, the image data is divided into a plurality of point sets to determine a plurality of line segments, including:

[0017] If the average change rate of the slope is greater than a preset slope threshold, the second point of the three points is taken as the first point of the adjacent three points in the next calculation, the third point of the three points is taken as the second point of the adjacent three points in the next calculation, and the next point of the third point is taken as the third point of the adjacent three points in the next calculation to calculate a new change rate of the slope;

[0018] Meanwhile, the first point of the adjacent three points is taken as the last point of a previous point set, and the third point of the adjacent three points is taken as the first point of a next point set to determine a plurality of line segments.

[0019] In some embodiments, the approximate average curvature of each adjacent three points in the image data is calculated, including:

[0020] The initial coordinate values of each adjacent three points in the image data are obtained, wherein the three points include a first point, a second point and a third point;

[0021] The angle difference of two line segments corresponding to adjacent two points among the three points is calculated, wherein the angle difference is less than 180 degrees;

[0022] According to the angle difference, the approximate average curvature of the two line segments corresponding to the adjacent two points among the three points is calculated.

[0023] In some embodiments, the image data is divided into a plurality of point sets according to the approximate average curvature to determine a plurality of line segments, comprising:

[0024] If the approximate average curvature is greater than a preset distance threshold, the second point of the three points is taken as the first point of the next three adjacent points, the third point of the three points is taken as the second point of the next three adjacent points, and the next point of the third point is taken as the third point of the next three adjacent points to calculate a new approximate average curvature.

[0025] Meanwhile, the first point of the three adjacent points is taken as the last point of the previous point set, and the third point of the three adjacent points is taken as the first point of the next point set to determine a plurality of line segments.

[0026] In some embodiments, the image data is divided into a plurality of point sets to determine a plurality of line segments, further comprising:

[0027] Suppose the point set includes N points, N is an integer and N≥4, then:

[0028] The first angle value of the first point and the second point, the second angle value of the second point and the third point, the third angle value of the third point and the fourth point, …, the N-1 angle value of the N-1 point and the N point are calculated;

[0029] The sum of the difference values of the second angle value, the third angle value, …, the N-1 angle value to the first angle value is averaged to obtain a difference value average;

[0030] If the difference value average is greater than a preset average threshold, the point set corresponding to the N-1 points before the N point is divided into a line segment, and the next segmentation is started from the N point to determine a plurality of line segments.

[0031] In some embodiments, the plurality of line segments are polynomial fitted to obtain a lane line corresponding to each line segment, comprising:

[0032] The difference absolute value of the first point and the last point in the plurality of line segments is calculated, wherein the difference absolute value includes a horizontal coordinate difference absolute value and a vertical coordinate difference absolute value;

[0033] If the horizontal coordinate difference absolute value is less than the vertical coordinate difference absolute value, the horizontal coordinate and the vertical coordinate of each point on the line segment are swapped, and each point is input into a matrix of polynomial fitting;

[0034] Based on matrix operation, a lane line corresponding to each line segment is obtained.

[0035] In some embodiments, after obtaining the lane line corresponding to each line segment, each adjacent two line segments are connected based on a Bezier curve, including:

[0036] obtaining initial coordinate values of end points of each adjacent two line segments, wherein the end points include a first end point and a second end point;

[0037] obtaining initial vectors of the end points, and calculating an intersection point of the initial vectors;

[0038] calculating a middle point of the curve line segment according to the intersection point and a second order Bezier curve formula;

[0039] performing polynomial fitting on the curve line segment according to the middle point to obtain a lane line corresponding to the curve line segment, so as to connect each adjacent two line segments.

[0040] In some embodiments, the obtaining of the initial vectors of the end points and the calculating of the intersection point of the initial vectors include:

[0041] obtaining initial vectors of the end points, wherein the initial vectors include a first initial vector and a second initial vector;

[0042] calculating an intersection point of the first initial vector and the second initial vector.

[0043] In a second aspect, an embodiment of the present application provides an electronic device, including:

[0044] at least one processor; and

[0045] a memory connected with the at least one processor in communication; wherein

[0046] the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform part or all of the method as described in the first aspect.

[0047] In a third aspect, an embodiment of the present application provides an automobile, including:

[0048] the electronic device as described in the second aspect.

[0049] The beneficial effects of the embodiments of the present application are: different from the prior art, the lane line fitting method, the electronic device and the automobile provided by the embodiments of the present application, the lane line fitting method comprises: acquiring image data; according to the order of the point set, calculating the average change rate of the slope of each adjacent three points in the image data and / or the approximate average curvature of each adjacent three points in the image data; according to the average change rate of the slope and / or the approximate average curvature, the image data is divided into several point sets to determine several line segments; polynomial fitting is performed on the several line segments to obtain the lane line corresponding to each line segment; after obtaining the lane line corresponding to each line segment, connecting each adjacent two line segments based on the Bezier curve to determine the lane line corresponding to the image data.

[0050] The present application can improve the accuracy of lane line fitting, ensure the smoothness of lane curve, and further improve the display effect of lane line by dividing the point set into several line segments, polynomial fitting each segment, and connecting adjacent two line segments based on the Bezier curve. BRIEF DESCRIPTION OF DRAWINGS

[0051] One or more embodiments are illustrated by way of example with reference to the accompanying drawings, which are schematic and not intended to be limiting of the embodiments, and in which like reference numerals designate similar elements in the figures and wherein the use of "for example", "e.g.", "of the embodiments", "an embodiment", "one embodiment", "some embodiments", "apparent to those skilled in the art", "clear to those skilled in the art" and "apparent from" and like phrases in the description denotes different possibilities of the embodiments and do not mean that a specific set of embodiments is necessarily used, and wherein the figures are not necessarily drawn to scale, unless otherwise indicated.

[0052] Figure 1 is a schematic diagram of an application environment provided by the embodiments of the present application;

[0053] Figure 2 is a flowchart of a lane line fitting method provided by the embodiments of the present application;

[0054] Figure 3 is Figure 2 is a detailed flowchart of step S22 in

[0055] Figure 4 is Figure 2 is a detailed flowchart of step S23 in

[0056] Figure 5 is a flowchart of another division of several point sets provided by the embodiments of the present application;

[0057] Figure 6 is Figure 2 is a detailed flowchart of step S24 in

[0058] Figure 7 is Figure 2 is a detailed flowchart of step S25 in

[0059] Figure 8 is Figure 7A detailed flowchart of step S252 is shown in the figure;

[0060] Figure 9 A schematic diagram of curve fitting is provided by the embodiment of the present application;

[0061] Figure 10 A schematic diagram of the structure of an electronic device is provided by the embodiment of the present application;

[0062] Figure 11 A schematic diagram of the structure of an automobile is provided by the embodiment of the present application. DETAILED DESCRIPTION

[0063] To make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0064] It should be noted that each feature in the embodiments of the present application can be combined with each other if there is no conflict, and all falls within the protection scope of the present application. In addition, although the functional modules are divided in the device schematic diagram, and the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order from the module division in the device or the order in the flowchart. In addition, the "first", "second", "third" and the like used herein do not limit the data and execution order, but only distinguish the same items or similar items with basically the same function and effect.

[0065] Unless otherwise defined, all technical and scientific terms used in the present application have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used in the specification of the present application are only for the purpose of describing the specific embodiments and are not used to limit the present application. The term "and / or" used in the present application includes any and all combinations of one or more related listed items.

[0066] In addition, the technical features involved in each embodiment of the present application described below can be combined with each other as long as there is no conflict.

[0067] Please refer to Figure 1 , Figure 1 A schematic diagram of an application environment is provided by the embodiment of the present application;

[0068] As Figure 1As shown, the application environment includes: a head-up display system 10 and a driving assistance system 20, both of which are arranged in a car and are in communication connection. The communication connection can be wired or wireless. When the communication connection is wireless, the head-up display system 10 and the driving assistance system 20 can be connected through a certain communication protocol, for example, through Bluetooth to establish wireless communication and realize data interaction between the head-up display system 10 and the driving assistance system 20.

[0069] The head-up display system 10, also known as a parallel display system (HUD, Head Up Display), is a car function device for projecting the moving speed of the guide line, navigation and other important driving information onto the windshield in front of the driver. It can enable the driver to obtain the driving information of the car while keeping a straight view of the front, thereby improving the safety of the driver's driving. In the embodiment of the application, the head-up display system 10 can at least be used to display the lane lines and the guide lines between the two sides of the lane, and is in communication connection with the driving assistance system 20. The guide line is used to suggestively guide the driver to drive.

[0070] The driving assistance system 20 is a system for outputting driving information such as driving speed, navigation information, fuel information and other driving information in the car. It includes but is not limited to a lane keeping assistance system, an automatic parking assistance system, a brake assistance system, a reversing assistance system and a driving assistance system, etc. In the embodiment of the application, it at least includes a driving assistance system. The system is provided with various sensors and cameras, which can be used to collect image data of the road surface, the image data including a set of lane line points, and transmit the road surface information to the head-up display system 10 after processing, so as to display the lane lines and the guide lines of the lane to the driver and realize the navigation function.

[0071] The embodiment of the application provides a lane line fitting method, which can be executed by the head-up display system 10 described above, and the car further includes a driving assistance system.

[0072] Please refer to Figure 2 , Figure 2 is a flowchart of a lane line fitting method provided by the embodiment of the application;

[0073] As Figure 2 shown, the lane line fitting method includes:

[0074] Step S21: acquiring image data, wherein the image data includes a plurality of point sets.

[0075] In the embodiments of the present application, first, the image of the road on which the vehicle travels is collected by the camera on the driving assistance system, and then a feature recognition algorithm is used to recognize the lane lines in the image. For example, first, the image data is converted into a grayscale image, then the image is filtered to remove noise in the image, and then edge detection is achieved through binarization processing, and according to the characteristics of the lane lines, such as a straight line or a curve that penetrates at least one side of the image, the point set on the lane lines on both sides of the lane is extracted.

[0076] Step S22: According to the order of the point set, the average change rate of the slope of each adjacent three points in the image data and / or the approximate average curvature of each adjacent three points in the image data is calculated.

[0077] After the point set on both sides of the lane is extracted, the point set can be segmented according to the bending degree of the lane lines on both sides. Specifically, in the embodiments of the present application, the slope change rate of each adjacent three points and the approximate average curvature of each adjacent three points are calculated in the order of the points in the point set to determine whether the line segment corresponding to the point set is greater than a threshold value, and the point set greater than the threshold value is segmented, so as to improve the fitting accuracy in the subsequent fitting process.

[0078] Step S23: According to the average change rate of the slope and / or the approximate average curvature, the image data is divided into a plurality of point sets to determine a plurality of line segments.

[0079] On the one hand, the point set on the lane line can be calculated in the order of the points to calculate the slope of the two line segments corresponding to the adjacent two points between each adjacent three points, and then the average change rate of the slope of the two line segments corresponding to the adjacent two points between the three points is calculated according to the slope, to determine whether the average change rate of the slope is greater than a threshold value. If it is greater than the threshold value, the middle point of the three points is discarded, and the middle point is stored as the next calculated point set, and the point set input into the vehicle is divided into a plurality of straight lines or curves.

[0080] On the other hand, the point set on the lane line can also be calculated in the order of the points to calculate the angle difference between the two line segments corresponding to the adjacent two points between each adjacent three points. It should be noted that the angle difference is less than 180 degrees, and then the approximate average curvature of the two line segments corresponding to the adjacent two points between the three points is calculated according to the angle difference, to determine whether the approximate average curvature is greater than a threshold value. If it is greater than the threshold value, the middle point of the three points is discarded, and the middle point is stored as the next calculated point set, and the point set input into the vehicle is divided into a plurality of straight lines or curves.

[0081] In another aspect, the point set can also be divided into N points, N being an integer and N≥4, a first angle value of the first point and the second point, a second angle value of the second point and the third point, a third angle value of the third point and the fourth point, and an (N-1)th angle value of the (N-1)th point and the Nth point are calculated; a mean value of the sum of the difference between the second angle value, the third angle value, and the (N-1)th angle value and the first angle value is obtained; it is determined whether the mean value of the difference is greater than a preset mean value threshold; if yes, the N-1 points before the Nth point are divided into a line segment, and the next segmentation is performed from the Nth point, so as to divide the point set of the passing car into several line segments. For example, the first angle value of the line connected by the first point and the second point is 30°, the second angle value of the line connected by the second point and the third point is 40°, and the third angle value of the line connected by the third point and the fourth point is 50°. The mean value of the sum of the difference between the first angle value is ((40°-30°)+(50°-30°)) / 2, which is equal to 15°. If the mean value of the difference is greater than the mean value threshold 10, the next segmentation is performed from the third point.

[0082] Step S24: polynomial fitting is performed on the several line segments to obtain a lane line corresponding to each line segment.

[0083] In the embodiment of the present application, when the polynomial fitting is performed on the several line segments, further, the absolute value of the difference between the horizontal coordinate and the vertical coordinate of the first point and the last point in the several line segments is calculated, so as to determine whether the line segment is a horizontal line segment or a vertical line segment. Specifically, it is determined whether the absolute value of the horizontal coordinate difference is less than the absolute value of the vertical coordinate difference. If yes, the horizontal coordinate and the vertical coordinate of each point on the line segment are swapped, and each point is input into a matrix of the polynomial fitting. Based on the matrix operation, a lane line corresponding to each line segment is obtained.

[0084] For example, the polynomial fitting performs a matrix operation of a monomial cubic equation on the multiple points. If the absolute value of the horizontal coordinate difference of a line segment is not less than the absolute value of the vertical coordinate difference, the line segment is a horizontal line segment. In this case, the horizontal coordinate and the vertical coordinate of each point on the line segment do not need to be swapped. Otherwise, the line segment is a vertical line segment. In this case, the horizontal coordinate and the vertical coordinate of each point on the line segment are swapped. In this way, the one-to-one mapping of the vertical coordinate to the horizontal coordinate is compensated.

[0085] Step S25: after the lane line corresponding to each line segment is obtained, each adjacent two line segments are connected based on a Bezier curve, so as to determine a lane line corresponding to the image data.

[0086] After the several line segments are obtained by polynomial fitting, there are several line segments between two adjacent line segments of the several line segments, and the change rates of the several line segments are greater than the threshold value. In the embodiment of the present application, the initial coordinate values of the two adjacent end points are selected at one end close to the middle arc part between the point sets of the segmented line segments with the change rates greater than the threshold value, two vectors are obtained respectively, the intersection point of the two vectors is obtained by the method of vector intersection, the segmented line segments are fitted by substituting the two end points and the intersection point into the Bezier curve formula, and the lane line corresponding to the image data is determined.

[0087] The embodiment of the present application provides a lane line fitting method, which comprises the following steps: acquiring image data; calculating the average change rate of the slope of every three adjacent points in the image data and / or the approximate average curvature of every three adjacent points in the image data according to the order of the point sets; dividing the image data into several point sets according to the average change rate of the slope and / or the approximate average curvature, so as to determine several line segments; performing polynomial fitting on the several line segments, so as to obtain the lane line corresponding to each line segment; and connecting every two adjacent line segments based on a Bezier curve after the lane line corresponding to each line segment is obtained, so as to determine the lane line corresponding to the image data.

[0088] The embodiment of the present application can improve the accuracy of lane line fitting, ensure the smoothness of the lane curve, and further improve the display effect of the lane line by dividing the point sets into several line segments, fitting each segmented line segment less than a threshold value by polynomial fitting, and connecting two adjacent line segments greater than the threshold value based on a Bezier curve.

[0089] Specifically, refer to Figure 3 , Figure 3 is Figure 2 a detailed flowchart of step S22 in

[0090] As shown in Figure 3 , the step S22: calculating the average change rate of the slope of every three adjacent points in the image data and / or the approximate average curvature, comprises but is not limited to the following steps:

[0091] Step S221: acquiring the initial coordinate values of every three adjacent points in the image data.

[0092] In the embodiment of the present application, the initial coordinate values of every three adjacent points in the image data are acquired, wherein the three points comprise a first point, a second point and a third point.

[0093] Step S222: calculating the slope and / or the angle difference of two line segments corresponding to two adjacent points between the three points respectively.

[0094] In the embodiment of the present application, on the one hand, the slope of two line segments corresponding to two adjacent points between the three points is calculated.

[0095] On the other hand, the angle difference of the two line segments corresponding to the adjacent two points among the three points can also be calculated first.

[0096] Step S223: According to the slope and / or the angle difference, the average change rate of the slope of the two line segments corresponding to the adjacent two points among the three points and / or the approximate average curvature is calculated.

[0097] In the embodiments of the present application, on the one hand, the average change rate of the slope of the two line segments corresponding to the adjacent two points among the three points is calculated, and specifically, the following formula can be referred to:

[0098]

[0099] where (x1, y1) is the coordinate of the first point, (x2, y2) is the coordinate of the second point, (x3, y3) is the coordinate of the third point, k 1, is the slope of the straight line formed by the first point and the second point, k 2, is the slope of the straight line formed by the second point and the third point, and Δk is the average change rate of the slope.

[0100] On the other hand, the approximate average curvature of the two line segments corresponding to the adjacent two points among the three points can also be calculated. Specifically, the angle difference of the angle value of the two adjacent line segments among the three points is less than 180°, for example, one of the two line segments corresponds to an angle value of 30 degrees, and the other line segment corresponds to an angle value of -170 degrees, at this time, the angle difference of the two line segments is 200 degrees, and according to the theorem, the angle difference of 160 degrees is taken for further calculation.

[0101] And then divided by the distance between the first point and the third point among the three points, and specifically, the following formula can be referred to:

[0102]

[0103] where (x1, y1) is the coordinate of the first point, (x2, y2) is the coordinate of the second point, (x3, y3) is the coordinate of the third point, arctan(k 1, ) is the angle value of the straight line formed by the first point and the second point, arctan(k 2, ) is the angle value of the straight line formed by the second point and the third point, k 1, is the slope of the straight line formed by the first point and the second point, k 2, is the slope of the straight line formed by the second point and the third point. arctan(k 2, )-arctan(k 1, ) is the angle difference, is the distance between the first point and the third point among the three points, and Δu is the approximate average curvature.

[0104] The application can determine the change rate degree of the line point set by calculating the average change rate of the slope and / or the approximate average curvature between adjacent points, so as to distinguish the point set with a large change rate degree in the subsequent process, and then fit the adjacent two line segments by using the Bezier curve, thereby improving the fitting accuracy of the lane line, ensuring the smoothness of the lane curve, and further improving the display effect of the lane line.

[0105] Specifically, please refer to Figure 4 , Figure 4 is Figure 2 the detailed flowchart of step S23 in

[0106] As shown in Figure 4 , the step S23: according to the average change rate of the slope and / or the approximate average curvature, the image data is divided into several point sets to determine several line segments, including but not limited to the following steps.

[0107] Step S231: obtaining the average change rate of the slope and / or the approximate average curvature of the two line segments corresponding to the adjacent two points among the three points.

[0108] Specifically, the average change rate of the slope and / or the approximate average curvature of the two line segments corresponding to the adjacent two points among the three points is obtained.

[0109] Step S232: judging whether the average change rate of the slope and / or the approximate average curvature is greater than the preset slope threshold and / or the preset distance threshold.

[0110] Wherein, the average change rate of the slope is the difference between the slope of the straight line formed by the first point and the second point of the adjacent three points and the slope of the straight line formed by the second point and the third point, divided by the value of the difference between the first point and the third point; the approximate average curvature is the difference between the angle value of the straight line formed by the first point and the second point of the adjacent three points and the angle value of the straight line formed by the second point and the third point, that is, the angle difference, taking the angle difference of the angle less than 180°, and then dividing by the value of the distance between the first point and the third point.

[0111] Wherein, since the reference thresholds of the average change rate of the slope and the approximate average curvature are different, the slope threshold should be determined according to the average change rate of the slope. The distance threshold should be determined according to the approximate average curvature.

[0112] If no, go to step S233: take the second point of the three points as the first point of the next three points, take the third point of the three points as the second point of the next three points, and take the next point of the third point as the third point of the next three points, and go to step S231: obtain the average change rate of the slopes of the two line segments corresponding to the two adjacent points of the three points and / or the approximate average curvature.

[0113] If the average change rate of the slopes of the two line segments corresponding to the two adjacent points of the three points and / or the approximate average curvature is less than or equal to the preset slope threshold and / or the preset distance threshold, it indicates that the change rate between the three adjacent point sets is small, and the average change rate of the slopes of the two line segments corresponding to the two adjacent points of the new three points is calculated again, wherein the first point of the next three points is the second point of the previous three points, the second point of the next three points is the third point of the previous three points, and the third point of the next three points is the next point of the third point of the previous three points.

[0114] If yes, go to step S234: take the first point of the three adjacent points as the last point of the previous point set, and take the third point of the three adjacent points as the first point of the next point set, to determine the line segments.

[0115] If the average change rate of the slopes of the two line segments corresponding to the two adjacent points of the three points and / or the approximate average curvature is greater than the preset slope threshold and / or the preset distance threshold, it indicates that the change rate between the three adjacent point sets is large, and the point set is segmented, specifically, the first point of the divided three points is taken as the last point of the previous point set, to complete the segmentation of a line segment, the second point is discarded, and the third point is taken as the first point of the next point set, wherein the first point of the next point set is also taken as the first point of the next three points, to continue to calculate the average change rate of the slopes of the two line segments corresponding to the two adjacent points of the new three points and / or the approximate average curvature.

[0116] The application divides the point set greater than the threshold, discards the second point of the three points in the point set, to achieve the purpose of segmenting the point set, and then fits the missing part between the two adjacent line segments segmented above by using the Bezier curve formula, which can improve the accuracy of lane line fitting, ensure the smoothness of the lane curve, and further improve the display effect of the lane line.

[0117] Specifically, refer to Figure 5 , Figure 5 which is another flowchart for dividing a plurality of point sets provided by the embodiment of the application;

[0118] As Figure 5As shown, the image data is divided into several point sets to determine several line segments, including:

[0119] Step S51: assuming that the point set includes N points, N is an integer and N≥4.

[0120] In the embodiments of the present application, in addition to dividing the point sets according to the above-mentioned manner of calculating the average change rate of the slope and / or the approximate average curvature of each adjacent three points, the point sets can also be divided by calculating the average value of the sum of the difference between the angle of the first line segment corresponding to the first point and the angle of the kth line segment corresponding to each adjacent k points.

[0121] Step S52: calculating the first angle value of the first point and the second point, the second angle value of the second point and the third point, the third angle value of the third point and the fourth point, …, and the N-1th angle value of the N-1th point and the Nth point.

[0122] Specifically, the first angle value of the straight line formed by the first point and the second point, the second angle value of the straight line formed by the second point and the third point, the third angle value of the straight line formed by the third point and the fourth point, …, and the N-1th angle value of the straight line formed by the N-1th point and the Nth point are calculated.

[0123] Step S53: taking the average of the sum of the difference between the second angle value, the third angle value, …, and the N-1th angle value and the first angle value to obtain the average difference value.

[0124] The difference value is accumulated and averaged to obtain the average difference value. Specifically, reference can be made to the following formula:

[0125] X=(arctan(k 2, )-arctan(k 1, )+arctan(k 3, )-arctan(k 1, )+…

[0126] +arctan(k n-1,n )-arctan(k 1, )) / n-1

[0127] wherein X is the average difference value, arctan(k 1, ) is the first angle value, arctan(k 2, ) is the second angle value, arctan(k 3, ) is the third angle value, and arctan(k n-1,n ) is the N-1th angle value.

[0128] The second angle value and the first angle value are subtracted to obtain arctan(k 2,) - arctan(k 1, ), wherein arctan(k 2, ) - arctan(k 1, ) is an angle less than 180°, and similarly, the third angle value is subtracted from the first angle value to obtain arctan(k 3, ) - arctan(k 1, ), which is an angle less than 180°, and so on, the N-1th angle value is subtracted from the first angle value to obtain arctan(k n-1,n ) - arctan(k 1, ), which is an angle less than 180°.

[0129] Step S54: determining whether the average value of the difference values is greater than a preset average value threshold.

[0130] In the embodiments of the present application, the preset average value threshold can be 10, 15, 20, and the specific value is determined according to the experimental results and experimental data in actual application, which is not limited herein.

[0131] If yes, go to step S55: dividing the point set corresponding to the N-1 points before the Nth point into a line segment, and starting the next segmentation from the Nth point to determine a plurality of line segments.

[0132] If no, go to step S53.

[0133] The present application also provides a method for segmenting a point set, which determines whether the average value of the difference values of the angles of the N-1th line segment and the first line segment is greater than an average value threshold, and if yes, divides the point set corresponding to the N-1 points before the Nth point into a line segment, and starts the next segmentation from the Nth point to achieve the purpose of segmenting the point set.

[0134] Specifically, please refer to Figure 6 , Figure 6 is Figure 2 a detailed flowchart of step S24 in

[0135] As shown in Figure 6 , this step S24: polynomial fitting is performed on a plurality of line segments to obtain a lane line corresponding to each line segment, including but not limited to the following steps.

[0136] Step S241: calculating the absolute value of the difference between the first point and the last point in the plurality of line segments.

[0137] It can be understood that polynomial fitting is essentially a matrix operation on a plurality of points, for example, a matrix operation is performed on the point set in the plurality of line segments to obtain the best constant values a, b, c, d, so that Y = aX^3 + bX^2 + cX + d.

[0138] In this embodiment, if the X-direction distance of the line segment is greater than the Y-direction distance, that is, the absolute value of the difference between the x-coordinates of the first and last points on the line segment is greater than the absolute value of the difference between the y-coordinates of the first and last points, then the line segment is oriented towards horizontal development; if the X-direction distance of the line segment is less than the Y-direction distance, that is, the absolute value of the difference between the x-coordinates of the first and last points on the line segment is less than the absolute value of the difference between the y-coordinates of the first and last points, then the line segment is oriented towards vertical development. In this case, the x-coordinates and y-coordinates of each point are swapped, and then the polynomial fitting matrix is ​​input to obtain the optimal a, b, c, d, such that X = aY^3 + bY^2 + cY + d.

[0139] This method can reduce the error of the fitted line segments and improve the accuracy of lane line fitting.

[0140] Step S242: Determine whether the absolute value of the difference in the horizontal coordinates is less than the absolute value of the difference in the vertical coordinates.

[0141] Specifically, before performing polynomial fitting on several line segments, it is first determined whether the absolute value of the difference between the x-coordinates of the first and last points on the line segment is less than the absolute value of the difference between the y-coordinates of the first and last points.

[0142] If not, proceed to step S243: input the polynomial fitting matrix for each point.

[0143] If so, proceed to step S244: swap the x and y coordinates of each point on the line segment, and input each point into the polynomial fitting matrix.

[0144] Step S245: Based on matrix operations, obtain the lane lines corresponding to each line segment.

[0145] Specifically, matrix operations are performed on multiple points on several line segments to fit the lane lines corresponding to those line segments.

[0146] For details, please see Figure 7 , Figure 7 yes Figure 2 A detailed flowchart of step S25;

[0147] like Figure 7 As shown, step S25 involves obtaining the lane line corresponding to each line segment, then connecting each pair of adjacent line segments based on a Bezier curve, including but not limited to the following steps.

[0148] Step S251: Obtain the initial coordinate values ​​of the end points of each pair of adjacent line segments.

[0149] Specifically, initial coordinate values of end points of each of the two adjacent line segments are obtained, wherein the end points include a first end point and a second end point.

[0150] In the embodiment of the present application, after the corresponding point set is calculated according to the polynomial, two adjacent end points of each of the two adjacent line segments are obtained from the point set of the two adjacent line segments, two vectors are then calculated respectively, the intersection point of the two vectors is calculated, and finally the intermediate point is calculated according to the two end points and the intersection point by using the second-order Bezier curve formula. In this way, the two adjacent line segments greater than the threshold value can be connected, so as to complete the fitting of the entire lane line, improve the accuracy of the lane line fitting, ensure the smoothness of the lane curve, and further improve the display effect of the lane line.

[0151] Step S252: obtaining an initial vector of the end point and calculating the intersection point of the initial vector.

[0152] Specifically, the adjacent point of the end point in the two adjacent line segments is taken, and the initial vector is made according to the end point and the adjacent point of the end point.

[0153] Specifically, please refer to Figure 8 , Figure 8 is Figure 7 the detailed flowchart of step S252;

[0154] As shown in Figure 8 , this step S252: obtaining an initial vector of the end point and calculating the intersection point of the initial vector, includes but is not limited to the following steps.

[0155] Step S2521: obtaining an initial vector of the end point.

[0156] The initial vector of the end point is obtained, wherein the initial vector includes a first initial vector and a second initial vector. Specifically, the adjacent point of the end point in the two adjacent line segments is taken, and the initial vector is made according to the end point and the adjacent point of the end point, wherein the initial vector includes the first initial vector made by the previous line segment in the two adjacent line segments and the second initial vector made by the subsequent line segment in the two adjacent line segments.

[0157] Step S2522: calculating the intersection point of the first initial vector and the second initial vector.

[0158] According to the first initial vector and the second initial vector, the intersection point of the two vectors is calculated.

[0159] Step S253: calculating the intermediate point of the curve line segment according to the intersection point and the second-order Bezier curve formula.

[0160] Specifically, the formula for calculating the intermediate point according to the intersection point and the second-order Bezier curve formula is as follows:

[0161] p = (1-t 2p0+2t(1-t)p1+t 2 p2

[0162] wherein, p is the intermediate point, p0 is the first end point, p2 is the second end point, p1 is the intersection of the first initial vector and the second initial vector, and t is in the range of [0, 1].

[0163] Step S254: According to the intermediate point, polynomial fitting is performed on the curve segment to obtain the lane line corresponding to the curve segment, so as to connect every two adjacent segments.

[0164] Specifically, the fitted point is the intermediate point. For example, the horizontal coordinates of the first end point, the second end point and the intersection are brought into the Bezier curve formula to fit the motion trajectory of the intermediate point in the horizontal coordinate, and the vertical coordinates of the first end point, the second end point and the intersection are brought into the Bezier curve formula to fit the motion trajectory of the intermediate point in the vertical coordinate, wherein the motion trajectories of the horizontal coordinate and the vertical coordinate of the intermediate point constitute the curve segment.

[0165] Specifically, please refer to Figure 9 , Figure 9 which is a schematic diagram of curve fitting provided by the embodiment of the present application;

[0166] As shown in Figure 9 , it is assumed that the line segments include N line segments, and N is an integer.

[0167] The two adjacent line segments are the N-1th line segment and the Nth line segment.

[0168] The end points include the first end point on the N-1th line segment and the second end point on the Nth line segment.

[0169] The intersection is the intersection of the initial vector of the first end point and the initial vector of the second end point.

[0170] The curve segment is obtained by fitting the motion trajectory of the intermediate point by substituting the end points and the intersection into the second-order Bezier curve formula.

[0171] In the embodiment of the present application, the initial vector of the end point is obtained, the intersection of the initial vector is calculated, the intermediate point of the curve segment is calculated according to the intersection and the second-order Bezier curve formula, and the curve segment corresponding to the lane line is obtained by polynomial fitting according to the intermediate point, so as to connect every two adjacent segments. The curve segment supplements the gap between the two adjacent segments segmented above, thereby completing the fitting of the entire lane line, improving the accuracy of the lane line fitting, ensuring the smoothness of the lane curve, and further improving the display effect of the lane line.

[0172] Please refer to Figure 10 , Figure 10 which is a structural schematic diagram of an electronic device provided by the embodiment of the present application;

[0173] As shown in Figure 10 , the electronic device 100 comprises: at least one processor 101; and a memory 102 connected with the at least one processor 101, Figure 10 The memory 102 stores instructions executable by the at least one processor 101, and the instructions are executed by the at least one processor 101 to enable the at least one processor 101 to perform the lane line fitting method described above. Figures 2 to 8 The processor 101 and the memory 102 can be connected by a bus or other means, Figure 10 For example, the connection by the bus is taken as an example.

[0174] In the embodiments of the present application, the electronic device 100 includes but is not limited to:

[0175] (1) Mobile communication device: the feature of this kind of device is to have mobile communication function, and to provide voice and data communication as the main target. This kind of electronic device includes: smart phone (such as iPhone), multimedia phone, functional phone, and low-end phone, etc.

[0176] (2) Mobile personal computer device: this kind of device belongs to the category of personal computer, has the functions of calculation and processing, and generally has the characteristics of mobile Internet. This kind of electronic device includes: PDA, MID and UMPC device, etc., such as iPad.

[0177] (3) Portable entertainment device: this kind of device can display and play video content, and generally has the characteristics of mobile Internet. This kind of device includes: video player, handheld game console, and smart toy and portable car navigation device.

[0178] (4) Other electronic devices with video playback function and Internet function.

[0179] The memory 102 is a kind of non-volatile computer readable storage medium, which can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as program instructions / modules corresponding to the lane line fitting method in the embodiments of the present application. The processor 101 performs various functional applications and data processing of the server by running the non-volatile software programs, instructions and modules stored in the memory 102, that is, implements the lane line fitting method in the method embodiment.

[0180] The memory 102 can include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required by a function; the data storage area can store data created by use of the lane line fitting device, etc. In addition, the memory 102 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state memory device. In some embodiments, the memory 102 can optionally include a memory disposed remotely with respect to the processor 101, and these remote memories can be connected to the lane line fitting device through a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0181] One or more modules are stored in the memory 102, and when executed by the one or more processors 101, perform the lane line fitting method in any of the above method embodiments, for example, perform the method steps described above. Figures 2 to 8

[0182] The above product can perform the method provided by the embodiments of the present application, has the function modules and beneficial effects corresponding to the execution method. Technical details not described in detail in the embodiments can refer to the method provided by the embodiments of the present application.

[0183] The embodiments of the present application also provide a non-volatile computer readable storage medium, and the computer readable storage medium stores computer executable instructions, and the computer executable instructions are executed by one or more processors, for example, perform the method steps described above. Figures 2 to 8

[0184] The embodiments of the present application also provide a computer program product, including a computer program stored on a non-volatile computer readable storage medium, and the computer program includes program instructions, and when the program instructions are executed by a computer, the computer executes the lane line fitting method in any of the above method embodiments, for example, performs the method steps described above. Figures 2 to 8

[0185] Please refer to Figure 11 , Figure 11 is a structural schematic diagram of an automobile provided by the embodiments of the present application,

[0186] As shown in Figure 11 , the automobile 110 includes an electronic device frame 111, and the electronic device frame 111 includes an electronic device capable of executing the lane line fitting method in any of the above method embodiments, for example, performing the method steps described above. Figures 2 to 8

[0187] ​​​​The method for fitting a lane line provided in the embodiment of the application comprises the following steps: first, acquiring image data; then, calculating a slope average change rate of each three adjacent points in the image data and / or an approximate average curvature of each three adjacent points in the image data according to the order of the point set; then, dividing the image data into a plurality of point sets according to the slope average change rate and / or the approximate average curvature, so as to determine a plurality of line segments; then, performing polynomial fitting on the plurality of line segments, so as to obtain a lane line corresponding to each line segment; and finally, connecting each two adjacent line segments based on a Bezier curve, so as to determine a lane line corresponding to the image data. The method provided in the embodiment of the application can improve the accuracy of lane line fitting, ensure the smoothness of the lane curve, and further improve the display effect of the lane line.

[0188] It should be noted that the apparatus embodiments described above are only illustrative, and the units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, i.e., they can be located in one place or distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the embodiment.

[0189] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software and a general hardware platform, and of course, it can also be implemented by hardware. Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above-mentioned embodiment methods. The storage medium can be a magnetic disc, an optical disc, a read-only memory (ROM) or a random access memory (RAM), etc.

[0190] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; under the idea of the present application, the technical features in the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other changes of different aspects of the present application as described above. In order to be brief, they are not provided in detail; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method of fitting a lane line, characterized by, The method comprises: acquiring image data, wherein the image data comprises a plurality of point sets; calculating a slope average change rate of each adjacent three points in the image data and / or an approximate average curvature of each adjacent three points in the image data according to an order of the point sets; dividing the image data into a plurality of point sets according to the slope average change rate and / or the approximate average curvature to determine a plurality of line segments; performing polynomial fitting on the plurality of line segments to obtain a lane line corresponding to each line segment; connecting each adjacent two line segments based on a Bezier curve to determine a lane line corresponding to the image data after obtaining the lane line corresponding to each line segment; the dividing the image data into a plurality of point sets to determine a plurality of line segments further comprises: assuming that the point set comprises N points, N is an integer and N≥4, then: calculating a first angle value of a first point and a second point, a second angle value of the second point and a third point, a third angle value of the third point and a fourth point, …, an N-1 angle value of an N-1 point and an N point; taking an average of a sum of differences between the second angle value, the third angle value, …, and the N-1 angle value and the first angle value to obtain a difference average value; if the difference average value is greater than a preset average threshold value, then dividing N-1 points before the N point into a point set of a line segment, and starting the next segmentation from the N point to determine a plurality of line segments.

2. The method of claim 1, wherein, the calculating the slope average change rate of each adjacent three points in the image data comprises: acquiring initial coordinate values of each adjacent three points in the image data, wherein the three points comprise a first point, a second point and a third point; respectively calculating slopes of two line segments corresponding to adjacent two points among the three points; calculating a slope average change rate of the two line segments corresponding to the adjacent two points among the three points according to the slopes.

3. The method of claim 2, wherein, the dividing the image data into a plurality of point sets according to the slope average change rate to determine a plurality of line segments comprises: if the slope average change rate is greater than a preset slope threshold value, then taking the second point among the three points as a first point of adjacent three points in the next calculation, taking the third point among the three points as a second point of adjacent three points in the next calculation, and taking a next point of the third point as a third point of adjacent three points in the next calculation to calculate a new slope change rate; at the same time, taking the first point among the adjacent three points as a last point of a previous point set, and taking the third point among the adjacent three points as a first point of a next point set to determine a plurality of line segments.

4. The method of claim 1, wherein, the calculating the approximate average curvature of each adjacent three points in the image data comprises: acquiring initial coordinate values of each adjacent three points in the image data, wherein the three points comprise a first point, a second point and a third point; calculating an angle difference of two line segments corresponding to adjacent two points among the three points, wherein the angle difference is less than 180 degrees; calculating an approximate average curvature of the two line segments corresponding to the adjacent two points among the three points according to the angle difference.

5. The method of claim 4, wherein, The image data is divided into a plurality of point sets according to the approximate average curvature to determine a plurality of line segments, including: If the approximate average curvature is greater than a preset distance threshold, a second point of the three points is taken as a first point of three adjacent points in the next calculation, a third point of the three points is taken as a second point of three adjacent points in the next calculation, and a next point of the third point is taken as a third point of three adjacent points in the next calculation to calculate a new approximate average curvature; Meanwhile, a first point of the three adjacent points is taken as a last point of a previous point set, and a third point of the three adjacent points is taken as a first point of a next point set to determine a plurality of line segments.

6. The method of claim 1, wherein, The plurality of line segments are polynomial fitted to obtain a lane line corresponding to each of the line segments, including: An absolute value of a difference between a first point and a last point of the plurality of line segments is calculated, wherein the absolute value of the difference includes an absolute value of a horizontal coordinate difference and an absolute value of a vertical coordinate difference; If the absolute value of the horizontal coordinate difference is less than the absolute value of the vertical coordinate difference, a horizontal coordinate and a vertical coordinate of each point on the line segment are swapped, and the each point is input into a matrix of polynomial fitting; Based on matrix operation, a lane line corresponding to each line segment is obtained.

7. The method of claim 1, wherein, After obtaining the lane line corresponding to each line segment, each two adjacent line segments are connected based on a Bezier curve, including: Initial coordinate values of end points of each two adjacent line segments are obtained, wherein the end points include a first end point and a second end point; Initial vectors of the end points are obtained, and an intersection of the initial vectors is calculated; A middle point of the curve line segment is calculated according to the intersection and a second-order Bezier curve formula; The curve line segment is polynomial fitted according to the middle point to obtain a lane line corresponding to the curve line segment, so as to connect each two adjacent line segments.

8. The method of claim 7, wherein, The initial vectors of the end points are obtained, and the intersection of the initial vectors is calculated, including: The initial vectors of the end points are obtained, wherein the initial vectors include a first initial vector and a second initial vector; The intersection of the first initial vector and the second initial vector is calculated.

9. An electronic device, comprising: including: at least one processor; and a memory connected in communication with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8.

10. An automobile characterized by comprising: including: the electronic device of claim 9.

Images