Road surface height estimation method, device, equipment, medium and vehicle

Abstract

The invention relates to the technical field of vehicle auxiliary driving, and discloses a road surface height estimation method, device and equipment, a medium and a vehicle. Extracting sampling points of lane lines on two sides of the lane from the lane image; determining straightway segments and curve segments according to the sampling points; fitting a first straight lane line and a curve lane line on the two sides of the lane respectively for the straight segment and the curve segment; converting the curve lane line into a second straight lane line; splicing the first straight lane line and the second straight lane line to obtain a complete lane line; according to the method, the curve lane line is correspondingly converted into the straight lane line, paired points are sampled from the lane lines on the two sides to determine the lane width, and then the actual corresponding height of each position of the lane is determined through the function relation between the lane line width and the ground height. Therefore, the problem that the road surface height cannot be accurately estimated under the curve condition is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of vehicle assisted driving, and in particular to a road height estimation method, device, equipment, medium and vehicle. Background Art

[0002] At present, in order to reduce the cost of road height detection, the relevant technology has proposed a method for road height recognition based on images. This method uses images to capture the road, then collects lane lines, and samples pairs of points from the lane lines on both sides according to the distance in front of the camera to determine the lane width. Due to the distortion of the image captured by the camera, the lane width will appear to change in the image when going uphill and downhill, but the lane lines are parallel in the actual scene, so the lane width is fixed. Based on the parallel constraint of the lane lines, the functional relationship between the lane line width and the ground height is used to determine the actual corresponding height of each position in the lane. However, the current method only considers straight roads and is not applicable to curves, because the paired points sampled from the lane lines on both sides of the camera to capture the distance in front are not necessarily the point pairs corresponding to the actual lanes, resulting in errors in the subsequent estimation of road height. Therefore, it is an urgent problem to accurately estimate the road height for both curves and straight roads. Summary of the invention

[0003] In view of this, the present invention provides a road height estimation method, device, equipment, medium and vehicle to solve the problem of inaccurate road height estimation based on images.

[0004] In a first aspect, the present invention provides a method for estimating road surface height, the method comprising: acquiring a lane image; extracting sampling points of lane lines on both sides of the lane from the lane image; determining straight segments and curved segments based on the sampling points; fitting a first straight lane line and a curved lane line on both sides of the lane for the straight segments and the curved lane line respectively; converting the curved lane line into a second straight lane line; splicing the first straight lane line and the second straight lane line to obtain a complete lane line; and estimating the image road surface height based on the complete lane line.

[0005] According to the above technical means, the curved lane lines are converted into straight lane lines, and then the lane width is determined by sampling pairs of points from the lane lines on both sides based on the distance in front of the camera. Then, the functional relationship between the lane line width and the ground height is used to determine the actual height corresponding to each position of the lane, thereby solving the problem that the road surface height cannot be accurately estimated in curved conditions.

[0006] In some optional embodiments, converting a curved lane line into a second straight lane line includes: drawing tangents for sampling points on the curved lane line; pairing sampling points parallel to the tangents on both sides of the curve, and drawing the curved lane line as the second straight lane line based on the tangent distance between the paired points.

[0007] According to the above technical means, tangents are drawn for each sampling point on the curve. The sampling points on the curve can be paired under the condition that the tangents at corresponding positions with the same curvature of the curve are necessarily parallel. The curve is then gradually redrawn as a straight road using the distance between the paired points, and the road width is estimated using the drawn distance. This solves the problem that the current visual algorithm cannot accurately estimate the width of a curve, thereby improving the accuracy of estimating the road surface height.

[0008] In some optional embodiments, determining straight segments and curved segments based on sampling points includes: clustering the sampling points to obtain multiple sampling point sets for representing each lane line; dividing the current sampling point set into multiple sampling point segments; determining the normal of the current sampling point segment; calculating the angle between each sampling point and the corresponding normal in the current sampling point segment; calculating the angle difference between adjacent angles in the current sampling point segment, and calculating the angle sum of all angle differences; when the angle sum is greater than a preset angle threshold, judging that the current sampling point segment is a curved segment; when the angle sum is less than or equal to the preset angle threshold, judging that the current sampling point segment is a straight segment.

[0009] According to the above technical means, the sampling point set is divided into multiple sampling point segments to represent a lane section, and a normal line is roughly calibrated for the sampling point shape of each lane section. The angles of other sampling points to the calibrated normal line are calculated, and then the difference is calculated using the angles of adjacent sampling points. Finally, these calculated differences are added together to obtain the angle sum. If the current lane section is a straight road, then the adjacent angles are closer, the calculated difference is smaller, and the angle sum is smaller, and will not be greater than the preset angle threshold. If the current lane section is a curve, the adjacent angle difference is larger, the calculated difference is larger, and the angle sum is larger, and will be greater than the preset angle threshold. The method based on this embodiment can realize the determination of whether the sampling point belongs to a straight road section or a curve section, and then use different equations to fit the lane line based on the determination result, which can improve the fitting accuracy of subsequent lane lines.

[0010] In some optional embodiments, the first straight lane line and the curved lane line on both sides of the lane are fitted for the straight segment and the curved segment respectively, including: fitting the sampling point segment corresponding to the straight segment based on a linear equation to obtain the first straight lane line; fitting the sampling point segment corresponding to the curved segment based on a cubic polynomial to obtain the curved lane line.

[0011] According to the above technical means, the fitting accuracy of straight lane lines and curved lane lines can be further improved by using linear equations to fit straight lines and using cubic polynomials to fit curves.

[0012] In some optional embodiments, image road surface height estimation is performed based on the complete lane line, including: from the complete lane line, a group of data point pairs are collected at preset distances according to the direction of road advancement; the plane pixel coordinates of each data point in the data point pair are determined; the mapping relationship between the plane and the space is determined, and the mapping relationship is used to project the plane pixel coordinates of each data point into the space to obtain the spatial world coordinates; the lateral coordinate expression of each data point in the space is calculated based on the plane pixel coordinates of each data point and the mapping relationship, and the lateral coordinate expression is used to calculate the coordinates of each data point in the lane width direction, and the lateral coordinate expression includes the height coordinate of the data point in the space as the height parameter to be solved; the two lateral coordinate expressions of the current data point pair are subtracted to obtain a lane width expression, and the lane width expression is used to represent the lane width; the height parameter in the lane width expression is calculated based on the lane width expression and the lane line parallel constraint, and the height parameter is used as the road surface height at the location of the current data point.

[0013] According to the above technical means, the mapping relationship between the lane width and the road surface height at different positions on the complete lane line is used to calculate the specific road surface height, and finally the road surface height is allocated to the location of the corresponding collected data point, thereby achieving accurate road surface height estimation for both straight roads and curves.

[0014] In some optional embodiments, before determining the mapping relationship between the plane and the space, the method also includes: calculating the point distance between two data points in each group of data point pairs; determining whether each point distance falls within a preset width interval; if it falls within the preset width interval, continuing to execute the step of determining the mapping relationship between the plane and the space; if it does not fall within the preset width interval, returning to the step of collecting a group of data point pairs from the complete lane line at a preset distance according to the direction of road advancement.

[0015] According to the above technical means, because the width of each lane position should be basically the same, the embodiment of the present invention first analyzes whether the distances of paired data points are within a preset lane width range (preset width interval) based on the collected data points of the lane line before predicting the road surface height. If they do not fall on the curve, it is considered that the collected data points are more likely not points on the lane line, and should be resampled until the required position is met, thereby improving the accuracy of lane line fitting, and then improving the accuracy of subsequent road surface height estimation.

[0016] In some optional embodiments, before continuing to execute the step of determining the mapping relationship between the plane and the space if it falls within a preset width interval, the method also includes: obtaining the distances of several points closest to the current position of the vehicle, and calculating the average of the obtained point distances to obtain an average distance; calculating the difference between the remaining point distances and the average distance to obtain multiple distance differences; when each distance difference is less than a preset distance threshold, executing the step of continuing to execute the step of determining the mapping relationship between the plane and the space if it falls within a preset width interval, otherwise, returning to the step of collecting a set of data point pairs from the complete lane line at preset distances according to the direction of road advancement.

[0017] Based on the above technical means, it is further determined whether there are outliers in the data points collected on the lane line, so that resampling is performed when there are outliers, and road height estimation is continued only when there are no outliers, avoiding inaccurate calculation of road width and further reducing the problem of inaccurate road height estimation.

[0018] In a second aspect, the present invention provides a road surface height estimation device, which includes: an image acquisition module for acquiring a lane image; a sampling module for extracting sampling points of lane lines on both sides of the lane from the lane image; a road type recognition module for determining straight segments and curved segments based on the sampling points; a lane line fitting module for fitting a first straight lane line and a curved lane line on both sides of the lane for the straight segment and the curved segment respectively; a tangent fitting module for drawing tangents for the sampling points on the curved lane line; a curve-to-straight lane conversion module for pairing sampling points parallel to the tangents on both sides of the curve, and drawing the curved lane line as a second straight lane line based on the tangent distance between the paired points; a combination module for splicing the first straight lane line and the second straight lane line to obtain a complete lane line; and a height estimation module for performing image road surface height estimation based on the complete lane line.

[0019] In a third aspect, the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0020] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to cause a computer to execute the method of the first aspect or any corresponding embodiment thereof.

[0021] In a fifth aspect, the present invention provides a vehicle, comprising the computer device provided in the third aspect.

[0022] The technical solution provided by the present invention has the following advantages:

[0023] (1) According to the above technical means, the curved lane lines are converted into straight lane lines, and then the lane width is determined by sampling pairs of points from the lane lines on both sides based on the distance in front of the camera. Then, the functional relationship between the lane line width and the ground height is used to determine the actual height corresponding to each position of the lane, thereby solving the problem that the road surface height cannot be accurately estimated in curved conditions.

[0024] (2) According to the above technical means, tangents are drawn for each sampling point on the curve. The sampling points on the curve can be paired under the condition that the tangents at corresponding positions with the same curvature of the curve are necessarily parallel. The curve is then gradually redrawn as a straight road using the distance between the paired points. The drawn straight road is then used to estimate the road width, which solves the problem that the current visual algorithm cannot accurately estimate the width of the curve, thereby improving the accuracy of estimating the road surface height.

[0025] (3) According to the above technical means, a section of lane is represented by dividing the sampling point set into multiple sampling point segments, and a normal line is roughly calibrated for the sampling point shape of each section of lane. The angles of other sampling points to the calibrated normal line are calculated, and then the difference is calculated using the angles of adjacent sampling points. Finally, these calculated differences are added together to obtain the angle sum. If the current section of lane is a straight road, then the adjacent angles are closer, the calculated difference is smaller, and the angle sum is smaller, and will not be greater than the preset angle threshold. If the current section of lane is a curve, then the adjacent angle difference is larger, the calculated difference is larger, and the angle sum is larger, and will be greater than the preset angle threshold. The method based on this embodiment can realize the determination of whether the sampling point belongs to a straight road or a curve, and based on the determination result, the lane line is set using the painless equation, which can improve the fitting accuracy of subsequent lane lines.

[0026] (4) According to the above technical means, the fitting accuracy of straight lane lines and curved lane lines can be further improved by using linear equations to fit straight lines and cubic polynomials to fit curves.

[0027] (5) According to the above technical means, the specific road surface height is calculated by using the mapping relationship between the lane width and the road surface height at different positions on the complete lane line. Finally, the road surface height is assigned to the location of the corresponding collected data point, thereby achieving accurate road surface height estimation for both straight roads and curves.

[0028] (6) Based on the above technical means, the unreasonable lane width is further screened. Before predicting the road surface height, the embodiment of the present invention first analyzes whether the distances of paired data points are within a preset lane width range (preset width interval) based on the collected data points of the lane line. If they do not fall on the curve, it is considered that the collected data points are more likely not on the lane line and should be resampled until the required position is met, thereby improving the accuracy of lane line fitting and further improving the accuracy of subsequent road surface height estimation.

[0029] (7) Based on the above technical means, it is further determined whether there are outliers in the data points collected on the lane line, so that resampling is performed when there are outliers, and road height estimation is continued only when there are no outliers, thereby avoiding inaccurate calculation of road width and reducing the problem of inaccurate road height estimation. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0031] Figure 1 is a schematic diagram of the effect of photographing a lane according to an embodiment of the present invention;

[0032] Figure 2 is another schematic diagram of the effect of photographing a lane according to an embodiment of the present invention;

[0033] Figure 3 is a schematic diagram of the effect of collecting data points on a curve according to the relevant technology of an embodiment of the present invention;

[0034] Figure 4 is a schematic flow chart of a road height estimation method according to an embodiment of the present invention;

[0035] Figure 5 is a schematic diagram of a tangent line fitting for a curve according to an embodiment of the present invention;

[0036] Figure 6 is a projection schematic diagram according to an embodiment of the present invention;

[0037] Figure 7 is a schematic diagram of collecting data points on a lane line according to an embodiment of the present invention;

[0038] Figure 8 is another flowchart of a road height estimation method according to an embodiment of the present invention;

[0039] Fig. 9 is a schematic structural diagram of a road height estimation device according to an embodiment of the present invention;

[0040] Fig.10 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0042] like Figure 1 As shown in the figure, when a vehicle moves from a flat road to an uphill road, the width of the uphill lane is displayed wider due to the camera distortion. Figure 2 As shown, the lane width will appear narrower, but in fact the lane width should be constant. Related technologies collect paired data points from both sides of the road in the direction of the camera shooting, and then use the distance between the data points to calculate the lane width, and then estimate the road height based on a mapping relationship between the lane width and the road height. This method is more accurate for straight road scenes, but it is not suitable for curved scenes, such as Figure 3 As shown in FIG. 1 , the related technology collects paired data points from both sides of the road in the direction of camera shooting. Because the line connecting the collected data points is perpendicular to the direction of camera shooting, point pairs that are not actual corresponding points (such as point pairs A and B) may be collected. In fact, the corresponding point pairs on the image are not perpendicular to the camera shooting direction (such as point pairs A and C). Based on this, the lane width cannot be accurately estimated, which leads to inaccurate estimation of the road surface height.

[0043] According to an embodiment of the present invention, an embodiment of a road surface height estimation method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0044] In this embodiment, a method for estimating road height is provided. Figure 4 1 is a flow chart of a road height estimation method according to an embodiment of the present invention, the flow chart includes the following steps:

[0045] Step S101, acquiring a lane image;

[0046] Step S102, extracting sampling points of lane lines on both sides of the lane from the lane image;

[0047] Step S103, determining the straight segment and the curved segment according to the sampling points;

[0048] Step S104, fitting the first straight lane line and the curved lane line on both sides of the lane for the straight segment and the curved segment respectively;

[0049] Step S105, converting the curved lane line into a second straight lane line;

[0050] Step S106, splicing the first straight lane line and the second straight lane line to obtain a complete lane line;

[0051] Step S107, estimating the image road surface height based on the complete lane line.

[0052] Specifically, before estimating the road height, the embodiment of the present invention first extracts sampling points of lane lines on both sides of the lane from the acquired lane image. The specific lane line recognition method and sampling point sampling method are prior art and will not be described in detail in the present invention. Afterwards, the sampling points are used to determine the straight segment and the curved segment in the lane. For example, the lines connecting each adjacent sampling point are drawn, and then the inclination angles between the adjacent lines are determined in turn. If the differences between multiple inclination angles are all greater than the inclination angle threshold, it can be determined that the current lane segment is a curve, otherwise it is a straight road.

[0053] Afterwards, different mathematical equations are used to fit the lane lines of the straight segment and the curved segment. For example, the sampling points in the straight segment can be fitted using linear equation fitting, and the fitted straight lane lines are more accurate. The sampling point segments corresponding to the curved segment can be fitted using a cubic polynomial, and the nonlinear characteristics of the cubic polynomial can ensure that the curved lane lines are more accurate.

[0054] For the curved lane lines, the present invention uses a computer program to convert the curved lane lines into corresponding straight lane lines, and visually presents the curved lanes forcibly changed in direction to become new straight lanes (second straight lane lines) in the same direction as the original straight lanes (first straight lane lines). At this time, based on the distance in front of the camera, paired points are sampled from the lane lines on both sides to determine the lane width, and then the image road surface height estimation algorithm is called to use the functional relationship between the lane line width and the ground height to determine the actual corresponding height of each lane position, thereby solving the problem that the road surface height cannot be accurately estimated in the case of curved roads.

[0055] In some optional implementations, the above step S105 includes:

[0056] Step a1, drawing tangent lines for sampling points on the curved lane line;

[0057] Step a2, pairing the sampling points whose tangents are parallel on both sides of the curve, and drawing the curve lane line as the second straight lane line according to the tangent distance between the paired points.

[0058] Specifically, Figure 5As shown, the embodiment of the present invention can determine sampling points on the curved lane line, and then draw the tangent of each sampling point. On both sides of the curve, if there are two sampling points with corresponding positions, their tangents should be in a parallel relationship. Therefore, the embodiment of the present invention determines the paired sampling points on both sides of the curve by searching for tangents that are parallel to each other, and then obtains the tangent distance between multiple paired points. Then, according to the direction of the straight segment, the line segments between each paired point are drawn as the road width according to the distance between the paired points after the first straight lane line. Then, the endpoints of each line segment can be connected to draw the second straight lane line behind the first straight lane line, and the complete lane lines are all turned into straight roads. Subsequently, the error of the curve can be ignored through the image road height estimation algorithm, thereby improving the accuracy of road height estimation.

[0059] In some optional implementations, the above step S103 includes:

[0060] Step b1, clustering the sampling points to obtain a plurality of sampling point sets for representing each lane line;

[0061] Step b2, dividing the current sampling point set into a plurality of sampling point segments;

[0062] Step b3, determining the normal of the current sampling point segment;

[0063] Step b4, calculating the angle between each sampling point in the current sampling point segment and the corresponding normal line;

[0064] Step b5, calculating the angle difference between adjacent angles in the current sampling point segment, and calculating the angle sum of all angle differences;

[0065] Step b6, when the angle sum is greater than a preset angle threshold, determining that the current sampling point segment is a curve segment;

[0066] Step b7: when the sum of the angles is less than or equal to the preset angle threshold, it is determined that the current sampling point segment is a straight segment.

[0067] Specifically, because there are two lane lines on both sides of each lane, there may be multiple lanes on an image. Therefore, the present invention first clusters all the collected sampling points. Each clustered sampling point set can be used to represent a lane line. The clustering algorithm is a prior art and will not be described in detail in the present invention.

[0068] Afterwards, since a lane line may have a straight road or a curve, and there may be multiple curves and straight roads, it is necessary to perform lane line conversion for the curves separately. Therefore, the embodiment of the present invention cuts any sampling point set to obtain several small sampling point segments.

[0069] For each sampling point segment, first roughly draw the curve connecting the sampling points, and then arbitrarily determine a normal line of the curve as the normal line of the current sampling point segment. The normal line refers to a line on the plane that is perpendicular to the tangent line of the curve at a certain point. After that, for other sampling points on the curve, the angles between the sampling points and the normal line are calculated respectively. If the current sampling point segment represents a straight segment, the angles calculated for adjacent sampling points should be very close, so the calculated angle difference is also very small, and the sum of the angles obtained by summing up the subsequent angle differences is also small and will not be greater than the preset angle threshold. If the current lane is a curve, the adjacent angle difference is larger, the calculated difference is larger, and the sum of the angles is larger, and it will be greater than a certain preset angle threshold, as shown in the following formula.

[0070] ∑ i |α i -α i-1 |≤P

[0071] In the formula, α i Represents the angle between the i-th sampling point and the corresponding normal, α i-1 represents adjacent angles, and P represents the preset angle threshold.

[0072] This method can be used to identify whether the sampling point is a straight road or a curve, and determine whether the sampling point belongs to a straight road or a curve. Based on the determination result, different equations are used to fit the lane line, which can significantly improve the fitting accuracy of subsequent lane lines.

[0073] In some optional implementations, the above step S107 includes:

[0074] Step c1, collecting a set of data point pairs from the complete lane line at a preset distance according to the road advancing direction;

[0075] Step c2, determining the plane pixel coordinates of each data point in the data point pair;

[0076] Step c3, determining a mapping relationship between the plane and the space, the mapping relationship is used to project the plane pixel coordinates of each data point into the space to obtain the spatial world coordinates;

[0077] Specifically, after obtaining the complete lane line in the form of a straight road, the embodiment of the present invention collects a set of data point pairs at preset distances according to the direction of road advancement to measure the road width at preset distances, so that the road height at the location of the data point pair can be determined using the road width.

[0078] First, the plane pixel coordinates (u, v) of each data point need to be determined in the image plane, where u represents the lateral distance, which is used to measure the width offset from the lane center, and v represents the height distance, which is used to measure the height of the road surface.

[0079] After that, the mapping relationship between the plane and the space is determined according to the camera's intrinsic and extrinsic matrix. The effect of the mapping relationship can be referred to Figure 6 , as shown in the following formula:

[0080]

[0081] In the formula, A matrix representing the plane pixel coordinates of a data point, The matrix representing the spatial world coordinates of a data point, X w Indicates the horizontal coordinate in the road width direction, Y w The height coordinate representing the height of the road, Z w The depth coordinate representing the road depth in the camera shooting direction, that is, the longitudinal distance of the road; is the camera intrinsic parameter matrix, is the camera extrinsic matrix, f x 、f y 、c x and c y are the parameters of the extrinsic matrix, R is the rotation matrix, and t is the translation vector.

[0082] After transformation, the above formula can be obtained as follows.

[0083] Z w u=m 11 X w +m 12 Y w +m 13 Z w +m 14

[0084] Z w v=m 21 X w +m 22 Y w +m 23 Z w +m 24

[0085] In the above formula, m 11 、m 12 、m 13 、m 14 、m 21 、m 22 、m 23 、m 24 They are all elements of the left multiplied matrix obtained by multiplying the inner parameter matrix by the outer parameter matrix.

[0086] Step c4, calculating the lateral coordinate expression of each data point in space based on the plane pixel coordinates and the mapping relationship of each data point, the lateral coordinate expression is used to calculate the coordinate of each data point in the lane width direction, and the lateral coordinate expression includes the height coordinate of the data point in space as the height parameter to be solved;

[0087] Specifically, after obtaining the above equations, the horizontal coordinate expression can be derived by equation transformation, as shown below, which is used to calculate the horizontal coordinate X w .

[0088] X w =-(m 12 Y w +m 13 Z w +m 14 -Z w u) / m 11

[0089] X w =-(m 22 Y w +m 23 Z w +m 24 -Z w v) / m 21

[0090] Combining the above two equations, we get X w =f(Y w ,u,v)+m 32 Z w , it can be seen that the lateral coordinate expression is a function of the plane pixel coordinate, height coordinate and depth coordinate, where the height coordinate to be solved is Y w .

[0091] Step c5, subtracting the two lateral coordinate expressions of the current data point pair to obtain a lane width expression, which is used to represent the lane width.

[0092] Specifically, Figure 7 As shown, the two horizontal coordinates X1 and X2 corresponding to the horizontal direction can be substituted into the formula X by the two corresponding plane pixel coordinates (u1, v1) and (u2, v2) respectively. w =f(Y w ,u,v)+m 32 Z w Calculate, that is:

[0093] X1=f(Y1,u1,v1)+m 32 Z1

[0094] X2=f(Y2,u2,v2)+m32 Z2

[0095] Road width ΔX=X2-X1=f(Y1, u1, v1)-f(Y2, u2, v2).

[0096] Step c6, calculating the height parameter in the lane width expression based on the lane width expression and the lane line parallel constraint, and using the height parameter as the road surface height at the location of the current data point.

[0097] Specifically, the lane line parallel constraint means that the lane lines on both sides of the road are parallel, so the lane width is a known constant, and thus ΔX is known. This embodiment defines that the road height at the location of the data point pair is the same, so the road height to be solved h=Y1=Y2, so that in the final formula ΔX=f(h, u1, v1)-f(h, u2, v2), only the road surface height h is the parameter to be solved. Through the technical solution provided by the embodiment of the invention, different plane pixel coordinates (u, v) correspond to different road widths. Substituting the plane pixel coordinates of the collected data point pair into the above formula can calculate the corresponding road surface height, thereby realizing an image-based road surface height estimation method. Combined with the aforementioned embodiment for the conversion of curves, after converting the curve into a straight road, a group of data point pairs are collected at a preset distance according to the direction of the road, and the influence of the curve can be ignored, thereby ensuring that the subsequent estimated road surface height is more accurate.

[0098] In some optional implementation manners, before the above step c3, the method further includes:

[0099] Step d1, calculating the point distance between two data points in each group of data point pairs;

[0100] Step d2, determining whether the distance between each point falls within a preset width interval;

[0101] Step d3: if it falls within the preset width range, continue to execute the step of determining the mapping relationship between the plane and the space;

[0102] Step d4, if it does not fall within the preset width interval, return to the step of collecting a set of data point pairs from the complete lane line at preset distances according to the direction of the road.

[0103] Specifically, Figure 8As shown, the embodiment of the present invention also determines whether the distances between the data point pairs collected on the lane line all fall within the preset width interval, so as to limit the unreasonable road width within the preset width interval. For example, the preset width interval is [a, b], which means that the distance between the data point pairs is not less than a and not greater than b to meet the condition. When the distances between the points fall within the preset width interval, the subsequent steps of projecting the plane pixel points into space can be continued. Otherwise, it is determined that the sampling point is abnormal, and the sampling step is returned to re-obtain the data point pairs and analyze them, which further improves the reliability of the overall process.

[0104] In some optional implementation manners, before the above step d3, the method further includes:

[0105] Step e1, obtaining the distances of several points closest to the current position of the vehicle, and calculating the average value of the obtained point distances to obtain an average distance;

[0106] Step e2, calculating the difference between the remaining point distances and the average distance to obtain multiple distance differences;

[0107] Step e3, when all distance differences are less than the preset distance threshold, if they fall within the preset width interval, continue to execute the step of determining the mapping relationship between the plane and the space; otherwise, return to the step of collecting a set of data point pairs from the complete lane line at preset distances according to the direction of road advancement.

[0108] Specifically, the embodiment of the present invention further detects whether there are outliers in the data points collected on the lane line. Considering that the near-end data points captured by the camera are more reliable, for the point distance between each data point pair, the distances of several points closest to the current position of the vehicle are first obtained, for example, the distances of the first three points closest to the current position of the vehicle are obtained, and then the average of the obtained point distances is calculated, for example, the average of the distances of the first three points closest to the current position of the vehicle is calculated. After that, the difference between the distance of the remaining points far from the vehicle and the average distance is calculated to obtain multiple distance differences. Each distance difference is analyzed using a preset distance threshold. When each distance difference is less than the preset distance threshold, the subsequent steps are continued. Otherwise, it is considered that the data point corresponding to the distance difference not less than the preset distance threshold is unreasonable and belongs to an outlier. It is necessary to return to the step of collecting a set of data point pairs at a preset distance from the complete lane line according to the road forward direction, and re-collect the data points on the lane line. Through this scheme, resampling is performed when there are outliers, and the road height estimation is continued when there are no outliers, thereby avoiding inaccurate calculation of road width and reducing the problem of inaccurate road height estimation.

[0109] In this embodiment, a road height estimation device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0110] This embodiment provides a road height estimation device, such as Fig. 9 As shown, including:

[0111] An image acquisition module 801 is used to acquire a lane image;

[0112] A sampling module 802, used to extract sampling points of lane lines on both sides of the lane from the lane image;

[0113] A road type identification module 803 is used to determine a straight road segment and a curved road segment according to the sampling points;

[0114] A lane line fitting module 804 is used to fit the first straight lane line and the curved lane line on both sides of the lane for the straight segment and the curved segment respectively;

[0115] A curve lane conversion module 805, used to convert the curve lane line into a second straight lane line;

[0116] A combining module 806 is used to combine the first straight lane line and the second straight lane line to obtain a complete lane line;

[0117] The height estimation module 807 is used to estimate the image road surface height based on the complete lane line.

[0118] In some optional implementations, the curve line conversion module 805 includes:

[0119] A tangent fitting unit is used to draw tangents for sampling points on the curved lane line;

[0120] The second straight line drawing unit is used to pair the sampling points whose tangents are parallel on both sides of the curve, and draw the curve lane line as the second straight lane line according to the tangent distance between the paired points.

[0121] In some optional implementations, the road type identification module 803 includes:

[0122] A clustering unit, used to cluster the sampling points to obtain a plurality of sampling point sets used to represent each lane line;

[0123] A sampling point segment division unit, used for dividing a current sampling point set into a plurality of sampling point segments;

[0124] A normal determination unit, used to determine the normal of the current sampling point fragment;

[0125] An angle calculation unit, used to calculate the angle between each sampling point and the corresponding normal line in the current sampling point segment;

[0126] An angle sum calculation unit, used to calculate the angle difference between adjacent angles in the current sampling point segment, and calculate the angle sum of all angle differences;

[0127] A curve determination unit, configured to determine that the current sampling point segment is a curve segment when the angle sum is greater than a preset angle threshold;

[0128] The straight road determination unit is used to determine that the current sampling point segment is a straight road segment when the angle sum is less than or equal to a preset angle threshold.

[0129] In some optional implementations, the lane fitting module 804 includes:

[0130] A linear fitting unit, used for fitting the sampling point segment corresponding to the straight road segment based on a linear equation to obtain a first straight road lane line;

[0131] The nonlinear fitting unit is used to fit the sampling point segment corresponding to the curve segment based on the cubic polynomial to obtain the curve lane line.

[0132] In some optional implementations, the height estimation module 807 includes:

[0133] A data point pair collection unit, used to collect a set of data point pairs from the complete lane line at a preset distance according to the road advancing direction;

[0134] A plane coordinate unit, used to determine the plane pixel coordinates of each data point in a data point pair;

[0135] A projection unit, used to determine a mapping relationship between a plane and a space, the mapping relationship being used to project the plane pixel coordinates of each data point into the space to obtain a spatial world coordinate;

[0136] A spatial transverse coordinate expression unit, used to calculate the transverse coordinate expression of each data point in space based on the plane pixel coordinates and mapping relationship of each data point, the transverse coordinate expression is used to calculate the coordinate of each data point in the lane width direction, and the transverse coordinate expression includes the height coordinate of the data point in space as a height parameter to be solved;

[0137] The lane width expression unit is used to obtain a lane width expression by subtracting two lateral coordinate expressions of the current data point pair. The lane width expression is used to represent the lane width.

[0138] The height estimation unit is used to calculate the height parameter in the lane width expression based on the lane width expression and the lane line parallel constraint, and use the height parameter as the road surface height at the location of the current data point.

[0139] In some optional implementations, before the projection unit, the method further includes:

[0140] A distance statistics unit is used to calculate the point distance between two data points in each group of data point pairs;

[0141] An interval detection unit is used to determine whether the distance of each point falls within a preset width interval;

[0142] A first continuing execution unit, configured to continue executing the step of determining the mapping relationship between the plane and the space if the width falls within the preset width interval;

[0143] The first returning unit is used to return to the step of collecting a set of data point pairs from the complete lane line at a preset distance according to the road advancing direction if it does not fall within the preset width interval.

[0144] In some optional implementation modes, before the first continuing execution unit, the method further includes:

[0145] The average distance calculation unit is used to obtain the distances of several points closest to the current position of the vehicle, and calculate the average value of the obtained point distances to obtain the average distance;

[0146] A difference calculation unit, used for calculating the difference between the remaining point distances and the average distance, to obtain a plurality of distance differences;

[0147] The outlier analysis unit is used to execute the step of determining the mapping relationship between the plane and the space if the distance differences are all less than the preset distance threshold, if they fall within the preset width interval; otherwise, return to the step of collecting a set of data point pairs from the complete lane line at preset distances according to the direction of road advancement.

[0148] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0149] The road height estimation device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0150] The embodiment of the present invention also provides a computer device having the above Figure 8 The road height estimation device shown.

[0151] See also Fig.10 , Fig.10 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Fig.10 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Fig.10 A processor 10 is taken as an example.

[0152] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0153] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0154] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0155] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0156] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0157] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0158] A part of the present invention may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present invention through the operation of the computer. Those skilled in the art should understand that the existence of the computer program instruction in a computer-readable medium includes, but is not limited to, a source file, an executable file, an installation package file, etc., and accordingly, the way in which the computer program instruction is executed by the computer includes, but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium may be any available computer-readable storage medium or communication medium accessible to the computer.

[0159] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A road height estimation method, characterized in that: The method comprises: Get lane image; Extracting sampling points of lane lines on both sides of the lane from the lane image; Determine a straight segment and a curved segment according to the sampling points; Fitting the first straight lane line and the curved lane line on both sides of the lane for the straight segment and the curved segment respectively; Converting the curved lane line into a second straight lane line; Splicing the first straight lane line and the second straight lane line to obtain a complete lane line; The image road surface height is estimated based on the complete lane line.

2. The method according to claim 1, characterized in that The converting the curved lane line into a second straight lane line comprises: Drawing tangent lines for sampling points on the curved lane line; The sampling points whose tangents are parallel on both sides of the curve are paired, and the curve lane line is drawn as the second straight lane line according to the tangent distance between the paired points.

3. The method according to claim 1, characterized in that The determining of the straight segment and the curved segment according to the sampling points comprises: Clustering the sampling points to obtain a plurality of sampling point sets for representing each lane line; Divide the current sampling point set into multiple sampling point segments; Determine the normal of the fragment at the current sampling point; Calculate the angle between each sampling point in the current sampling point segment and the corresponding normal line; Calculating the angle differences between adjacent angles in the current sampling point segment, and calculating the angle sum of all the angle differences; When the sum of the included angles is greater than a preset included angle threshold, determining that the current sampling point segment is a curve segment; When the sum of the included angles is less than or equal to the preset included angle threshold, it is determined that the current sampling point segment is a straight segment.

4. The method according to claim 3, characterized in that The step of fitting the first straight lane line and the curved lane line on both sides of the lane for the straight segment and the curved segment respectively includes: Fitting the sampling point segment corresponding to the straight road segment based on a linear equation to obtain the first straight road lane line; The sampling point segment corresponding to the curve segment is fitted based on a cubic polynomial to obtain the curve lane line.

5. The method according to claim 1, characterized in that The estimating the image road surface height based on the complete lane line includes: From the complete lane line, a set of data point pairs is collected at preset distances according to the road advancing direction; determining the planar pixel coordinates of each data point in the data point pair; Determine a mapping relationship between the plane and the space, wherein the mapping relationship is used to project the plane pixel coordinates of each data point into the space to obtain the spatial world coordinates; Calculating a transverse coordinate expression of each data point in space based on the plane pixel coordinates of each data point and the mapping relationship, wherein the transverse coordinate expression is used to calculate the coordinate of each data point in the lane width direction, and the transverse coordinate expression includes the height coordinate of the data point in space as a height parameter to be solved; Subtract the two horizontal coordinate expressions of the current data point pair to obtain a lane width expression, where the lane width expression is used to represent the lane width; The height parameter in the lane width expression is calculated based on the lane width expression and the lane line parallel constraint, and the height parameter is used as the road surface height at the location of the current data point.

6. The method according to claim 5, characterized in that Before determining the mapping relationship between the plane and the space, the method further includes: Calculating the point distance between two data points in each group of the data point pairs; Determine whether the distance between each point falls within a preset width interval; If it falls within the preset width range, continue to execute the step of determining the mapping relationship between the plane and the space; If it does not fall within the preset width interval, return to the step of collecting a set of data point pairs from the complete lane line at preset distances according to the road advancing direction.

7. The method according to claim 6, characterized in that Before continuing to perform the step of determining the mapping relationship between the plane and the space if the width falls within the preset width interval, the method further includes: Obtain the distances of several points closest to the current position of the vehicle, and calculate the average of the obtained point distances to obtain the average distance; Calculate the difference between the remaining point distances and the average distance to obtain multiple distance differences; When each of the distance differences is less than a preset distance threshold, if it falls within the preset width interval, the step of continuing to determine the mapping relationship between the plane and the space is executed; otherwise, the step of returning to the step of collecting a set of data point pairs from the complete lane line at preset distances according to the direction of road advancement.

8. A road height estimation device, characterized in that: The device comprises: An image acquisition module, used for acquiring lane images; A sampling module, used to extract sampling points of lane lines on both sides of the lane from the lane image; A road type identification module, used to determine a straight road segment and a curved road segment according to the sampling points; A lane line fitting module, used for fitting the first straight lane line and the curved lane line on both sides of the lane for the straight segment and the curved segment respectively; A curve lane conversion module, used for converting the curve lane line into a second straight lane line; A combining module, used for splicing the first straight lane line and the second straight lane line to obtain a complete lane line; A height estimation module is used to estimate the image road surface height based on the complete lane line.

9. A computer device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method according to any one of claims 1 to 7 by executing the computer instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 7.

11. A vehicle, characterized in that: The computer device comprising the method provided in claim 9.

Images