Obstacle distance measurement method and device, electronic equipment and storage medium

By identifying parallel line groups and vanishing points from the road image behind the vehicle, the problem of degradation of obstacle distance measurement accuracy in the prior art is solved, and high-accuracy obstacle distance measurement in complex scenarios is achieved.

CN118913220AActive Publication Date: 2024-11-08CHONGQING SELIS PHOENIX INTELLIGENT INNOVATION TECH CO LTD

Patent Information

Application Number
CN202410870249.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-01
Publication Date
2024-11-08
Estimated Expiration
2044-07-01

AI Technical Summary

Technical Problem

In the prior art, the calculation of road slope slope based on the vehicle inclination angle and ground angle leads to a decrease in the accuracy of obstacle distance measurement results, especially in complex terrain and dynamic driving scenarios.

Method used

By obtaining image information of the road behind the vehicle, identifying parallel lines groups and determining the location of vanishing points, using these vanishing points to calculate the road slope, and then correcting the obstacle distance measurement results to avoid relying on vehicle parameters and static measurement data.

Benefits of technology

It improves the accuracy of obstacle distance measurement results, can accurately calculate road slopes under complex terrain and dynamic driving scenarios, and enhances the accuracy and safety of obstacle distance measurements behind the vehicle.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118913220A_ABST
    Figure CN118913220A_ABST
Patent Text Reader

Abstract

The invention relates to an obstacle distance measuring method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring image information of a road behind a vehicle; straight lines in the image information are recognized, a parallel line set parallel to the first preset direction is selected from the straight lines, and the parallel line set parallel to the first preset direction comprises straight lines corresponding to multiple parallel lane lines on the road; a plurality of vanishing point positions are determined through a parallel line set parallel to the first preset direction, the slope gradient of the road slope is determined based on the vanishing point positions, the vanishing point positions are located in a plurality of different preset directions, and the preset directions comprise the first preset direction; and calculating the distance between the obstacle and the vehicle according to the gradient of the road slope to obtain a distance measurement result of the obstacle. Therefore, the gradient of the road slope can be accurately calculated even in a complex terrain change and dynamic driving scene, so that the accuracy of an obstacle distance measurement result is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of intelligent driving technology, and in particular to an obstacle ranging method, device, electronic device and storage medium. Background Art

[0002] During the process of automatic driving, it is necessary to measure the distance of obstacles behind the vehicle to determine whether the obstacles are within the blind spot warning range, which is crucial for safe driving. When using a monocular camera for distance measurement, if the vehicle is driving on a flat road, the distance measurement error is relatively small, but when the vehicle is driving on a slope, the distance measurement error will increase significantly due to changes in viewing angle and distance. Therefore, it is necessary to accurately calculate the slope of the road where the vehicle is located in order to correct the monocular camera distance measurement error.

[0003] However, the existing technology usually calculates the road slope gradient based on the vehicle's tilt angle and the ground angle, but this method relies on accurate vehicle parameters and static measurement data, and cannot cope with complex terrain changes and dynamic driving scenarios. As a result, the calculated road slope gradient is less accurate, thereby reducing the accuracy of the obstacle ranging results. Summary of the invention

[0004] The present application provides an obstacle ranging method, device, electronic device and storage medium to solve the problem in the prior art that the road slope gradient is usually calculated based on the vehicle tilt angle and the ground angle, resulting in a decrease in the accuracy of the obstacle ranging result.

[0005] In a first aspect, an embodiment of the present application provides an obstacle ranging method, the method comprising:

[0006] Acquire image information of the road behind the vehicle;

[0007] Identify straight lines in the image information, and select a group of parallel lines parallel to a first preset direction from the straight lines, wherein the group of parallel lines parallel to the first preset direction includes straight lines corresponding to a plurality of mutually parallel lane lines on the road;

[0008] Determine a plurality of vanishing point positions by using the group of parallel lines parallel to the first preset direction, and determine the road slope gradient based on the plurality of vanishing point positions, wherein the plurality of vanishing point positions are located in different plurality of preset directions, and the plurality of preset directions include the first preset direction;

[0009] The distance between the obstacle and the vehicle is calculated according to the slope of the road to obtain a distance measurement result of the obstacle.

[0010] Optionally, identifying the straight line in the image information includes:

[0011] Filtering and denoising the image information, and performing edge detection on the filtered and denoised image information to obtain edge information corresponding to the image information;

[0012] Detecting a plurality of candidate lines from the edge information using an improved Hough transform algorithm;

[0013] The plurality of candidate lines are filtered and fitted to obtain the straight line.

[0014] Optionally, the detecting a plurality of candidate lines from the edge information using an improved Hough transform algorithm includes:

[0015] Downsampling the edge information multiple times to obtain edge information corresponding to multiple scales;

[0016] Performing Hough transform on edge points contained in edge information corresponding to each scale of edge information corresponding to the multiple scales, and updating a value in an accumulator space according to a Hough transform result of each edge point, wherein the accumulator space is used to count the number of edge points detected on each straight line;

[0017] A plurality of straight lines whose cumulative number of edge points reaches a preset threshold are determined as the plurality of candidate lines.

[0018] Optionally, the determining a plurality of vanishing point positions by using the group of parallel lines parallel to the first preset direction includes:

[0019] Clustering the group of parallel lines parallel to the first preset direction to obtain K candidate vanishing point positions;

[0020] respectively calculating a reprojection error of each candidate vanishing point position among the K candidate vanishing point positions, and determining the candidate vanishing point position with the smallest reprojection error as the first vanishing point position in the first preset direction;

[0021] Constructing an equivalent spherical surface based on the first vanishing point position, and determining a second vanishing point position in a second preset direction from the equivalent spherical surface, wherein the second preset direction is any direction in a plane perpendicular to the first preset direction;

[0022] Based on the second vanishing point position, a third vanishing point position in a third preset direction is determined from the equivalent spherical surface, wherein the third preset direction is a direction perpendicular to the first preset direction and the second preset direction, and the first vanishing point position, the second vanishing point position and the third vanishing point position all belong to the multiple vanishing point positions.

[0023] Optionally, the formula for determining the road slope gradient is as follows:

[0024]

[0025] Wherein, θ represents the road slope gradient, x1 and y1 represent the first vanishing point position, and x3 and y3 represent the third vanishing point position.

[0026] Optionally, calculating the distance between the obstacle and the vehicle according to the slope of the road to obtain a distance measurement result of the obstacle includes:

[0027] Determine the coordinate position of the obstacle in the image information;

[0028] Determine a theoretical distance between the obstacle and the vehicle at the same road height according to the coordinate position of the obstacle in the image information and preset camera parameters;

[0029] The actual distance between the obstacle and the vehicle is calculated according to the road slope gradient and the theoretical distance, and the actual distance is determined as the distance measurement result of the obstacle.

[0030] Optionally, before identifying the straight line in the image information, the method further includes:

[0031] Identify lane lines in the image information, and determine whether the slope of the lane lines has changed;

[0032] When the slope of the lane line changes, the step of identifying a straight line in the image information is performed.

[0033] In a second aspect, an embodiment of the present application further provides an obstacle ranging device, the device comprising:

[0034] An acquisition module, used to acquire image information of the road behind the vehicle;

[0035] a first recognition module, configured to recognize straight lines in the image information, and select a group of parallel lines parallel to a first preset direction from the straight lines, wherein the group of parallel lines parallel to the first preset direction includes straight lines corresponding to a plurality of mutually parallel lane lines on the road;

[0036] a determination module, configured to determine a plurality of vanishing point positions using the group of parallel lines parallel to the first preset direction, and determine the slope of the road based on the plurality of vanishing point positions, wherein the plurality of vanishing point positions are located in different plurality of preset directions, the plurality of preset directions including the first preset direction;

[0037] The calculation module is used to calculate the distance between the obstacle and the vehicle according to the slope of the road to obtain a distance measurement result of the obstacle.

[0038] In a third aspect, an embodiment of the present application further provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus;

[0039] Memory, used to store computer programs;

[0040] The processor is used to implement the obstacle ranging method described in any one of the first aspects when executing the program stored in the memory.

[0041] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the obstacle ranging method described in any one of the first aspects is implemented.

[0042] The above technical solution provided by the embodiment of the present application has the following advantages over the prior art: the method provided by the embodiment of the present application obtains image information of the road behind the vehicle; identifies straight lines in the image information, and selects a group of parallel lines parallel to a first preset direction from the straight lines, wherein the group of parallel lines parallel to the first preset direction includes straight lines corresponding to a plurality of mutually parallel lane lines on the road; determines a plurality of vanishing point positions by using the group of parallel lines parallel to the first preset direction, and determines the road slope gradient based on the plurality of vanishing point positions, wherein the plurality of vanishing point positions are located in a plurality of different preset directions, and the plurality of preset directions include the first preset direction; calculates the distance between the obstacle and the vehicle according to the road slope gradient to obtain the distance measurement result of the obstacle. Through the above method, a plurality of vanishing point positions in a plurality of preset directions can be determined by using a group of parallel lines parallel to the first preset direction identified from the image information, and then the road slope gradient can be calculated based on the plurality of vanishing point positions without relying on vehicle parameters and static measurement data. Therefore, even in complex terrain changes and dynamic driving scenarios, the road slope gradient can be accurately calculated, thereby improving the accuracy of obstacle ranging results. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0044] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0045] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0046] Figure 1 A flowchart of an obstacle ranging method provided in an embodiment of the present application;

[0047] Figure 2 A schematic diagram of an equivalent spherical surface provided in an embodiment of the present application;

[0048] Figure 3 A schematic diagram of a geometric relationship for calculating the road slope gradient θ provided in an embodiment of the present application;

[0049] Figure 4 A flowchart of another obstacle ranging method provided in an embodiment of the present application;

[0050] Figure 5 A schematic diagram of the distribution of characteristic points of a lane line provided in an embodiment of the present application;

[0051] Figure 6 A schematic diagram of the structure of an obstacle distance measuring device provided in an embodiment of the present application;

[0052] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0054] The disclosure below provides many different embodiments or examples to implement different structures of the present invention. In order to simplify the disclosure of the present invention, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present invention. In addition, the present invention can repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed.

[0055] In order to solve the problem that the slope of a road is usually calculated based on the vehicle's tilt angle and the ground angle in the prior art, resulting in a decrease in the accuracy of the obstacle ranging result, the present application provides an obstacle ranging method, device, electronic device and storage medium, which can improve the accuracy of the obstacle ranging result.

[0056] See also Figure 1 , Figure 1 A flowchart of an obstacle ranging method provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the obstacle ranging method may include the following steps:

[0057] Step 101: Acquire image information of the road behind the vehicle.

[0058] Specifically, when obtaining image information of the road behind the vehicle, it can be obtained by using a camera installed on an electronic rearview mirror, or by using a camera installed at the rear of the vehicle, which is not specifically limited in the embodiments of the present application. It should be noted that the image information of the road behind the vehicle includes information such as lane lines and road boundary lines.

[0059] Step 102: Identify straight lines in the image information, and select a group of parallel lines parallel to a first preset direction from the straight lines, wherein the group of parallel lines parallel to the first preset direction includes straight lines corresponding to a plurality of mutually parallel lane lines on the road.

[0060] In this step, an improved Hough transform algorithm or other algorithms can be used to perform straight line detection on the image information to identify the straight lines in the image information, and then a group of parallel lines parallel to the first preset direction is selected from these straight lines. It should be noted that the first preset direction here can be understood as a direction parallel to the vertical direction of the image information. The group of parallel lines parallel to the first preset direction includes at least straight lines corresponding to multiple mutually parallel lane lines on the road, so that the vanishing point position of the lane line in the image can be determined based on the straight line parameters corresponding to the lane line.

[0061] Step 103: using a group of parallel lines parallel to the first preset direction, determine a plurality of vanishing point positions, and based on the plurality of vanishing point positions, determine the slope of the road, wherein the plurality of vanishing point positions are located in different plurality of preset directions, and the plurality of preset directions include the first preset direction.

[0062] Specifically, in three-dimensional space, all parallel lines will converge to a point at infinity. The projection of this point in the perspective projection image is the vanishing point, also called the vanishing point.

[0063] In this step, a group of parallel lines parallel to the first preset direction can be used to determine the vanishing point position in the first preset direction, and then based on the vanishing point position in the first preset direction, vanishing points in other preset directions can be determined on the equivalent spherical surface of the vanishing point position in the first preset direction. Then, based on the multiple vanishing point positions, the rotation angle of the camera can be determined, that is, the slope of the road can be determined.

[0064] Step 104: Calculate the distance between the obstacle and the vehicle according to the slope of the road to obtain a distance measurement result of the obstacle.

[0065] After determining the slope of the road, the actual distance between the obstacle and the vehicle can be calculated using geometric relationships to obtain the obstacle distance measurement result.

[0066] In this embodiment, a group of parallel lines parallel to the first preset direction identified from the image information can be used to determine multiple vanishing point positions in multiple preset directions, and then the road slope gradient can be calculated based on the multiple vanishing point positions without relying on vehicle parameters and static measurement data. Therefore, even in complex terrain changes and dynamic driving scenarios, the road slope gradient can be accurately calculated, thereby improving the accuracy of obstacle ranging results.

[0067] Furthermore, the above step 102, identifying the straight line in the image information, includes:

[0068] Filtering and denoising the image information, and performing edge detection on the filtered and denoised image information to obtain edge information corresponding to the image information;

[0069] Using the improved Hough transform algorithm to detect multiple candidate lines from edge information;

[0070] Multiple candidate lines are filtered and fitted to obtain a straight line.

[0071] Specifically, since real image information often contains noise, the image information can be smoothed to remove noise before edge detection. This can be achieved through various filters, such as Gaussian filter, mean filter, median filter and bilateral filter. As an optional implementation, a Gaussian filter can be used for denoising.

[0072] The core of edge detection is to calculate the gradient of the image. The magnitude and direction of the gradient can represent the intensity and direction of the change in image brightness. The gradient can be calculated by the first-order or second-order derivative. The calculation of the gradient magnitude G is usually implemented using the Sobel operator (i.e., Sobel operator) or the Prewitt operator (i.e., Prewitt operator), which are special cases of spatial difference operators and are used to estimate the first-order derivative of the image. The calculation formula for the gradient magnitude G and direction θ of the Sobel operator is:

[0073]

[0074] Among them, G x is the gradient component of the image brightness function in the horizontal direction, G y is the gradient component of the image brightness function in the vertical direction, G x and G y Given by the Sobel operator, G x =|H x *I|,G y =|H y *I|,H x and H y are all convolution kernels of Sobel operator, and I is the original image information.

[0075] After calculating the gradient amplitude using the above formula, the gradient amplitude of each pixel can be non-maximum suppressed to determine the edge of the image. If a pixel is not a local maximum in the gradient direction, it is considered that the pixel is not part of the edge. Finally, by setting a high threshold and a low threshold, the pixels that truly belong to the edge are further screened out. A pixel is retained as part of the edge only when its gradient amplitude is higher than the high threshold or is connected to a neighboring point with a gradient amplitude higher than the high threshold. In this way, the edge information corresponding to the image information can be obtained.

[0076] After obtaining the edge information corresponding to the image information, the improved Hough transform algorithm can be used to detect multiple candidate lines from the edge information. It should be noted that the Hough transform is an algorithm for detecting straight lines from an image. It maps the straight lines in the image space to the parameter space so that all collinear points intersect at one point in the parameter space. The basic principle of the Hough transform is that in the Cartesian coordinate system, any straight line can be expressed in the form of y=mx+b, where m is the slope of the line and b is the y-intercept. However, the Hough transform prefers to use the polar coordinate system to represent the straight line, that is, ρ=x*cos(θ)+y*sin(θ), where ρ is the distance from the origin to the straight line and θ is the angle between the straight line and the x-axis. The principle of the traditional Hough transform algorithm is: first, edge detection is performed on the input image to obtain the edge information of the image; then, an accumulator space is created for the Hough transform, and each cell of the space corresponds to a possible line parameter (such as ρ and θ); then, each pixel in the edge information is traversed, and the line parameter corresponding to each pixel in the accumulator space is calculated according to the coordinates of each pixel, and the count of the corresponding cell is increased; thereafter, local maxima are found in the accumulator space, and these local maxima represent the parameters of the line in the image; finally, the line in the image is determined according to these local maxima. In this way, the Hough transform can identify the straight line segments in the image, and has good robustness even in the presence of noise and incomplete lines.

[0077] Although the Hough transform is very effective in detecting geometric shapes such as straight lines and circles in images, it has some obvious disadvantages, mainly including large computational complexity, high memory consumption, sensitivity to noise, difficulty in processing complex shapes, and difficulty in parameter selection. These disadvantages limit the use of the Hough transform in real-time or resource-constrained application scenarios, and require carefully designed parameter adjustments to optimize its performance. Therefore, in this application, the traditional Hough transform is optimized and improved, and the improved Hough transform algorithm is used to detect multiple candidate lines from edge information, and then the multiple candidate lines are filtered and fitted to detect the straight lines in the image information.

[0078] The purpose of line filtering is to remove noise and false positive detection results in the Hough transform accumulator space, thereby retaining the features that truly represent the line. A threshold is set for each cell in the accumulator space, and only cells that exceed this threshold are considered to be valid evidence of a line. Local maximum points are found in the accumulator space, which usually correspond to the parameters of the line. All points smaller than the local maximum are removed from the neighborhood of each local maximum, which is called non-maximum suppression (NMS).

[0079] Line fitting is to determine the exact parameters of the line based on the filtered results. The parameters of the line, such as slope and intercept, are extracted from the local maximum of the accumulator space. The extracted parameters are used to fit the line in the image space. In the Cartesian coordinate system, the line can be expressed in the form of y=mx+b, where m is the slope and b is the y-intercept. Fitting involves using regression analysis to minimize the sum of squared errors of the line equation.

[0080] Through the above method, the straight line in the image information of the road behind the vehicle can be accurately identified, which is convenient for subsequent vanishing point detection.

[0081] Furthermore, the above step of detecting multiple candidate lines from edge information using an improved Hough transform algorithm includes:

[0082] Downsample the edge information multiple times to obtain edge information corresponding to multiple scales;

[0083] Performing Hough transform on edge points contained in edge information corresponding to each scale in edge information corresponding to multiple scales, and updating a value in an accumulator space according to a Hough transform result of each edge point, wherein the accumulator space is used to count the number of edge points detected on each straight line;

[0084] A plurality of straight lines whose cumulative number of edge points reaches a preset threshold are determined as a plurality of candidate lines.

[0085] In one embodiment, when multiple candidate lines are detected from edge information using the improved Hough transform algorithm, edge information corresponding to multiple scales can be generated by continuous multiple downsampling, which can be expressed as: s =G*I s-1 , where I s is the image at scale s, G is the downsampling filter, I s-1 is the image at scale s-1. At scale s, the straight line can be represented as: y i =m s *x i +b s , where m s and b s They are the slope and intercept of the straight line at scale s, respectively. The straight line equation is converted into polar coordinates for updating the accumulator in the Hough space: ρ = x·cos(θ) + y·sin(θ). For each edge point, a vote is performed on the corresponding straight line in the accumulator space: A(ρ,θ) = A(ρ,θ) + 1. Finally, multiple straight lines whose cumulative number of edge points reaches a preset threshold are determined as multiple candidate lines. It should be noted that the detected straight line parameters can also be converted from different scales to a normalized scale to facilitate the unified representation of the results. The formula for the normalization process can be:

[0086]

[0087] In the traditional Hough transform algorithm, usually only shapes of a specific scale can be detected, while in the improved Hough transform algorithm, shapes at multiple scales can be detected, thus improving the flexibility and applicability of the algorithm.

[0088] Furthermore, the above step 103, using a group of parallel lines parallel to the first preset direction to determine the positions of multiple vanishing points, includes:

[0089] Clustering the parallel line groups parallel to the first preset direction to obtain K candidate vanishing point positions;

[0090] Calculating the reprojection error of each candidate vanishing point position among the K candidate vanishing point positions respectively, and determining the candidate vanishing point position with the smallest reprojection error as the first vanishing point position in the first preset direction;

[0091] constructing an equivalent spherical surface based on the first vanishing point position, and determining a second vanishing point position in a second preset direction from the equivalent spherical surface, wherein the second preset direction is any direction in a plane perpendicular to the first preset direction;

[0092] Based on the second vanishing point position, a third vanishing point position in a third preset direction is determined from the equivalent spherical surface, wherein the third preset direction is a direction perpendicular to the first preset direction and the second preset direction, and the first vanishing point position, the second vanishing point position and the third vanishing point position all belong to multiple vanishing point positions.

[0093] In one embodiment, the parallel line groups parallel to the first preset direction may be clustered to obtain K candidate vanishing point positions, and then the reprojection errors of the K candidate vanishing point positions are calculated respectively, and the candidate vanishing point position with the smallest reprojection error is determined as the first vanishing point position in the first preset direction, and then an equivalent sphere is constructed based on the first vanishing point position, and a second vanishing point position in the second preset direction is determined from the equivalent sphere, and finally, based on the second vanishing point position, a third vanishing point position in the third preset direction is determined from the equivalent sphere.

[0094] Specifically, in the Cartesian coordinate system, the direction of a line can be represented by the slope m of the line. The slope m can be calculated from the two points (x1, y1) and (x2, y2) of the line as follows:

[0095]

[0096] Vanishing points are usually divided into three categories, corresponding to parallel lines in three main directions. The first vanishing point is the vanishing point corresponding to the line parallel to the vertical direction of the image; the second vanishing point is the vanishing point corresponding to the line parallel to the horizontal direction of the image; the third vanishing point is the vanishing point corresponding to the line parallel to the diagonal direction of the image. For the first vanishing point, the detected straight lines are classified according to their directions. This can be done by mapping the slope to a one-dimensional space, for example, using the radian value θ to represent the direction of the straight line: θ = arctan (m). Cluster analysis is performed on the straight lines of each direction category to find the central tendency of the straight lines. Specifically, the K-means clustering algorithm can be used, where K is equal to the number of candidate vanishing points, and the set of straight lines corresponding to each cluster center represents a set of coplanar parallel lines. The coordinates of the cluster center can be used as the candidate vanishing point position, and the objective function of the K-means clustering algorithm is shown in the following formula:

[0097]

[0098] Among them, x i represents the i-th data point, μ j represents the jth cluster center, K is the number of cluster centers, and N is the total number of data points.

[0099] Due to image noise and other factors, the candidate vanishing points need to be further optimized and accurately located. This can be done by minimizing the reprojection error. Specifically, the formula for the reprojection error is as follows:

[0100]

[0101] Among them, L i is the equation of the ith line, P i is a point on the i-th straight line, D i It's point P i At the observation position in the image, M is the number of lines considered, that is, the total number of lines parallel to the first preset direction. Therefore, the first vanishing point position can be obtained by a clustering algorithm.

[0102] After determining the first vanishing point position p1, the second vanishing point position p2 can be obtained on the outer circle of the equivalent sphere of p1. For the second vanishing point position p2, it is the vanishing point corresponding to a set of parallel lines orthogonal to p1. Considering the orthogonal constraint, Figure 2 A point on the dotted circle is randomly selected as the second vanishing point position p2.

[0103] In perspective geometry, the third vanishing point is orthogonal to the first and second vanishing points. When two vanishing points P1 and P2 are known, the third vanishing point P3 can be found using the vector cross product. The cross product is a method of calculating a pseudo vector in a three-dimensional space that is perpendicular to two vectors. First, the vanishing points P1 and P2 need to be represented as vectors on the image plane. If P1 = (x1, y1) and P2 = (x2, y2), then the vector P1P2 can be represented as:

[0104]

[0105] The result of the cross product is a scalar, which represents the directed area of ​​the parallelogram formed by the two vectors. In the two-dimensional case, the direction of the unit vector perpendicular to the two vectors can be obtained by calculating the cross product of the two vectors. Their cross product result is:

[0106] P1×P2=x1y2-x2y1

[0107] In the equivalent spherical model, all vanishing points are located on the sphere. Therefore, the position of the third vanishing point can be determined.

[0108] Through the above method, a plurality of vanishing point positions can be accurately determined by using a group of parallel lines parallel to the first preset direction, thereby facilitating subsequent accurate calculation of the road slope gradient according to the plurality of vanishing point positions.

[0109] Furthermore, the formula for determining the road slope gradient is as follows:

[0110]

[0111] Wherein, θ represents the road slope gradient, x1 and y1 represent the first vanishing point position, and x3 and y3 represent the third vanishing point position.

[0112] It should be noted that, since the first vanishing point position P1 represents the vanishing point position corresponding to the parallel lines perpendicular to the image direction (i.e., the Z axis), the second vanishing point position P2 represents the vanishing point position corresponding to the parallel lines in the horizontal direction of the image (i.e., the x axis), and the third vanishing point position P3 represents the vanishing point position corresponding to the parallel lines in the vertical direction of the image (i.e., the y axis). Therefore, the rotation angle of the camera, i.e., the road slope gradient θ, can be determined based on the angle between the straight line where the first vanishing point position P1 and the third vanishing point position P3 are located and the Z axis. Specifically, Figure 3 As shown, since the slope m of a straight line is the ratio of the ordinate to the abscissa of the direction vector, that is, m = (y3-y1) / (x3-x1), and the angle θ between the straight line and the Z axis can be calculated by the slope m, the inverse tangent function is used to calculate the slope of the road as θ = arctan(m), from which the above formula can be inferred.

[0113] By using the above formula, the road slope gradient can be accurately calculated according to the first vanishing point position and the third vanishing point position.

[0114] Furthermore, the above step 104, calculating the distance between the obstacle and the vehicle according to the slope of the road to obtain the distance measurement result of the obstacle, includes:

[0115] Determine the coordinate position of the obstacle in the image information;

[0116] According to the coordinate position of the obstacle in the image information and the preset camera parameters, the theoretical distance between the obstacle and the vehicle at the same road height is determined;

[0117] According to the road slope gradient and the theoretical distance, the actual distance between the obstacle and the vehicle is calculated, and the actual distance is determined as the distance measurement result of the obstacle.

[0118] In one embodiment, when calculating the distance between an obstacle and a vehicle based on the slope of a road, the coordinate position of the obstacle in the image information can be determined by a target detection algorithm. As an optional implementation, the coordinates (xbase, ybase) of the center point of the bottom frame of the obstacle can be obtained by a target detection algorithm to estimate the grounding point of the obstacle for distance calculation. Then, based on the coordinate position of the obstacle in the image information and the preset camera parameters, the theoretical distance d between the obstacle and the vehicle at the same road height is determined. image , and then the distance between the obstacle and the vehicle is calculated according to the road slope θ to obtain the obstacle distance measurement result. Specifically, the Pythagorean theorem combined with the slope can be used to estimate the actual distance d of the obstacle actual , using the following formula:

[0119]

[0120] Among them, d actual Indicates the actual distance between the obstacle and the vehicle, d image represents the theoretical distance, and θ represents the slope of the road.

[0121] Through the above method, a more accurate actual distance of obstacles can be obtained. This can be applied to the blind spot detection alarm function of the electronic rearview mirror. The algorithm uses image processing technology to identify edges, straight lines and vanishing points, and determines the location and distance of obstacles based on this. This function can detect obstacles in the blind spot in time and provide warnings to the driver, thereby enhancing driving safety.

[0122] Furthermore, before the above step 101 of identifying the straight line in the image information, the method further includes:

[0123] Identify the lane lines in the image information and determine whether the slope of the lane lines has changed;

[0124] When the slope of the lane line changes, the step of: identifying the straight line in the image information.

[0125] In one embodiment, if the real-time calculation of the road slope gradient will consume computing power, it is possible to first determine whether there is a road slope and then calculate the distance, which can achieve accurate distance measurement more efficiently. The specific process is as follows Figure 4 As shown in Figure 1, image information is first obtained through the electronic rearview mirror, and then lane line detection is performed to extract the feature points of the lane line. A lane line feature point on a slope is shown in Figure 1. Figure 5 As shown, pl and pr represent a series of points, G1 and G2 represent different road surfaces, G1 represents a flat road, and G2 represents a sloped road.

[0126] The main purpose of lane line detection is to use slope changes to analyze road surface features in order to determine the slope changes of the road behind the vehicle. From a visual perspective, when the slope of the road behind changes, the lane line will have an inflection point at the road boundary. Therefore, after completing the lane line fitting, you first need to extract a series of points on the lane line and calculate the slope between two adjacent points:

[0127]

[0128] Among them, pl i.y and pl i+1.y is a feature point on one of the lane lines, pr j.y and pr j+1.y is a feature point on another lane line, and i and j are integers greater than 1.

[0129] Next, the slope of the road is analyzed according to the changes in these slopes kl and kr, and the road section is divided into different areas. In this way, the slope changes of the road can be understood more accurately. If there is a slope change, it is necessary to calculate the slope and then calculate the obstacle distance. If there is no slope change, there is no need to calculate the slope and the obstacle distance can be calculated directly.

[0130] In the embodiment of the present application, computer vision technology can be used to infer the inclination angle of the ground by identifying the vanishing point in the image information, and then calculate the slope gradient. The method detects parallel lines in the road image captured by the electronic rearview mirror and determines the position of their vanishing points. The inclination angle of the ground can be inferred by measuring the offset of the vanishing point relative to the center of the image, as well as the parameters of the camera, such as focal length and field of view angle. Compared with the traditional method, the embodiment of the present application does not rely on vehicle parameters, is suitable for dynamic scenes, and can more accurately capture the slope changes of complex terrain. Finally, the position and size of the object in the image are determined by the target detection algorithm. The pixel coordinates in the image can be converted into coordinates in the real world by using the intrinsic and extrinsic parameters of the camera, as well as the slope information, so that the distance between the vehicle and the rear obstacle can be obtained more accurately, and the blind spot alarm function can be implemented according to the distance.

[0131] This application can bring significant benefits to the blind spot detection alarm function of the electronic rearview mirror. In this solution, by comprehensively using key technologies such as image processing, straight line detection, vanishing point detection, slope calculation and target detection, accurate estimation of obstacle distance is achieved. This solution can accurately identify the location of obstacles, effectively avoid collision risks in blind spots, and improve driving safety. At the same time, it fully considers the ground inclination factor, making distance estimation more accurate and reliable. This technological innovation provides drivers with a more comprehensive perception of road conditions and provides strong support for driving safety.

[0132] See also Figure 6 , Figure 6 This is a schematic diagram of the structure of an obstacle distance measurement device provided in an embodiment of the present application. Figure 6 As shown, the device 600 includes:

[0133] An acquisition module 601 is used to acquire image information of the road behind the vehicle;

[0134] A first recognition module 602 is used to recognize straight lines in the image information and select a group of parallel lines parallel to a first preset direction from the straight lines, wherein the group of parallel lines parallel to the first preset direction includes straight lines corresponding to a plurality of mutually parallel lane lines on the road;

[0135] A determination module 603 is used to determine a plurality of vanishing point positions using a group of parallel lines parallel to a first preset direction, and determine a road slope gradient based on the plurality of vanishing point positions, wherein the plurality of vanishing point positions are located in different plurality of preset directions, and the plurality of preset directions include the first preset direction;

[0136] The calculation module 604 is used to calculate the distance between the obstacle and the vehicle according to the slope of the road to obtain the distance measurement result of the obstacle.

[0137] Further, the first identification module 602 includes:

[0138] The edge detection submodule is used to filter and denoise the image information, and perform edge detection on the filtered and denoised image information to obtain edge information corresponding to the image information;

[0139] A detection submodule, used for detecting multiple candidate lines from edge information using an improved Hough transform algorithm;

[0140] The filtering and fitting submodule is used to filter and fit multiple candidate lines to obtain a straight line.

[0141] Furthermore, the detection submodule includes:

[0142] A downsampling unit, used for performing multiple downsampling on the edge information to obtain edge information corresponding to multiple scales;

[0143] A Hough transform unit, configured to perform a Hough transform on edge points contained in edge information corresponding to each scale in edge information corresponding to multiple scales, and update a value in an accumulator space according to a Hough transform result of each edge point, wherein the accumulator space is used to count the number of edge points detected on each straight line;

[0144] The determining unit is used to determine a plurality of straight lines whose cumulative number of edge points reaches a preset threshold as a plurality of candidate lines.

[0145] Further, the determination module 603 includes:

[0146] A clustering submodule, used to cluster the parallel line groups parallel to the first preset direction to obtain K candidate vanishing point positions;

[0147] A first determination submodule, configured to respectively calculate a reprojection error of each of the K candidate vanishing point positions, and determine the candidate vanishing point position with the smallest reprojection error as a first vanishing point position in a first preset direction;

[0148] A second determination submodule is used to construct an equivalent spherical surface based on the first vanishing point position, and determine a second vanishing point position in a second preset direction from the equivalent spherical surface, wherein the second preset direction is any direction in a plane perpendicular to the first preset direction;

[0149] The third determination submodule is used to determine a third vanishing point position in a third preset direction from the equivalent spherical surface based on the second vanishing point position, wherein the third preset direction is a direction perpendicular to the first preset direction and the second preset direction, and the first vanishing point position, the second vanishing point position and the third vanishing point position all belong to multiple vanishing point positions.

[0150] Furthermore, the formula for determining the road slope gradient is as follows:

[0151]

[0152] Wherein, θ represents the road slope gradient, x1 and y1 represent the first vanishing point position, and x3 and y3 represent the third vanishing point position.

[0153] Further, the calculation module 604 includes:

[0154] The fourth determination submodule is used to determine the coordinate position of the obstacle in the image information;

[0155] A fifth determination submodule, for determining a theoretical distance between the obstacle and the vehicle at the same road height according to the coordinate position of the obstacle in the image information and preset camera parameters;

[0156] The sixth determination submodule is used to calculate the actual distance between the obstacle and the vehicle according to the road slope gradient and the theoretical distance, and determine the actual distance as the distance measurement result of the obstacle.

[0157] Furthermore, the device 600 further includes:

[0158] The second recognition module is used to recognize the lane line in the image information and determine whether the slope of the lane line has changed;

[0159] The execution module is used to execute the step of identifying a straight line in the image information when the slope of the lane line changes.

[0160] It should be noted that the device 600 can implement the obstacle ranging method provided by any of the aforementioned method embodiments and can achieve the same technical effects, which will not be described in detail here.

[0161] like Figure 7 As shown, the embodiment of the present application further provides an electronic device, including a processor 711, a communication interface 712, a memory 713 and a communication bus 714, wherein the processor 711, the communication interface 712, and the memory 713 communicate with each other through the communication bus 714.

[0162] Memory 713, used for storing computer programs;

[0163] In one embodiment of the present application, the processor 711 is used to implement the obstacle ranging method provided by any one of the aforementioned method embodiments when executing the program stored in the memory 713.

[0164] An embodiment of the present application further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the obstacle ranging method provided by any of the aforementioned method embodiments is implemented.

[0165] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0166] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0167] It should be understood that the terms used in the text are only for the purpose of describing specific example embodiments, and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "said" as used in the text may also be meant to include plural forms. The terms "include", "comprise", "contain", and "have" are inclusive, and therefore specify the existence of stated features, steps, operations, elements and / or parts, but do not exclude the existence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The method steps, processes, and operations described herein are not interpreted as necessarily requiring them to be performed in the specific order described or illustrated, unless the execution order is clearly indicated. It should also be understood that additional or alternative steps may be used.

[0168] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. An obstacle ranging method, characterized in that: The method comprises: Acquire image information of the road behind the vehicle; Identify straight lines in the image information, and select a group of parallel lines parallel to a first preset direction from the straight lines, wherein the group of parallel lines parallel to the first preset direction includes straight lines corresponding to a plurality of mutually parallel lane lines on the road; Determine a plurality of vanishing point positions by using the group of parallel lines parallel to the first preset direction, and determine the road slope gradient based on the plurality of vanishing point positions, wherein the plurality of vanishing point positions are located in different plurality of preset directions, and the plurality of preset directions include the first preset direction; The distance between the obstacle and the vehicle is calculated according to the slope of the road to obtain a distance measurement result of the obstacle.

2. The method according to claim 1, characterized in that The identifying the straight line in the image information includes: Filtering and denoising the image information, and performing edge detection on the filtered and denoised image information to obtain edge information corresponding to the image information; Detecting a plurality of candidate lines from the edge information using an improved Hough transform algorithm; The plurality of candidate lines are filtered and fitted to obtain the straight line.

3. The method according to claim 2, characterized in that The step of detecting a plurality of candidate lines from the edge information using an improved Hough transform algorithm includes: Downsampling the edge information multiple times to obtain edge information corresponding to multiple scales; Performing Hough transform on edge points contained in edge information corresponding to each scale of edge information corresponding to the multiple scales, and updating a value in an accumulator space according to a Hough transform result of each edge point, wherein the accumulator space is used to count the number of edge points detected on each straight line; A plurality of straight lines whose cumulative number of edge points reaches a preset threshold are determined as the plurality of candidate lines.

4. The method according to claim 1, characterized in that: The step of determining the positions of multiple vanishing points by using the group of parallel lines parallel to the first preset direction includes: Clustering the group of parallel lines parallel to the first preset direction to obtain K candidate vanishing point positions; respectively calculating a reprojection error of each candidate vanishing point position among the K candidate vanishing point positions, and determining the candidate vanishing point position with the smallest reprojection error as the first vanishing point position in the first preset direction; Constructing an equivalent spherical surface based on the first vanishing point position, and determining a second vanishing point position in a second preset direction from the equivalent spherical surface, wherein the second preset direction is any direction in a plane perpendicular to the first preset direction; Based on the second vanishing point position, a third vanishing point position in a third preset direction is determined from the equivalent spherical surface, wherein the third preset direction is a direction perpendicular to the first preset direction and the second preset direction, and the first vanishing point position, the second vanishing point position and the third vanishing point position all belong to the multiple vanishing point positions.

5. The method according to claim 4, characterized in that The formula for determining the slope of the road is as follows: Wherein, θ represents the road slope gradient, x1 and y1 represent the first vanishing point position, and x3 and y3 represent the third vanishing point position.

6. The method according to claim 1, characterized in that The calculating the distance between the obstacle and the vehicle according to the slope of the road to obtain the distance measurement result of the obstacle includes: Determine the coordinate position of the obstacle in the image information; Determine a theoretical distance between the obstacle and the vehicle at the same road height according to the coordinate position of the obstacle in the image information and preset camera parameters; The actual distance between the obstacle and the vehicle is calculated according to the road slope gradient and the theoretical distance, and the actual distance is determined as the distance measurement result of the obstacle.

7. The method according to claim 1, characterized in that Before identifying the straight line in the image information, the method further includes: Identify lane lines in the image information, and determine whether the slope of the lane lines has changed; When the slope of the lane line changes, the step of identifying a straight line in the image information is performed.

8. An obstacle distance measuring device, characterized in that: The device comprises: An acquisition module, used to acquire image information of the road behind the vehicle; a first recognition module, configured to recognize straight lines in the image information, and select a group of parallel lines parallel to a first preset direction from the straight lines, wherein the group of parallel lines parallel to the first preset direction includes straight lines corresponding to a plurality of mutually parallel lane lines on the road; a determination module, configured to determine a plurality of vanishing point positions using the group of parallel lines parallel to the first preset direction, and determine the slope of the road based on the plurality of vanishing point positions, wherein the plurality of vanishing point positions are located in different plurality of preset directions, the plurality of preset directions including the first preset direction; The calculation module is used to calculate the distance between the obstacle and the vehicle according to the slope of the road to obtain a distance measurement result of the obstacle.

9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; The processor is used to implement the obstacle ranging method described in any one of claims 1 to 7 when executing the program stored in the memory.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the obstacle ranging method according to any one of claims 1 to 7 is implemented.

Citation Information

Patent Citations

  • Lane video detection method based on half-machine study

    CN103383733A

  • Road slope calculation method and device based on monocular vision

    CN109900254A

  • Lane line recognition method and device

    CN112364869A

  • Automatic driving equipment distance measuring method and related device

    CN112525147A

Cited By

  • Autonomous getting-off method of robot, robot and system

    CN119440019A

  • A method, robot, and system for autonomous disembarkation of a robot

    CN119440019B