Obstacle ranging methods, devices, electronic equipment and storage media

By identifying parallel line groups and vanishing point locations from images of the road behind the vehicle and calculating the road slope, the problem of inaccurate obstacle ranging results in existing technologies is solved, and accurate ranging is achieved in complex terrain and dynamic driving scenarios.

CN118913220BActive Publication Date: 2026-11-17CHONGQING SELIS PHOENIX INTELLIGENT INNOVATION TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202410870249.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-07-01
Publication Date
2026-11-17
Estimated Expiration
2044-07-01

AI Technical Summary

Technical Problem

Existing technologies calculate road slope gradient based on vehicle tilt angle and ground angle, which leads to a decrease in the accuracy of obstacle ranging results, especially in complex terrain and dynamic driving scenarios where it is difficult to accurately correct ranging errors.

Method used

By acquiring image information of the road behind the vehicle, identifying and selecting a group of parallel lines parallel to the first preset direction, determining multiple vanishing point positions, and using these vanishing point positions to calculate the road slope gradient, thereby accurately measuring the distance.

Benefits of technology

In complex terrain and dynamic driving scenarios, it can accurately calculate road slope gradient, improve the accuracy of obstacle distance measurement results, and enhance driving safety.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118913220B_ABST
    Figure CN118913220B_ABST
Patent Text Reader

Abstract

This application relates to an obstacle ranging method, apparatus, electronic device, and storage medium. The method includes: acquiring image information of the road behind a vehicle; identifying straight lines in the image information and selecting a group of parallel lines parallel to a first preset direction from the straight lines, wherein the group of parallel lines parallel to the first preset direction includes straight lines corresponding to multiple parallel lane lines on the road; determining multiple vanishing point positions using the group of parallel lines parallel to the first preset direction, and determining the road slope gradient based on the multiple vanishing point positions, wherein the multiple vanishing point positions are located in different preset directions, including the first preset direction; and calculating the distance between the obstacle and the vehicle based on the road slope gradient to obtain the obstacle ranging result. Thus, even in complex terrain changes and dynamic driving scenarios, the road slope gradient can be accurately calculated, thereby improving the accuracy of the obstacle ranging result.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of intelligent driving technology, and in particular to an obstacle ranging method, device, electronic device and storage medium. Background Technology

[0002] During autonomous driving, it is necessary to measure the distance to obstacles behind the vehicle to determine whether they are within the blind spot warning range, which is crucial for safe driving. When using a monocular camera for distance measurement, the measurement error is relatively small when the vehicle is traveling on a flat road. However, when the vehicle is traveling on a slope, the measurement error increases significantly due to changes in the viewing angle and distance. Therefore, it is necessary to accurately calculate the slope of the road where the vehicle is located to correct the measurement error of the monocular camera.

[0003] However, existing technologies typically calculate road slope gradients based on factors such as vehicle tilt angle and ground angle. This approach relies on accurate vehicle parameters and static measurement data, which cannot handle complex terrain changes and dynamic driving scenarios. Consequently, the calculated road slope gradients are less accurate, which in turn reduces the accuracy of obstacle distance measurement results. Summary of the Invention

[0004] This application provides an obstacle ranging method, device, electronic device, and storage medium to solve the problem that the accuracy of obstacle ranging results is reduced because the road slope is usually calculated based on the vehicle tilt angle and the ground angle in the prior art.

[0005] In a first aspect, embodiments of this application provide an obstacle ranging method, the method comprising:

[0006] Acquire image information of the road behind the vehicle;

[0007] The system identifies straight lines in the image information and selects a group of parallel lines that are parallel to a first preset direction from the straight lines. The group of parallel lines that are parallel to the first preset direction includes straight lines corresponding to multiple parallel lane lines on the road.

[0008] Using the set of parallel lines parallel to the first preset direction, multiple vanishing point positions are determined, and the road slope is determined based on the multiple vanishing point positions, wherein the multiple vanishing point positions are located in different multiple preset directions, and the multiple preset directions include the first preset direction;

[0009] The distance between the obstacle and the vehicle is calculated based on the slope of the road, and the distance measurement result of the obstacle is obtained.

[0010] Optionally, identifying the straight line in the image information includes:

[0011] The image information is filtered and denoised, and edge detection is performed on the filtered and denoised image information to obtain the edge information corresponding to the image information;

[0012] Multiple candidate lines are detected from the edge information using an improved Hough transform algorithm;

[0013] The candidate lines are filtered and fitted to obtain the straight line.

[0014] Optionally, the step of detecting multiple candidate lines from the edge information using the improved Hough transform algorithm includes:

[0015] The edge information is downsampled multiple times to obtain edge information corresponding to multiple scales;

[0016] A Hough transform is performed on the edge points contained in the edge information corresponding to each of the multiple scales, and the values ​​in the accumulator space are updated according to the Hough transform results of each edge point. The accumulator space is used to count the number of detected edge points located on each straight line.

[0017] Multiple straight lines whose cumulative number of edge points reaches a preset threshold are identified as the multiple candidate lines.

[0018] Optionally, determining multiple vanishing point positions using the set of parallel lines parallel to the first preset direction includes:

[0019] Cluster the group of parallel lines parallel to the first preset direction to obtain K candidate vanishing point positions;

[0020] Calculate the reprojection error of each of the K candidate vanishing point positions, and determine the candidate vanishing point position with the smallest reprojection error as the first vanishing point position in the first preset direction.

[0021] An equivalent sphere is constructed based on the first vanishing point position, and the second vanishing point position in a second preset direction is determined from the equivalent sphere, wherein the second preset direction is any direction in a plane perpendicular to the first preset direction;

[0022] Based on the second vanishing point position, a third vanishing point position in a third preset direction is determined from the equivalent sphere, wherein the third preset direction is a direction perpendicular to the first preset direction and the second preset direction, and the first vanishing point position, the second vanishing point position, and the third vanishing point position all belong to the plurality of vanishing point positions.

[0023] Optionally, the formula for determining the slope of the road is as follows:

[0024]

[0025] Wherein, θ represents the slope of the road, x1 and y1 represent the positions of the first vanishing point, and x3 and y3 represent the positions of the third vanishing point.

[0026] Optionally, the step of calculating the distance between the obstacle and the vehicle based on the road slope gradient to obtain the distance measurement result of the obstacle includes:

[0027] Determine the coordinate position of the obstacle in the image information;

[0028] Based on the coordinate position of the obstacle in the image information and the preset camera parameters, the theoretical distance between the obstacle and the vehicle at the same road height is determined;

[0029] Based on the road slope gradient and the theoretical distance, the actual distance between the obstacle and the vehicle is calculated, and the actual distance is determined as the distance measurement result of the obstacle.

[0030] Optionally, before identifying the straight line in the image information, the method further includes:

[0031] Identify lane lines in the image information and determine whether the slope of the lane lines has changed;

[0032] If the slope of the lane line changes, the following step is performed: identify the straight line in the image information.

[0033] Secondly, embodiments of this application also provide an obstacle ranging device, the device comprising:

[0034] The acquisition module is used to acquire image information of the road behind the vehicle;

[0035] The first recognition module is used to recognize straight lines in the image information and select a group of parallel lines that are parallel to a first preset direction from the straight lines, wherein the group of parallel lines that are parallel to the first preset direction includes straight lines corresponding to multiple parallel lane lines on the road.

[0036] The determining module is used to determine multiple vanishing point positions using the set of parallel lines parallel to the first preset direction, and to determine the road slope based on the multiple vanishing point positions, wherein the multiple vanishing point positions are located in different multiple preset directions, and the multiple preset directions include the first preset direction;

[0037] The calculation module is used to calculate the distance between the obstacle and the vehicle based on the slope of the road, and to obtain the distance measurement result of the obstacle.

[0038] Thirdly, embodiments of this application also provide an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0039] Memory, used to store computer programs;

[0040] The processor, when executing a program stored in memory, implements the obstacle ranging method described in any one of the first aspects.

[0041] Fourthly, embodiments of this application also provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the obstacle ranging method described in any one of the first aspects.

[0042] Compared with the prior art, the technical solution provided in this application has the following advantages: The method provided in this application acquires image information of the road behind the vehicle; identifies straight lines in the image information, and selects a group of parallel lines parallel to a first preset direction from the straight lines, wherein the group of parallel lines parallel to the first preset direction includes straight lines corresponding to multiple parallel lane lines on the road; using the group of parallel lines parallel to the first preset direction, determines multiple vanishing point positions, and determines the road slope gradient based on the multiple vanishing point positions, wherein the multiple vanishing point positions are located in different multiple preset directions, and the multiple preset directions include the first preset direction; calculates the distance between the obstacle and the vehicle according to the road slope gradient to obtain the distance measurement result of the obstacle. By using the above method, the positions of multiple vanishing points in multiple preset directions can be determined by using a group of parallel lines identified from image information that are parallel to the first preset direction. Then, the road slope can be calculated based on these multiple vanishing point positions without relying on vehicle parameters and static measurement data. Therefore, even in complex terrain changes and dynamic driving scenarios, the road slope can be accurately calculated, thereby improving the accuracy of obstacle distance measurement results. Attached Figure Description

[0043] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0044] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0046] Figure 1 A flowchart illustrating an obstacle ranging method provided in an embodiment of this application;

[0047] Figure 2 A schematic diagram of an equivalent sphere provided for an embodiment of this application;

[0048] Figure 3 A schematic diagram of the geometric relationship for calculating the slope θ of a road is provided in an embodiment of this application;

[0049] Figure 4 A flowchart illustrating yet another obstacle ranging method provided in this application embodiment;

[0050] Figure 5 This application provides a schematic diagram of the distribution of feature points of a lane line according to an embodiment of the present application.

[0051] Figure 6 This is a schematic diagram of the structure of an obstacle ranging device provided in an embodiment of this application;

[0052] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0054] The following disclosure provides numerous different embodiments or examples for implementing various structures of the invention. To simplify the disclosure, specific examples of components and arrangements are described below. These are merely examples and are not intended to limit the scope of the invention. Furthermore, reference numerals and / or letters may be repeated in different examples. Such repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed.

[0055] To address the problem that existing technologies typically calculate road slope based on vehicle tilt angles and ground angles, leading to decreased accuracy in obstacle ranging results, this application provides an obstacle ranging method, apparatus, electronic device, and storage medium that can improve the accuracy of obstacle ranging results.

[0056] See Figure 1 , Figure 1 This is a flowchart illustrating an obstacle ranging method provided in an embodiment of this application. Figure 1 As shown, the obstacle ranging method may include the following steps:

[0057] Step 101: Obtain image information of the road behind the vehicle.

[0058] Specifically, when acquiring image information of the road behind the vehicle, it can be obtained using a camera mounted on the electronic rearview mirror or a camera mounted at the rear of the vehicle; this application embodiment does not impose a specific limitation. It should be noted that the image information of the road behind the vehicle includes lane lines, road boundary lines, and other information.

[0059] Step 102: Identify straight lines in the image information and select a group of parallel lines that are parallel to the first preset direction from the straight lines. The group of parallel lines that are parallel to the first preset direction includes straight lines corresponding to multiple parallel lane lines on the road.

[0060] In this step, an improved Hough transform algorithm or other algorithms can be used to detect lines in the image information to identify straight lines. Then, a group of parallel lines parallel to a first preset direction is selected from these lines. It should be noted that the first preset direction can be understood as a direction parallel to the perpendicular direction of the image information. The group of parallel lines parallel to the first preset direction includes at least the straight lines corresponding to multiple parallel lane lines on the road. Thus, the vanishing point of the lane lines in the image can be determined based on the straight line parameters corresponding to the lane lines.

[0061] Step 103: Using a set of parallel lines parallel to the first preset direction, determine multiple vanishing point positions, and based on the multiple vanishing point positions, determine the road slope gradient. The multiple vanishing point positions are located in different preset directions, including the first preset direction.

[0062] Specifically, in three-dimensional space, all parallel lines converge at a point at infinity. The projection of this point into a perspective projection image is called the vanishing point.

[0063] In this step, a set of parallel lines parallel to the first preset direction can be used to determine the vanishing point position in the first preset direction. Then, based on the vanishing point position in the first preset direction, vanishing points in other preset directions can be determined on the equivalent sphere of the vanishing point position in the first preset direction. Then, based on these multiple vanishing point positions, the rotation angle of the camera can be determined, that is, the slope of the road can be determined.

[0064] Step 104: Calculate the distance between the obstacle and the vehicle based on the road slope to obtain the distance measurement result of the obstacle.

[0065] After determining the slope of the road, geometric relationships can be used to calculate the actual distance between the obstacle and the vehicle, thus obtaining the distance measurement result of the obstacle.

[0066] In this embodiment, a group of parallel lines parallel to the first preset direction identified from image information can be used to determine the positions of multiple vanishing points in multiple preset directions. Then, the road slope can be calculated based on these multiple vanishing point positions without relying on vehicle parameters and static measurement data. Therefore, even in complex terrain changes and dynamic driving scenarios, the road slope can be accurately calculated, thereby improving the accuracy of obstacle ranging results.

[0067] Furthermore, step 102 above, identifying straight lines in the image information, includes:

[0068] The image information is filtered and denoised, and then edge detection is performed on the filtered and denoised image information to obtain the edge information corresponding to the image information;

[0069] Multiple candidate lines were detected from edge information using an improved Hough transform algorithm;

[0070] Multiple candidate lines are filtered and fitted to obtain a straight line.

[0071] Specifically, since real-world image information often contains noise, it can be smoothed to remove noise before edge detection. This can be achieved using various filters, such as Gaussian filters, mean filters, median filters, and bilateral filters. As an optional implementation, a Gaussian filter can be used for noise reduction.

[0072] The core of edge detection lies in calculating the gradient of an image. The magnitude and direction of the gradient represent the intensity and direction of changes in image brightness. The gradient can be calculated using the first or second derivative. The gradient magnitude G is typically calculated using the Sobel operator or the Prewitt operator, which are special cases of the spatial difference operator used to estimate the first derivative of the image. The formulas for calculating the gradient magnitude G and direction θ of the Sobel operator are:

[0073]

[0074] Among them, G x G is the gradient component of the image brightness function in the horizontal direction. y G is the gradient component of the image brightness function in the vertical direction. x and G y G is given by the Sobel operator. x =|H x *I|,G y =|H y *I|,H x and H y All are convolution kernels of the Sobel operator, and I represents the original image information.

[0075] After calculating the gradient magnitude using the above formula, non-maximum suppression (NMS) can be applied to the gradient magnitude of each pixel to determine image edges. If a pixel is not a local maximum in the gradient direction, it is considered not part of an edge. Finally, by setting a high threshold and a low threshold, pixels that truly belong to the edge are further filtered out. A pixel is retained as part of an edge only if its gradient magnitude is higher than the high threshold, or if it is connected to a neighboring point with a gradient magnitude higher than the high threshold. In this way, the edge information corresponding to the image information can be obtained.

[0076] After obtaining the edge information corresponding to the image information, an improved Hough Transform algorithm can be used to detect multiple candidate lines from the edge information. It should be noted that the Hough Transform is an algorithm for detecting lines in an image. It maps lines in image space to a parameter space, such that all collinear points intersect at a single point in the parameter space. The basic principle of the Hough Transform is that in a Cartesian coordinate system, any straight line can be represented as y = mx + b, where m is the slope of the line and b is the y-intercept. However, the Hough Transform prefers to represent lines using polar coordinates, i.e., ρ = x*cos(θ) + y*sin(θ), where ρ is the distance from the origin to the line, and θ is the angle between the line and the x-axis. The traditional Hough transform algorithm works as follows: First, edge detection is performed on the input image to obtain edge information. Then, an accumulator space is created for the Hough transform, where each cell corresponds to possible line parameters (such as ρ and θ). Next, each pixel in the edge information is traversed, and based on the coordinates of each pixel, its corresponding line parameter in the accumulator space is calculated, and the count of the corresponding cell is incremented. Afterward, local maxima are searched in the accumulator space; these local maxima represent the parameters of the lines in the image. Finally, the lines in the image are determined based on these local maxima. Using this method, the Hough transform can identify line segments in an image and exhibits good robustness even in the presence of noise and incomplete lines.

[0077] While the Hough transform is highly effective in detecting geometric shapes such as lines and circles in images, it suffers from several significant drawbacks, including high computational cost, high memory consumption, sensitivity to noise, difficulty in handling complex shapes, and challenges in parameter selection. These limitations restrict the use of the Hough transform in real-time or resource-constrained applications, and require carefully designed parameter tuning to optimize its performance. Therefore, this application optimizes and improves the traditional Hough transform by using an improved Hough transform algorithm to detect multiple candidate lines from edge information. These candidate lines are then filtered and fitted to obtain the straight lines in the image.

[0078] The purpose of line filtering is to remove noise and false positives from the Hough transform accumulator space, thus preserving the features that truly represent a line. A threshold is set for each cell in the accumulator space; only cells exceeding this threshold are considered valid evidence of a line. Local maxima are found in the accumulator space, which typically correspond to the parameters of the line. All points smaller than the local maximum are removed from the neighborhood of each local maximum; this is known as non-maximum suppression (NMS).

[0079] Line fitting determines the precise parameters of a line based on the filtered results. Parameters such as slope and intercept are extracted from the local maxima in accumulator space. The extracted parameters are then used to fit the line in image space. In Cartesian coordinates, a line can be represented as y = mx + b, where m is the slope and b is the y-intercept. Fitting involves using regression analysis to minimize the sum of squared errors in the line equation.

[0080] Using the above method, straight lines in the image information of the road behind the vehicle can be accurately identified, which facilitates subsequent vanishing point detection.

[0081] Furthermore, the above steps utilize an improved Hough transform algorithm to detect multiple candidate lines from the edge information, including:

[0082] The edge information is downsampled multiple times to obtain edge information corresponding to multiple scales;

[0083] A Hough transform is performed on the edge points contained in the edge information corresponding to each scale in the edge information corresponding to multiple scales, and the values ​​in the accumulator space are updated according to the Hough transform results of each edge point. The accumulator space is used to count the number of detected edge points located on each straight line.

[0084] Multiple straight lines whose cumulative number of edge points reaches a preset threshold are identified as multiple candidate lines.

[0085] In one embodiment, when detecting multiple candidate lines from edge information using the improved Hough transform algorithm, edge information corresponding to multiple scales can be generated through continuous downsampling, as expressed by the formula: I s =G*I s-1 , among which, I s It is an image at scale s, G is a downsampling filter, and I s-1 This is the image at scale s-1. At scale s, a straight line can be represented as: y i =m s *x i +b s , where m s and b s Here, ρ represents the slope and intercept of the line at scale s, respectively. The line equation is converted to polar coordinates for accumulator updates in Hough space: ρ = x·cos(θ) + y·sin(θ). For each edge point, voting is performed on the corresponding line in the accumulator space: A(ρ,θ) = A(ρ,θ) + 1. Finally, multiple lines whose cumulative edge point count reaches a preset threshold are identified as candidate lines. It should be noted that the detected line parameters can also be converted from different scales to a normalized scale for a unified representation of the results. The normalization formula can be:

[0086]

[0087] Traditional Hough transform algorithms can typically only detect shapes at a specific scale, while improved Hough transform algorithms can detect shapes at multiple scales, thus increasing the flexibility and applicability of the algorithm.

[0088] Furthermore, in step 103 above, using a group of parallel lines parallel to the first preset direction, multiple vanishing point positions are determined, including:

[0089] Cluster the group of parallel lines that are parallel to the first preset direction to obtain K candidate vanishing point positions;

[0090] Calculate the reprojection error of each of the K candidate vanishing point positions, and determine the candidate vanishing point position with the smallest reprojection error as the first vanishing point position in the first preset direction.

[0091] An equivalent sphere is constructed based on the first vanishing point position, and the second vanishing point position in the second preset direction is determined from the equivalent sphere. The second preset direction is any direction in a plane perpendicular to the first preset direction.

[0092] Based on the second vanishing point position, the third vanishing point position in the third preset direction is determined from the equivalent sphere. The third preset direction is a direction perpendicular to the first preset direction and the second preset direction. The first vanishing point position, the second vanishing point position, and the third vanishing point position are all multiple vanishing point positions.

[0093] In one embodiment, the parallel line group parallel to the first preset direction can be clustered to obtain K candidate vanishing point positions. Then, the reprojection error of each of the K candidate vanishing point positions is calculated, and the candidate vanishing point position with the smallest reprojection error is determined as the first vanishing point position in the first preset direction. Next, an equivalent sphere is constructed based on the first vanishing point position, and the second vanishing point position in the second preset direction is determined from the equivalent sphere. Finally, based on the second vanishing point position, the third vanishing point position in the third preset direction is determined from the equivalent sphere.

[0094] Specifically, in a Cartesian coordinate system, the direction of a straight line can be represented by its slope m. The slope m can be calculated from two points (x1, y1) and (x2, y2) of the line, as shown in the following formula:

[0095]

[0096] Vanishing points are typically categorized into three types, corresponding to parallel lines in three main directions. The first vanishing point corresponds to lines parallel to the vertical direction of the image; the second vanishing point corresponds to lines parallel to the horizontal direction of the image; and the third vanishing point corresponds to lines parallel to the diagonal direction of the image. For the first vanishing point, the detected lines are categorized according to their direction. This can be done by mapping the slope to a one-dimensional space, for example, using a radian value θ to represent the direction of the line: θ = arctan(m). Cluster analysis is performed on the lines in each direction category to find the central tendency of the lines. Specifically, the K-means clustering algorithm can be used, where K equals the number of candidate vanishing points, and the set of lines corresponding to each cluster center represents a set of coplanar parallel lines. The coordinates of the cluster centers can be used as the locations of the candidate vanishing points. The objective function of the K-means clustering algorithm is shown in the following formula:

[0097]

[0098] Where, x i Let μ represent the i-th data point. j Let K represent the j-th cluster center, K be the number of cluster centers, and N be the total number of data points.

[0099] Due to image noise and other factors, further optimization and precise localization of candidate vanishing points are required. This can be achieved by minimizing the reprojection error. Specifically, the formula for the reprojection error is as follows:

[0100]

[0101] Among them, L i P is the equation of the i-th line. i D is a point on the i-th straight line. i It is point P i In the image, M represents the number of lines considered, i.e., the total number of lines parallel to the first preset direction. Therefore, the location of the first vanishing point can be obtained through a clustering algorithm.

[0102] After determining the first vanishing point position p1, the second vanishing point position p2 can be obtained on the outer ring of the equivalent sphere of p1. The second vanishing point position p2 corresponds to the vanishing point of a set of parallel lines orthogonal to p1. Considering orthogonality constraints, we can... Figure 2 A point is randomly selected on the dashed circle shown to serve as the second vanishing point position p2.

[0103] In perspective geometry, the third vanishing point is orthogonal to the first and second vanishing points. Given two vanishing points P1 and P2, the third vanishing point P3 can be found using the cross product of vectors. The cross product is a method for calculating the pseudo-vector in the perpendicular direction of two vectors in three-dimensional space. First, the vanishing points P1 and P2 need to be represented as vectors on the image plane. If P1 = (x1, y1) and P2 = (x2, y2), then the vector P1P2 can be represented as:

[0104]

[0105] The cross product is a scalar representing the directed area of ​​the parallelogram formed by the two vectors. In two dimensions, the direction of the unit vector perpendicular to both vectors can be obtained by calculating the cross product. Their cross product is:

[0106] P1×P2=x1y2-x2y1

[0107] In the equivalent spherical model, all vanishing points lie on the sphere. Therefore, the location of the third vanishing point can be determined.

[0108] Using the above method, a group of parallel lines parallel to the first preset direction can be used to accurately determine the locations of multiple vanishing points, which facilitates the subsequent accurate calculation of the road slope based on the locations of multiple vanishing points.

[0109] Furthermore, the formula for determining the slope of a road is as follows:

[0110]

[0111] Where θ represents the road slope, x1 and y1 represent the first vanishing point position, and x3 and y3 represent the third vanishing point position.

[0112] It should be noted that the first vanishing point position P1 represents the vanishing point position corresponding to a line perpendicular to the image direction (i.e., the Z-axis), the second vanishing point position P2 represents the vanishing point position corresponding to a line parallel to the horizontal direction of the image (i.e., the x-axis), and the third vanishing point position P3 represents the vanishing point position corresponding to a line parallel to the vertical direction of the image (i.e., the y-axis). Therefore, the camera rotation angle, i.e., the road slope θ, can be determined based on the angle between the lines containing the first and third vanishing point positions P1 and P3 and the Z-axis. Specifically, as... Figure 3 As shown, since the slope m of a straight line is the ratio of the ordinate to the abscissa of the direction vector, i.e. m = (y3 - y1) / (x3 - x1), and the angle θ between the straight line and the Z-axis can be calculated by the slope m, the road slope is calculated by the arctangent function as θ = arctan(m). Therefore, the above formula can be deduced.

[0113] Using the above formula, the road slope can be accurately calculated based on the positions of the first and third vanishing points.

[0114] Further, step 104 above, calculating the distance between the obstacle and the vehicle based on the road slope gradient, yields the obstacle distance measurement result, including:

[0115] Determine the coordinate position of the obstacle in the image information;

[0116] Based on the coordinates of the obstacle in the image information and the preset camera parameters, the theoretical distance between the obstacle and the vehicle at the same road height is determined.

[0117] Based on the road slope and theoretical distance, the actual distance between the obstacle and the vehicle is calculated, and the actual distance is determined as the distance measurement result of the obstacle.

[0118] In one embodiment, when calculating the distance between an obstacle and a vehicle based on the road slope, the coordinates of the obstacle in the image information can first be determined using a target detection algorithm. As an optional implementation, the center point coordinates (xbase, ybase) of the obstacle's bottom border can be obtained using a target detection algorithm, and this coordinates can be used as an estimate of the obstacle's ground contact point for distance calculation. Then, based on the obstacle's coordinates in the image information and preset camera parameters, the theoretical distance d between the obstacle and the vehicle at the same road height is determined. image Next, the distance between the obstacle and the vehicle is calculated based on the road slope angle θ, yielding the obstacle's distance measurement result. Specifically, the Pythagorean theorem combined with the slope angle can be used to estimate the actual distance d of the obstacle. actual The formula is as follows:

[0119]

[0120] Where, d actual d represents the actual distance between the obstacle and the vehicle. image θ represents the theoretical distance, and θ represents the road slope.

[0121] The above method allows for a more accurate determination of the actual distance to obstacles. This can be applied to the blind spot detection and warning function of electronic rearview mirrors. The algorithm uses image processing technology to identify edges, straight lines, and vanishing points, and determines the position and distance of obstacles accordingly. This function can promptly detect obstacles in blind spots, providing drivers with warnings and thus enhancing driving safety.

[0122] Furthermore, before step 101 above, which identifies straight lines in the image information, the method further includes:

[0123] Identify lane lines in the image information and determine whether the slope of the lane lines has changed;

[0124] When the slope of the lane lines changes, the following steps are performed: identify the straight lines in the image information.

[0125] In one embodiment, continuously calculating the road slope gradient in real time would consume significant computing power. Therefore, it is advisable to first determine if a road slope exists before calculating the distance, thus achieving more efficient and accurate distance measurement. The specific process is as follows: Figure 4 As shown, image information is first obtained through the electronic rearview mirror, and then lane line detection is performed to extract the feature points of the lane lines. For example, the feature points of a lane line on a slope are shown below. Figure 5 As shown, pl and pr represent a series of points, G1 and G2 represent different road surfaces, G1 represents a flat road, and G2 represents a road with a slope.

[0126] The main purpose of lane detection is to analyze road surface features using slope changes to determine the gradient changes of the road behind the vehicle. From a visual perspective, when the gradient of the road behind changes, the lane line will show an inflection point at the road surface boundary. Therefore, after fitting the lane line, the first step is to extract a series of points on the lane line and calculate the slope between adjacent points:

[0127]

[0128] Among them, pl i.y and pl i+1.y It is a feature point on one of the lane lines, pr j.y and pr j+1.y These are feature points on another lane, where i and j are integers greater than 1.

[0129] Next, the slope of the road surface is analyzed based on the changes in these slopes kl and kr, and the road segment is divided into different areas. In this way, the slope changes of the road can be understood more accurately. If there is a slope change, the slope needs to be calculated, and then the distance to the obstacle needs to be calculated. If there is no slope change, the slope calculation is not required, and the distance to the obstacle can be calculated directly.

[0130] In this embodiment, computer vision technology can be used to infer the slope angle of the ground by identifying vanishing points in image information, thereby calculating the slope gradient. This method uses road images captured by an electronic rearview mirror to detect parallel lines and determine their vanishing point positions. By measuring the offset of the vanishing point relative to the image center, and using camera parameters such as focal length and field of view, the slope angle of the ground can be inferred. Compared to traditional methods, this embodiment does not rely on vehicle parameters, is suitable for dynamic scenes, and can more accurately capture slope changes in complex terrain. Finally, an object detection algorithm determines the position and size of objects in the image. Using the camera's intrinsic and extrinsic parameters, as well as slope information, the pixel coordinates in the image can be converted into real-world coordinates, thus providing a more accurate distance from the vehicle to obstacles behind it, and enabling blind spot warning functionality based on this distance.

[0131] This application brings significant benefits to the blind spot detection and warning function of electronic rearview mirrors. By comprehensively utilizing key technologies such as image processing, line detection, vanishing point detection, slope calculation, and target detection, this solution achieves accurate estimation of obstacle distances. It can precisely identify obstacle positions, effectively avoiding collision risks within blind spots and improving driving safety. Simultaneously, it fully considers ground tilt factors, making distance estimation more accurate and reliable. This technological innovation provides drivers with a more comprehensive perception of road conditions, offering strong support for driving safety.

[0132] See Figure 6 , Figure 6 This is a schematic diagram of an obstacle ranging device provided in an embodiment of this application. Figure 6 As shown, the device 600 includes:

[0133] The acquisition module 601 is used to acquire image information of the road behind the vehicle;

[0134] The first recognition module 602 is used to recognize straight lines in image information and select a group of parallel lines that are parallel to a first preset direction from the straight lines. The group of parallel lines that are parallel to the first preset direction includes straight lines corresponding to multiple parallel lane lines on the road.

[0135] The determining module 603 is used to determine multiple vanishing point positions using a group of parallel lines parallel to the first preset direction, and to determine the road slope based on the multiple vanishing point positions, wherein the multiple vanishing point positions are located in different multiple preset directions, and the multiple preset directions include the first preset direction;

[0136] The calculation module 604 is used to calculate the distance between obstacles and vehicles based on the road slope gradient, and obtain the distance measurement result of the obstacle.

[0137] Furthermore, the first identification module 602 includes:

[0138] The edge detection submodule is used to filter and denoise the image information, and then perform edge detection on the filtered and denoised image information to obtain the edge information corresponding to the image information.

[0139] The detection submodule is used to detect multiple candidate lines from edge information using an improved Hough transform algorithm;

[0140] The filtering and fitting submodule is used to filter and fit multiple candidate lines to obtain a straight line.

[0141] Furthermore, the detection submodule includes:

[0142] The downsampling unit is used to downsample the edge information multiple times to obtain edge information corresponding to multiple scales;

[0143] The Hough transform unit is used to perform Hough transform on the edge points contained in the edge information corresponding to each scale in the edge information corresponding to multiple scales, and update the values ​​in the accumulator space according to the Hough transform results of each edge point. The accumulator space is used to count the number of detected edge points located on each straight line.

[0144] The determination unit is used to identify multiple straight lines whose cumulative number of edge points reaches a preset threshold as multiple candidate lines.

[0145] Furthermore, module 603 is determined to include:

[0146] The clustering submodule is used to cluster the group of parallel lines that are parallel to the first preset direction to obtain K candidate vanishing point positions;

[0147] The first determining submodule is used to calculate the reprojection error of each of the K candidate vanishing point positions, and determine the candidate vanishing point position with the smallest reprojection error as the first vanishing point position in the first preset direction.

[0148] The second determining submodule is used to construct an equivalent sphere based on the first vanishing point position and determine the second vanishing point position in a second preset direction from the equivalent sphere, wherein the second preset direction is any direction in a plane perpendicular to the first preset direction;

[0149] The third determining submodule is used to determine the third vanishing point position in a third preset direction from the equivalent sphere based on the second vanishing point position. The third preset direction is a direction perpendicular to the first preset direction and the second preset direction. The first vanishing point position, the second vanishing point position, and the third vanishing point position all belong to multiple vanishing point positions.

[0150] Furthermore, the formula for determining the slope of a road is as follows:

[0151]

[0152] Where θ represents the road slope, x1 and y1 represent the first vanishing point position, and x3 and y3 represent the third vanishing point position.

[0153] Furthermore, the computing module 604 includes:

[0154] The fourth determination submodule is used to determine the coordinate position of obstacles in the image information;

[0155] The fifth determination submodule is used to determine the theoretical distance between the obstacle and the vehicle at the same road height based on the coordinate position of the obstacle in the image information and the preset camera parameters.

[0156] The sixth determination submodule is used to calculate the actual distance between the obstacle and the vehicle based on the road slope gradient and theoretical distance, and to determine the actual distance as the distance measurement result of the obstacle.

[0157] Furthermore, the device 600 also includes:

[0158] The second recognition module is used to identify lane lines in the image information and determine whether the slope of the lane lines has changed.

[0159] The execution module is used to perform the following steps when the slope of the lane line changes: identify straight lines in the image information.

[0160] It should be noted that the device 600 can implement the obstacle ranging method provided in any of the aforementioned method embodiments and achieve the same technical effect, which will not be elaborated here.

[0161] like Figure 7 As shown in the illustration, this application also provides an electronic device, including a processor 711, a communication interface 712, a memory 713, and a communication bus 714, wherein the processor 711, the communication interface 712, and the memory 713 communicate with each other via the communication bus 714.

[0162] Memory 713 is used to store computer programs;

[0163] In one embodiment of this application, when the processor 711 executes the program stored in the memory 713, it implements the obstacle ranging method provided in any of the foregoing method embodiments.

[0164] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the obstacle ranging method provided in any of the foregoing method embodiments.

[0165] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0166] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software plus a general-purpose hardware platform, or of course, using hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0167] It should be understood that the terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “described” as used herein may also include the plural forms. The terms “comprising,” “including,” “containing,” and “having” are inclusive and therefore indicate the presence of the stated features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or combinations thereof. The method steps, processes, and operations described herein are not construed as requiring them to be performed in a particular order described or illustrated unless the order of performance is explicitly indicated. It should also be understood that additional or alternative steps may be used.

[0168] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method of obstacle ranging, characterized by, The method includes: Acquire image information of the road behind the vehicle; The system identifies straight lines in the image information and selects a group of parallel lines that are parallel to a first preset direction from the straight lines. The group of parallel lines that are parallel to the first preset direction includes straight lines corresponding to multiple parallel lane lines on the road. Using the set of parallel lines parallel to the first preset direction, multiple vanishing point positions are determined, and the road slope is determined based on the multiple vanishing point positions, wherein the multiple vanishing point positions are located in different multiple preset directions, and the multiple preset directions include the first preset direction; The distance between the obstacle and the vehicle is calculated based on the road slope gradient to obtain the distance measurement result of the obstacle; The method of determining multiple vanishing point positions using the set of parallel lines parallel to the first preset direction includes: Cluster the group of parallel lines parallel to the first preset direction to obtain K candidate vanishing point positions; Calculate the reprojection error of each of the K candidate vanishing point positions, and determine the candidate vanishing point position with the smallest reprojection error as the first vanishing point position in the first preset direction. An equivalent sphere is constructed based on the first vanishing point position, and the second vanishing point position in a second preset direction is determined from the equivalent sphere, wherein the second preset direction is any direction in a plane perpendicular to the first preset direction; Based on the second vanishing point position, a third vanishing point position in a third preset direction is determined from the equivalent sphere, wherein the third preset direction is a direction perpendicular to the first preset direction and the second preset direction, the first vanishing point position, the second vanishing point position and the third vanishing point position all belong to the plurality of vanishing point positions, and the first vanishing point position, the second vanishing point position and the third vanishing point position are all located on the sphere of the equivalent sphere; The formula for calculating the slope of the road is as follows: ; in, This indicates the slope of the road. and This indicates the position of the first vanishing point. and This indicates the location of the third vanishing point.

2. The method according to claim 1, characterized in that, The process of identifying straight lines in the image information includes: The image information is filtered and denoised, and edge detection is performed on the filtered and denoised image information to obtain the edge information corresponding to the image information; Multiple candidate lines are detected from the edge information using an improved Hough transform algorithm; The candidate lines are filtered and fitted to obtain the straight line.

3. The method according to claim 2, characterized in that, The improved Hough transform algorithm is used to detect multiple candidate lines from the edge information, including: The edge information is downsampled multiple times to obtain edge information corresponding to multiple scales; A Hough transform is performed on the edge points contained in the edge information corresponding to each of the multiple scales, and the values ​​in the accumulator space are updated according to the Hough transform results of each edge point. The accumulator space is used to count the number of detected edge points located on each straight line. Multiple straight lines whose cumulative number of edge points reaches a preset threshold are identified as the multiple candidate lines.

4. The method according to claim 1, characterized in that, The step of calculating the distance between the obstacle and the vehicle based on the road slope gradient to obtain the distance measurement result of the obstacle includes: Determine the coordinate position of the obstacle in the image information; Based on the coordinate position of the obstacle in the image information and the preset camera parameters, the theoretical distance between the obstacle and the vehicle at the same road height is determined; Based on the road slope gradient and the theoretical distance, the actual distance between the obstacle and the vehicle is calculated, and the actual distance is determined as the distance measurement result of the obstacle.

5. The method according to claim 1, characterized in that, Before identifying the straight line in the image information, the method further includes: Identify lane lines in the image information and determine whether the slope of the lane lines has changed; If the slope of the lane line changes, the following step is performed: identify the straight line in the image information.

6. An obstacle ranging device, characterized in that, The device includes: The acquisition module is used to acquire image information of the road behind the vehicle; The first recognition module is used to recognize straight lines in the image information and select a group of parallel lines that are parallel to a first preset direction from the straight lines, wherein the group of parallel lines that are parallel to the first preset direction includes straight lines corresponding to multiple parallel lane lines on the road. The determining module is used to determine multiple vanishing point positions using the set of parallel lines parallel to the first preset direction, and to determine the road slope based on the multiple vanishing point positions, wherein the multiple vanishing point positions are located in different multiple preset directions, and the multiple preset directions include the first preset direction; The calculation module is used to calculate the distance between the obstacle and the vehicle based on the slope of the road, and to obtain the distance measurement result of the obstacle; The determining module includes: The clustering submodule is used to cluster the group of parallel lines parallel to the first preset direction to obtain K candidate vanishing point positions; The first determining submodule is used to calculate the reprojection error of each of the K candidate vanishing point positions, and determine the candidate vanishing point position with the smallest reprojection error as the first vanishing point position in the first preset direction. The second determining submodule is used to construct an equivalent sphere based on the first vanishing point position, and determine the second vanishing point position in a second preset direction from the equivalent sphere, wherein the second preset direction is any direction in a plane perpendicular to the first preset direction; The third determining submodule is used to determine the third vanishing point position in a third preset direction from the equivalent sphere based on the second vanishing point position, wherein the third preset direction is a direction perpendicular to the first preset direction and the second preset direction, the first vanishing point position, the second vanishing point position and the third vanishing point position all belong to the plurality of vanishing point positions, and the first vanishing point position, the second vanishing point position and the third vanishing point position are all located on the sphere of the equivalent sphere; The formula for calculating the slope of the road is as follows: ; in, This indicates the slope of the road. and This indicates the position of the first vanishing point. and This indicates the location of the third vanishing point.

7. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the obstacle ranging method according to any one of claims 1-5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the obstacle ranging method according to any one of claims 1-5.

Citation Information

Patent Citations

  • Lane video detection method based on half-machine study

    CN103383733A

  • Road slope calculation method and device based on monocular vision

    CN109900254A