A method, system and computer-readable storage medium for instant slope prediction based on binocular camera

Through a binocular camera, the road lane curve is detected, the auxiliary line equation is fitted, and the depth measurement is solved, and the problem of low prediction accuracy of forward slope when the car is driving is achieved, and the accurate measurement of forward slope and lateral slope is achieved, which is suitable for intelligent driving technology.

CN113345035BActive Publication Date: 2024-12-03WUHAN UNIV OF SCI & TECH
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202110805290.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-07-16
Publication Date
2024-12-03
Estimated Expiration
2041-07-16

AI Technical Summary

Technical Problem

The prior art cannot accurately predict the forward slope when a car is driving, especially the transverse slope cannot be measured, and the existing methods are costly or have low accuracy.

Method used

The real-time slope prediction method based on binocular camera is used to detect the road lane curve, fit the auxiliary line equation, and obtain the world coordinates of the reference point by depth measurement. The pre-stored slope solution formula is used to calculate the slope value of the road ahead to achieve the forward slope and Accurate prediction of transverse slope.

Benefits of technology

It realizes accurate prediction of the forward slope when the vehicle is driving, including measurement of longitudinal and transverse slopes, which improves accuracy and reduces costs, and is suitable for other assisted driving systems.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113345035B_ABST
    Figure CN113345035B_ABST
Patent Text Reader

Abstract

The present invention discloses a method and system for instant slope prediction based on a binocular camera and a readable storage medium for executing the method, specifically, detecting a road lane curve by a binocular camera; making two auxiliary lines perpendicular to the lane curve, wherein the auxiliary lines intersect the lane lines at four reference points; obtaining the coordinates of the four reference points in a pixel coordinate system by equations and auxiliary line equations; obtaining the coordinates of the four reference points in a world coordinate system by depth measurement; using the coordinate values ​​of the four reference points in the world coordinate system as input quantities, calculating and outputting the slope value of the road ahead by a pre-stored slope solution formula; the solution solves the problem that the prediction accuracy of the front slope is not high and the lateral slope cannot be measured during the driving of the vehicle, realizes the accurate prediction of the front slope and the simultaneous measurement of the lateral slope when the vehicle is driving, and also has the characteristics of high precision.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of automobile safety technology, and in particular to a real-time slope prediction method based on a binocular camera, which can accurately predict the front slope when a vehicle is traveling and realizes measurement of the lateral slope, and a system for realizing the method. Background Art

[0002] As the pace of development of smart cars continues to advance, for a moving vehicle, real-time acquisition of the slope of the road ahead is particularly important for effectively and accurately controlling vehicle operation.

[0003] In the existing road slope measurement, most of them use the method of combining sensors with vehicle dynamics, but this method can only measure the slope of the vehicle's current position and cannot be used to predict the slope of the road ahead. There is also a method of installing multiple radars to obtain the slope of the road ahead, but this method is too expensive and difficult to popularize.

[0004] Most of the current methods for predicting road slopes based on monocular cameras can predict the slope of the road ahead at a low cost, but the measurement accuracy is also low, and can only predict the slope value of a straight road, but cannot predict the lateral slope value. Summary of the invention

[0005] The purpose of the present invention is to solve the problem that the prediction accuracy of the front slope is not high and the lateral slope cannot be measured during the driving of a car, and to provide a real-time slope prediction method based on a binocular camera and a system for implementing the method. The prediction method and system are used to accurately predict the front slope and simultaneously measure the lateral slope when the vehicle is driving.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] A slope instant prediction method based on a binocular camera, the specific steps are:

[0008] Detect road lane curves through binocular cameras;

[0009] Draw two auxiliary lines perpendicular to the lane curve, and the auxiliary lines intersect with the lane line at four reference points;

[0010] The coordinates of the four reference points in the pixel coordinate system are calculated through the lane line fitting equation and the auxiliary line equation;

[0011] The depth measurement obtains the coordinates of the world coordinate system of the four reference points;

[0012] The coordinate values ​​of the world coordinate system of the four reference points are used as input, and the slope value of the road ahead is calculated and output through the pre-stored slope solution formula.

[0013] Detect road lane curves through binocular cameras, specifically:

[0014] The front lane line image is obtained through the left and right binocular cameras, and the internal and external parameters of the left and right binocular cameras are obtained through camera calibration;

[0015] Correct the left and right images so that they are in the same plane and parallel to each other;

[0016] The image coordinates of the lane feature points are obtained through the lane detection method; and the lane fitting quadratic function obtained by the least squares method is:

[0017]

[0018] where a 1Y 、b 1Y 、c 1Y with a 2Y 、b 2Y 、c 2Y are the coefficients of the lane line function of the first auxiliary line and the second auxiliary line in the right eye camera; a 1z 、b 1z 、c 1z with a 2z 、b 2z 、c 2z They are the coefficients of the lane line function of the first auxiliary line and the second auxiliary line in the left eye camera respectively.

[0019] For a straight line, the auxiliary line equation is the initial auxiliary line equation, which is obtained as follows:

[0020] Make two reference lines perpendicular to the lane lines on the road in the world coordinate system to ensure that the optical axis of the binocular camera is parallel to the lane lines. Use the lane line detection method to obtain the auxiliary lines of the two reference lines in the left and right binocular cameras. 1Z , 1 2Z with 1 1Y , 1 2Y The function expression is as follows

[0021]

[0022] where k 1Z 、b 1Z With k 1Y 、b 1Y are the slope and intercept of the first auxiliary line in the left eye camera and the right eye camera respectively; k 2Z 、b 2Z With k 2Y 、b 2Y are the slope and intercept of the second auxiliary line in the left eye camera and the right eye camera respectively;

[0023] For curved roads, the auxiliary line equation is used to predict the steering angle α of the road ahead using the steering angle prediction module;

[0024] Obtain the auxiliary line rotation angle β according to the steering angle of the road ahead;

[0025] α=β; get the function expression of the rotated auxiliary line in the left and right camera images:

[0026]

[0027] where k' 1Z , k' 2Z , k' 1Y , k' 1Y are the slopes of the function expressions of the rotated auxiliary lines in the left and right camera images, respectively. The calculation formula is as follows:

[0028]

[0029] The lane line fitting equation and the auxiliary line equation respectively obtain the coordinates of the auxiliary points in the left and right images, and the auxiliary points are P11 left, P21 left, P12 left, P22 left and P11 right, P21 right, P12 right, P22 right.

[0030] The world coordinates of the four auxiliary points obtained by the binocular camera depth measurement are as follows:

[0031] P 11 (X 11 , Y 11 , Z 11 ), P 21 (X 21 , Y 21 , Z 21 ), P 12 (X 12 , Y 12 , Z 12 ), P 22 (X 22 , Y 22 , Z 22 ).

[0032] A slope instant prediction system based on binocular camera, including

[0033] Binocular camera, used to obtain the lane line image ahead,

[0034] Processor, including

[0035] The calibration module is used to obtain the intrinsic and extrinsic parameters of the left and right binocular cameras through camera calibration; correct the left and right images so that the two images are located in the same plane and parallel to each other;

[0036] The lane line feature point image acquisition module is used to make two auxiliary lines perpendicular to the lane curve. The auxiliary lines intersect the lane line at four reference points. The image coordinates of the lane line feature points are obtained through the lane line detection method.

[0037] A lane line fitting module is used to fit the lane line feature points into quadratic function curve segments by using the least squares method; and to obtain the coordinates of four reference points in the pixel coordinate system by using the lane line fitting equation and the auxiliary line equation;

[0038] A depth detection module is used to measure the depth and obtain the coordinates of the world coordinate system of four reference points;

[0039] The slope calculation module is used to take the coordinate values ​​of the world coordinate system of the four reference points as input, calculate and output the slope value of the road ahead through the pre-stored slope solution formula.

[0040] The output module is used to output the relative transverse and longitudinal slope values ​​of the road ahead.

[0041] A computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor is caused to execute any of the methods described above.

[0042] Compared with the prior art, the beneficial effects of the present invention are:

[0043] The present invention can predict the slope value of the road ahead in real time through a binocular camera, and can also predict the transverse slope value and the longitudinal slope value of the curve.

[0044] The present invention calculates the slope value by projecting the auxiliary points on the lane line into the world coordinates, with higher accuracy.

[0045] The results obtained by the lane line detection module and the steering angle prediction module in the present invention can be used in other auxiliary driving systems or intelligent driving technologies, such as lane departure warning, automatic driving above L3, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a flow chart of the real-time slope prediction method based on binocular camera of the present invention;

[0047] Figure 2 This is a schematic diagram for explaining the calculation of the cross slope;

[0048] Figure 3 A bird's-eye view diagram for the calculation of longitudinal slope;

[0049] Figure 4 This is a schematic diagram showing the front view of the longitudinal slope calculation. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] The instant slope prediction method based on a binocular camera includes a lane line fitting step and a road slope calculation step.

[0052] Embodiment 1, as Figure 1 As shown:

[0053] A real-time slope prediction method based on a binocular camera, the method comprising:

[0054] S10: Acquire the front lane line image through the left and right binocular cameras; and acquire the internal and external parameters of the left and right cameras through camera calibration.

[0055] S20: For the lane line image, the image coordinates of the lane line feature points are obtained by a lane line detection method, and the fitting quadratic function of the lane line is obtained by a lane line fitting method.

[0056] S30: The auxiliary line function pre-stored in the system and the lane line function calculated in S2 are combined to obtain the image coordinates of the four auxiliary points in the left and right cameras respectively.

[0057] S40: Using the depth detection module of the binocular camera to obtain the world coordinates corresponding to the four auxiliary points.

[0058] S50: Calculate the forward slope value through the slope calculation method.

[0059] In order to clearly explain the real-time slope prediction method based on binocular cameras, Figure 2 Each step in the implementation example of the method of the present invention is described in detail.

[0060] The real-time slope prediction method based on a binocular camera according to an embodiment of the present invention includes steps S10 to S50, and each detailed step is as follows:

[0061] Step S10: Acquire the front lane line image through the left and right binocular cameras; acquire the internal and external parameters of the left and right cameras through camera calibration; and correct the left and right images.

[0062] In one embodiment of the present invention, a binocular camera is installed on the top of a vehicle to ensure that the road in front of the vehicle can be clearly photographed.

[0063] In this example, a binocular camera is used, but this function can also be achieved by two monocular cameras; the calibration method used in this example is Zhang Zhengyou's calibration method, but other binocular camera calibration methods can also be used. The present invention does not restrict the above two items.

[0064] The two images are corrected by the calibrated internal and external parameters so that the two images are located in the same plane and parallel to each other.

[0065] Step S20: For the lane line image, the image coordinates of the lane line feature points are obtained by a lane line detection method, and a lane line fitting quadratic function is obtained by a lane line fitting module.

[0066] The specific steps of the lane line detection method are:

[0067] S21: Image grayscale:

[0068] There are usually three methods for grayscale image processing of color images: mean method, maximum method and weighted average method. The mean method is to take the average of the three components of R, G, and B to obtain the grayscale value; the maximum method is to take the largest component of the three components as the grayscale value; the weighted average method is to give certain weights to the three components to calculate the final grayscale value. Since the purpose of this article is to extract road lane lines, and the lane lines are mainly yellow and white, this article adopts the weighted average method, and the calculation formula is as follows (the weight distribution is determined after the comparison experiment):

[0069] Gray=ω1R+ω2G+ω3B

[0070] S22: Perspective transformation:

[0071] Perspective transformation refers to the transformation that uses the perspective center, image point, and target point to be collinear, and according to the perspective rotation law, the image receiving surface (perspective surface) is rotated around the trace line (perspective axis) by a certain angle, destroying the original projection light beam, while still keeping the projection geometry on the image receiving surface unchanged. The principle formula is as follows:

[0072]

[0073] In the formula, (x, y) is the coordinate of the projected image, and (u, v) is the coordinate of the front view. a, b, c, d, e, f, g, h are distortion parameters. For lane lines, you can select an image of driving on a straight road, and select a trapezoidal area along the edges of the left and right lane lines. The real image of this area should be a rectangle. The four endpoints of the trapezoid can be selected as perspective transformation points, and the final projection result is like a "bird's eye view".

[0074] S23: Image Noise Reduction

[0075] When the camera acquires road images, noise points will appear on the image for some reasons, which increases the difficulty of detecting lane lines. Therefore, filtering and noise reduction are needed to eliminate the noise in the image. Common filtering algorithms include mean filtering, median filtering, bilateral filtering, etc.

[0076] S24: Image information separation (threshold processing and edge detection)

[0077] Since there are many other useless information in the image besides the road lines, it is necessary to remove the unnecessary information in the image. The operation of information separation on the image is mainly based on gradient threshold processing and edge detection;

[0078] Threshold processing: Threshold processing An image includes the target object, background and noise. To directly extract the target object from a multi-valued digital image, a common method is to set a threshold T and use T to divide the image data into two parts: a group of pixels greater than T and a group of pixels less than T. This is the most special method for studying grayscale transformation, called image binarization. The principle formula is:

[0079]

[0080] In the formula, g(i,j) is the grayscale value of the pixel on the image after the threshold operation, and f(i,j) is the grayscale value of the pixel on the image before the threshold operation. When the grayscale value of the pixel on the image is greater than the threshold, the grayscale value is set to 255, and when the grayscale value of the pixel on the image is less than the threshold, the grayscale value is set to 0.

[0081] S25: Lane feature point extraction:

[0082] Since Sobel edge extraction cannot handle dark and shadowed roads well, we consider using color space to extract white lane lines and yellow lane lines (convert the RGB channel image to the HLS channel image, then segment the L channel to extract the white lane lines in the image; convert the RGB channel image to the Lab channel image, then segment the B channel to extract the yellow lane lines in the image), and then merge the two images.

[0083] S26: Lane line fitting obtains the lane line fitting equation:

[0084] The approximate position of the lane line is found by using the image histogram. The histogram is a digital image with a grayscale value range of [0, L-1]. The histogram is a discrete function: h(r k )=n k ;

[0085] Find the column number corresponding to the maximum value in the left half of the histogram, which is the approximate location of the left lane line; find the column number corresponding to the maximum value in the right half of the histogram, which is the approximate location of the right lane line.

[0086] Use the sliding window method to search the left and right lanes:

[0087] First, according to the histogram method introduced earlier, find the approximate positions of the left and right lane lines, and use these two approximate positions as the starting points. Define a rectangular area, called a "window", and use the two starting points as the midpoints of the lower edge of the window, and store the horizontal coordinates of all white points in the square. Then take the average of the stored horizontal coordinates, and use the column where the average is located and the position of the upper edge of the first "window" as the midpoint of the lower edge of the next "window" to continue searching. Repeat this process until all rows have been searched. All white points that fall within the window are the candidate points for the left and right lane lines.

[0088] Fit the searched points with a quadratic curve:

[0089] Use the least squares method to determine a, b, c of the quadratic function Y = aX2 + bX + c;

[0090]

[0091] Among them, a, b, and c are the quadratic, linear, and constant coefficients of the lane line quadratic function respectively; xi and yi are the horizontal and vertical coordinates of the feature point in the image.

[0092] The lane line detection and fitting method may also adopt other detection algorithms that can be used for curved lanes, and the present invention does not limit this.

[0093] Step S30: The auxiliary line function pre-stored in the system (the left and right image baseline equations pre-stored in the ECU) and the lane line function calculated in S20 are combined to obtain the image coordinates of the four auxiliary points in the left and right cameras respectively.

[0094] For straight lines such as Figure 2 As shown, the auxiliary line function in step S3 is the initial auxiliary line function, and the way to obtain the function is as follows:

[0095] You need to first make two baselines perpendicular to the lane lines on the road in the world coordinate system, and ensure that the optical axis of the camera is parallel to the lane lines. Use the lane line detection principle to get the two baselines to get the auxiliary lines 1 1Z , 1 2Z with 1 1Y , 1 2Y The function expression of .

[0096] The function expression of the auxiliary line is as follows:

[0097]

[0098] where k 1Z 、b 1Z With k 1Y 、b 1Y are the slope and intercept of the first auxiliary line in the left eye camera and the right eye camera respectively; k 2Z 、b 2Z With k 2Y 、b 2Y are the slope and intercept of the second auxiliary line in the left eye camera and the right eye camera respectively.

[0099] For curved lines Figure 3 , 4 As shown, the auxiliary line function in step S3 uses the steering angle prediction module to predict the steering angle α of the road ahead.

[0100] The auxiliary line rotation angle β is obtained according to the steering angle of the road ahead:

[0101] α=β; further, the function expression of the rotated auxiliary line in the left and right camera images can be obtained:

[0102]

[0103] where k' 1Z , k' 2Z , k' 1Y , k' 1Y They are the slopes of the function expressions of the rotated auxiliary lines in the left and right camera images respectively.

[0104] Further, the calculation formula is as follows:

[0105]

[0106] Step S3 obtains the coordinates of auxiliary points in the left and right images respectively by combining the road line and the auxiliary line function. The auxiliary points are P11 left, P21 left, P12 left, P22 left and P11 right, P21 right, P12 right, P22 right.

[0107] Step S40: using the depth detection module of the binocular camera to obtain the world coordinates corresponding to the four points.

[0108] The formula of the depth detection module is as follows:

[0109]

[0110] Among them, d is the parallax; u and v are the horizontal and vertical coordinates of the object point in the right eye image respectively; the world coordinates of the object point are (XW / W, YW / W, ZW / W).

[0111] Furthermore, the matrix Q is:

[0112]

[0113] Among them, f x is the intrinsic parameter value obtained by camera calibration; b is the image length (in pixels); u0 and v0 are the horizontal and vertical coordinates of the object point in the left eye image, respectively.

[0114] Step S50: Calculate the front slope value through the slope calculation module.

[0115] The formula for the slope calculation module is as follows:

[0116] For the cross slope angle λ:

[0117] λ=arcsin[(P 22 B / P 21 P 22 +P 12 B / P 11 P 12 ) / 2]

[0118] For the longitudinal slope angle γ:

[0119] γ=arcsin[(P 11 A / P 11 P 21 +P 12 A / P 12 P 22 ) / 2]

[0120] The slope instant prediction system based on binocular camera includes image acquisition module, calibration module, lane feature point image acquisition module, lane fitting module, steering angle prediction module, depth detection module, slope calculation module and output module;

[0121] Binocular camera, used to obtain the lane line image ahead,

[0122] Processor, including

[0123] The calibration module is used to obtain the intrinsic and extrinsic parameters of the left and right binocular cameras through camera calibration; correct the left and right images so that the two images are located in the same plane and parallel to each other;

[0124] The lane line feature point image acquisition module is used to make two auxiliary lines perpendicular to the lane curve. The auxiliary lines intersect the lane line at four reference points. The image coordinates of the lane line feature points are obtained through the lane line detection method.

[0125] A lane line fitting module is used to fit the lane line feature points into quadratic function curve segments by using the least squares method; and to obtain the coordinates of four reference points in the pixel coordinate system by using the lane line fitting equation and the auxiliary line equation;

[0126] A depth detection module is used to measure the depth and obtain the coordinates of the world coordinate system of four reference points;

[0127] The slope calculation module is used to take the coordinate values ​​of the world coordinate system of the four reference points as input, calculate and output the slope value of the road ahead through the pre-stored slope solution formula.

[0128] The output module is used to output the relative transverse and longitudinal slope values ​​of the road ahead.

[0129] The steering angle prediction module is configured as the following formula:

[0130]

[0131] Where α is the steering angle; f Z (x) and f Y (x) are the fitting equations of the left and right lane lines; x0 and x1 are the horizontal coordinates of the intersection of the image center line and the left and right lane lines.

[0132] The depth detection module is configured as the following calculation formula:

[0133]

[0134] Among them, d is the parallax; u and v are the horizontal and vertical coordinates of the object point in the right eye image respectively; the world coordinates of the object point are (XW / W, YW / W, ZW / W).

[0135] Furthermore, the matrix Q is:

[0136]

[0137] Among them, f x is the intrinsic parameter value obtained by camera calibration; b is the image length (in pixels); u0 and v0 are the horizontal and vertical coordinates of the object point in the left eye image, respectively.

[0138] The slope calculation module is configured with the following calculation formula:

[0139] For the cross slope angle λ:

[0140] λ=arcsin[(P 22 B / P 21 P 22 +P 12 B / P 11 P 12 ) / 2]

[0141] For the longitudinal slope angle γ:

[0142] γ=arcsin[(P 11 A / P 11 P 21 +P 12 A / P 12 P 22 ) / 2]

[0143] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0144] The relevant modules involved in this system are all hardware system modules or functional modules that combine computer software programs or protocols with hardware in the prior art. The computer software programs or protocols involved in the functional modules are themselves technologies that are well known to those skilled in the art and are not improvements of this system. The improvements of this system are the interaction or connection relationships between the modules, that is, improvements to the overall structure of the system to solve the corresponding technical problems to be solved by this system.

[0145] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for instant slope prediction based on a binocular camera, characterized in that: Detect road lane curves through binocular cameras; Draw two auxiliary lines perpendicular to the lane curve, and the auxiliary lines intersect with the lane line at four reference points; The coordinates of the four reference points in the pixel coordinate system are calculated through the lane line fitting equation and the auxiliary line equation; The depth measurement obtains the coordinates of the world coordinate system of the four reference points; The coordinate values ​​of the world coordinate system of the four reference points are used as input, and the slope value of the road ahead is calculated and output through the pre-stored slope solution formula.

2. The method for instant slope prediction based on a binocular camera according to claim 1, characterized in that: The detecting of the road lane curve by using a binocular camera is specifically as follows: The front lane line image is obtained through the left and right binocular cameras, and the internal and external parameters of the left and right binocular cameras are obtained through camera calibration; Correct the left and right images so that they are in the same plane and parallel to each other; The image coordinates of the lane feature points are obtained through the lane detection method; and the lane fitting quadratic function obtained by the least squares method is: where a 1Y 、b 1Y 、c 1Y with a 2Y 、b 2Y 、c 2Y are the coefficients of the lane line function of the first auxiliary line and the second auxiliary line in the right eye camera; a 1z 、b 1z 、c 1z with a 2z 、b 2z 、c 2z They are the coefficients of the lane line function of the first auxiliary line and the second auxiliary line in the left eye camera respectively.

3. The method for instant slope prediction based on a binocular camera according to claim 1, characterized in that: For a straight line, the auxiliary line equation is the initial auxiliary line equation, which is obtained as follows: Make two reference lines perpendicular to the lane lines on the road in the world coordinate system to ensure that the optical axis of the binocular camera is parallel to the lane lines. Use the lane line detection method to obtain the auxiliary lines of the two reference lines in the left and right binocular cameras. 1Z , 1 2Z with 1 1Y , 1 2Y The function expression is as follows where k 1Z , b 1Z With k 1Y , b 1Y are the slope and intercept of the first auxiliary line in the left eye camera and the right eye camera respectively; k 2Z , b 2Z With k 2Y , b 2Y are the slope and intercept of the second auxiliary line in the left eye camera and the right eye camera respectively; For curved roads, the auxiliary line equation is used to predict the steering angle α of the road ahead using the steering angle prediction module; Obtain the auxiliary line rotation angle β according to the steering angle of the road ahead; α=β; get the function expression of the rotated auxiliary line in the left and right camera images: where k′ 1Z , k' 2Z , k′ 1Y , k′ 1Y are the slopes of the function expressions of the rotated auxiliary lines in the left and right camera images, respectively. The calculation formula is as follows: The lane line fitting equation and the auxiliary line equation respectively obtain the coordinates of the auxiliary points in the left and right images, and the auxiliary points are P11 left, P21 left, P12 left, P22 left and P11 right, P21 right, P12 right, P22 right.

4. The method for instant slope prediction based on binocular camera according to claim 1, characterized in that: The world coordinates of the four auxiliary points obtained by the binocular camera depth measurement are as follows: P 11 (X 11 ,Y 11 ,Z 11 )、P 21 (X 21 ,Y 21 ,Z 21 )、P 12 (X 12 ,Y 12 ,Z 12 )、P 22 (X 22 ,Y 22 ,Z 22 )。 5. A slope instant prediction system based on a binocular camera, characterized by: include Binocular camera, used to obtain the lane line image ahead, Processor, including The calibration module is used to obtain the intrinsic and extrinsic parameters of the left and right binocular cameras through camera calibration; correct the left and right images so that the two images are located in the same plane and parallel to each other; The lane line feature point image acquisition module is used to make two auxiliary lines perpendicular to the lane curve. The auxiliary lines intersect the lane line at four reference points. The image coordinates of the lane line feature points are obtained through the lane line detection method. A lane line fitting module is used to fit the lane line feature points into quadratic function curve segments by using the least squares method; and to obtain the coordinates of four reference points in the pixel coordinate system by using the lane line fitting equation and the auxiliary line equation; A depth detection module is used to measure the depth and obtain the coordinates of the world coordinate system of four reference points; The slope calculation module is used to take the coordinate values ​​of the world coordinate system of the four reference points as input, calculate and output the slope value of the road ahead through the pre-stored slope solution formula; The output module is used to output the relative transverse slope and longitudinal slope of the road ahead.

6. A computer-readable storage medium storing a computer program, wherein the computer program comprises program instructions, and when the program instructions are executed by a processor, the processor is caused to execute the method according to any one of claims 1 to 4.

Citation Information

Patent Citations

  • Monocular camera-based straightway relative gradient real-time prediction method, system and device

    CN111476106A

  • Vision-based road gradient calculation method and device

    CN112184792A