Height determination method and device based on monocular camera

By extracting the longitudinal coordinates of pixel points and the longitudinal distance under the world coordinate system from the road image, combining the small hole imaging principle and lane line detection model, the slope height information is calculated, and the positioning deviation of the monocular camera in the slope road scene is solved. The problem of this achieves low-cost and rapid slope recognition and perception capabilities.

CN115511938BActive Publication Date: 2024-11-29TIANJIN QINGZHI TECH CO LTD
View PDF 9 Cites 0 Cited by

Patent Information

Application Number
CN202211364431.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-11-02
Publication Date
2024-11-29
Estimated Expiration
2042-11-02

AI Technical Summary

Technical Problem

In the field of autonomous driving, target positioning based on monocular cameras has deviations in road scenarios with slopes, which affects subsequent functional decisions and control links, and it is difficult for the existing technology to obtain slope information at low cost and quickly and accurately.

Method used

By collecting road images, using the longitudinal coordinates of pixel points and the longitudinal distance under the world coordinate system, combining the small hole imaging principle and lane line detection model, the height information on the slope is calculated to determine the slope.

Benefits of technology

It realizes the low-cost, fast and accurate slope recognition based on a monocular camera, and improves the perception ability and positioning accuracy of the autonomous driving system.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115511938B_ABST
    Figure CN115511938B_ABST
Patent Text Reader

Abstract

The present application discloses a method and device for determining height based on a monocular camera. The method comprises: collecting a road image of the road on which the target vehicle is traveling; obtaining the height information of the first point according to the angle value corresponding to the longitudinal coordinate of the pixel point obtained from the road image and the longitudinal distance of the first point on the ramp in the world coordinate system, and the slope recognition can be completed by the camera, which can solve the technical problem of high cost of slope recognition in related technologies.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of autonomous driving, and more specifically, to a method and device for determining height based on a monocular camera. Background Art

[0002] As the field of autonomous driving continues to develop, the requirements for target perception capabilities are also increasing. Especially when driving on a sloped road, if the slope information of the road ahead can be accurately calculated in real time, it will help the driver to understand the surrounding environment in a timely manner and drive more safely and stably.

[0003] At present, although LiDAR and 4D millimeter-wave radar can measure point cloud data with slope information, it is difficult to popularize them due to the large amount of point cloud data and high computing and equipment costs. In the perception based on monocular camera, due to the lack of depth information due to imaging, it is difficult to directly obtain slope information. In addition, in the existing target positioning methods, the road surface is assumed to be flat, which leads to a certain deviation in the target positioning based on the monocular camera in the road scene with slope, which in turn affects the subsequent functional decision-making, regulation and control and other links.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present application provide a method and device for determining height based on a monocular camera, so as to at least solve the technical problem of high cost of slope recognition in the related art.

[0006] According to one aspect of an embodiment of the present application, a height determination method based on a monocular camera is provided, including: acquiring a road image of the road on which a target vehicle is traveling; and obtaining height information of a first point on the ramp based on an angle value corresponding to the longitudinal coordinate of a pixel point obtained from the road image and a longitudinal distance of the first point on the ramp in a world coordinate system.

[0007] Optionally, before obtaining the height information of the first point based on the angle value corresponding to the longitudinal coordinate of the pixel point obtained from the road image and the longitudinal distance of the first point on the ramp in the world coordinate system, the method also includes: obtaining the angle value corresponding to the longitudinal coordinate of the pixel point from the road image; and obtaining the longitudinal distance of the first point on the ramp in the world coordinate system based on the road image.

[0008] Optionally, obtaining the angle value corresponding to the longitudinal coordinate of the pixel point from the road image includes: determining the angle range of the camera of the target vehicle in the pitch direction according to the resolution of the road image; Wherein, imgH is the pixel height of the road image, f is the focal length of the camera, and atan() is the inverse tangent function; according to the angle range VFOV, the angle value radPerPixel=VFOV / imgH corresponding to the longitudinal coordinate of the pixel in the road image is obtained.

[0009] Optionally, obtaining the longitudinal distance of the first point on the ramp in the world coordinate system according to the road image includes: performing lane line detection on the road image using a lane line detection model to obtain lane line point cloud data; performing linear fitting on the lane line point cloud data of the target lane to obtain a lane line equation in the pixel coordinate system; calculating the distance between the point on the lane line and the lane line according to the lane line equation, and determining the point where the distance value change is greater than the distance threshold as the starting point of the ramp; taking the starting point of the ramp as the starting point, fitting the left lane line and the right lane line of the ramp respectively to obtain the fitting equation of the left lane line and the fitting equation of the right lane line; and determining the coordinates (u ) of the first point using the fitting equation of the left lane line and the fitting equation of the right lane line. i ,v i ) and the coordinates of the second point (u j ,v j ), wherein the first point is the intersection of the horizontal line on the ramp and the left lane line, and the second point is the intersection of the horizontal line on the ramp and the right lane line; the lane pixel width value at the horizontal line is obtained by using the coordinates of the first point and the coordinates of the second point; and the longitudinal distance at the horizontal line in the world coordinate system is obtained according to the lane pixel width value by using the pinhole imaging principle.

[0010] Optionally, after obtaining the longitudinal distance at the horizontal line in the world coordinate system according to the lane pixel width by using the pinhole imaging principle, the method further includes: mapping the calculated longitudinal distance to the pixel coordinate system through coordinate transformation to obtain the pixel coordinates corresponding to the first point; and mapping the mapped pixel coordinate value v x And the pixel coordinate value v obtained by detection i Subtract and get the coordinate difference v d .

[0011] Optionally, obtaining the height information of the first point according to the angle value corresponding to the longitudinal coordinate of the pixel point obtained from the road image and the longitudinal distance of the first point on the ramp in the world coordinate system includes: obtaining the height information of the first point according to the angle value radPerPixel corresponding to the longitudinal coordinate of the pixel point obtained from the road image and the longitudinal distance L of the first point on the ramp in the world coordinate system. x And the coordinate difference v d , get the height information H of the first point:

[0012] H=Lx *tan(θ), θ=v d *radPerPixel.

[0013] Optionally, according to the angle value radPerPixel corresponding to the longitudinal coordinate of the pixel point obtained from the road image, the longitudinal distance L of the first point on the ramp in the world coordinate system x And the coordinate difference v d After obtaining the height information H of the first point, the method further includes: calculating the lateral position information of the point on the ramp in the world coordinate system according to the longitudinal distance of the first point on the ramp in the world coordinate system and the distance difference between the point on the lane line and the center axis of the picture.

[0014] According to another aspect of an embodiment of the present application, a height determination device based on a monocular camera is also provided, including: an acquisition unit, used to acquire a road image of the road on which a target vehicle is traveling; a determination unit, used to obtain the height information of the first point on the ramp based on the angle value corresponding to the longitudinal coordinate of the pixel point obtained from the road image and the longitudinal distance of the first point in the world coordinate system.

[0015] Optionally, the determination unit is also used to: obtain the angle value corresponding to the longitudinal coordinate of the pixel point from the road image before obtaining the height information of the first point based on the angle value corresponding to the longitudinal coordinate of the pixel point obtained from the road image and the longitudinal distance of the first point on the ramp in the world coordinate system; and obtain the longitudinal distance of the first point on the ramp in the world coordinate system based on the road image.

[0016] Optionally, the determination unit is further used to: determine the angle range of the camera of the target vehicle in the pitch direction according to the resolution of the road image. Wherein, imgH is the pixel height of the road image, f is the focal length of the camera, and atan() is the inverse tangent function; according to the angle range VFOV, the angle value radPerPixel=VFOV / imgH corresponding to the longitudinal coordinate of the pixel in the road image is obtained.

[0017] Optionally, the determination unit is further used to: use a lane line detection model to perform lane line detection on the road image to obtain lane line point cloud data; perform linear fitting on the lane line point cloud data of the target lane to obtain a lane line equation in a pixel coordinate system; calculate the distance between a point on the lane line and the lane line according to the lane line equation, and determine the point where the distance value change is greater than a distance threshold as the starting point of the ramp; with the starting point of the ramp as the starting point, respectively fit the left lane line and the right lane line of the ramp to obtain a fitting equation for the left lane line and a fitting equation for the right lane line; and determine the coordinates of the first point (u ) using the fitting equation for the left lane line and the fitting equation for the right lane line.i ,v i ) and the coordinates of the second point (u j ,v j ), wherein the first point is the intersection of the horizontal line on the ramp and the left lane line, and the second point is the intersection of the horizontal line on the ramp and the right lane line; the lane pixel width value at the horizontal line is obtained by using the coordinates of the first point and the coordinates of the second point; and the longitudinal distance at the horizontal line in the world coordinate system is obtained according to the lane pixel width value by using the pinhole imaging principle.

[0018] Optionally, the determination unit is further configured to: after obtaining the longitudinal distance at the horizontal line in the world coordinate system according to the lane pixel width value by using the pinhole imaging principle, map the calculated longitudinal distance to the pixel coordinate system through coordinate transformation to obtain the pixel coordinates corresponding to the first point; and map the mapped pixel coordinate value v x And the pixel coordinate value v obtained by detection i Subtract and get the coordinate difference v d .

[0019] Optionally, the determining unit is further configured to: determine the angle radPerPixel corresponding to the longitudinal coordinate of the pixel point obtained from the road image, the longitudinal distance L of the first point on the ramp in the world coordinate system, and the longitudinal distance L of the first point on the ramp in the world coordinate system. x And the coordinate difference v d , get the height information H of the first point:

[0020] H=L x *tan(θ), θ=v d *radPerPixel.

[0021] Optionally, the determining unit is further configured to: determine the angle value radPerPixel corresponding to the longitudinal coordinate of the pixel point obtained from the road image, the longitudinal distance L of the first point on the ramp in the world coordinate system, and the angle value radPerPixel corresponding to the longitudinal coordinate of the pixel point obtained from the road image. x And the coordinate difference v d After obtaining the height information H of the first point, the lateral position information of the point on the ramp in the world coordinate system is calculated according to the longitudinal distance of the first point on the ramp in the world coordinate system and the distance difference between the point on the lane line and the center axis of the picture.

[0022] According to another aspect of an embodiment of the present application, an electronic device is further provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the above method through the computer program.

[0023] According to one aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps of any one of the embodiments of the above method.

[0024] By applying the technical solution of the present invention, a road image of the road on which the target vehicle is traveling is collected; the height information of the first point on the ramp is obtained based on the angle value corresponding to the longitudinal coordinate of the pixel point obtained from the road image and the longitudinal distance of the first point on the ramp in the world coordinate system, and the slope recognition can be completed through the camera, which can solve the technical problem of high cost of slope recognition in related technologies.

[0025] In addition to the above-described purposes, features and advantages, the present invention has other purposes, features and advantages. The present invention will be further described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The accompanying drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the accompanying drawings:

[0027] Figure 1 is a flowchart of an optional method for determining height based on a monocular camera according to an embodiment of the present application;

[0028] Figure 2 is a schematic diagram of an optional pixel width value according to an embodiment of the present application;

[0029] Figure 3 It is a schematic diagram of an optional height determination device based on a monocular camera according to an embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0031] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0032] The purpose of this application is to provide a straight road slope estimation solution based on a monocular camera, which can calculate the slope information of the straight road based on the lane line information detected by the monocular camera, thereby solving the problem that existing sensors cannot predict the slope of the straight road ahead of the vehicle quickly and accurately at low cost.

[0033] According to one aspect of the embodiments of the present application, an embodiment of a method for determining a height based on a monocular camera is provided. The method for determining a height based on a monocular camera of the embodiment of the present application can be executed by a server or a terminal. The terminal executing the method for determining a height based on a monocular camera of the embodiment of the present application can also be executed by a client installed thereon. Figure 1 is a flowchart of an optional method for determining height based on a monocular camera according to an embodiment of the present application, such as Figure 1 As shown, the method may include the following steps:

[0034] Step S102, collecting a road image of the road on which the target vehicle is traveling.

[0035] Step S104, obtaining height information of the first point on the ramp according to the angle value corresponding to the longitudinal coordinate of the pixel point obtained from the road image and the longitudinal distance of the first point in the world coordinate system.

[0036] Optionally, before obtaining the height information of the first point based on the angle value corresponding to the longitudinal coordinate of the pixel point obtained from the road image and the longitudinal distance of the first point on the ramp in the world coordinate system, the method also includes: obtaining the angle value corresponding to the longitudinal coordinate of the pixel point from the road image; and obtaining the longitudinal distance of the first point on the ramp in the world coordinate system based on the road image.

[0037] Optionally, obtaining the angle value corresponding to the longitudinal coordinate of the pixel point from the road image includes: determining the angle range of the camera of the target vehicle in the pitch direction according to the resolution of the road image; Wherein, imgH is the pixel height of the road image, f is the focal length of the camera, and atan() is the inverse tangent function; according to the angle range VFOV, the angle value radPerPixel=VFOV / imgH corresponding to the longitudinal coordinate of the pixel in the road image is obtained.

[0038] Optionally, obtaining the longitudinal distance of the first point on the ramp in the world coordinate system according to the road image includes: performing lane line detection on the road image using a lane line detection model to obtain lane line point cloud data; performing linear fitting on the lane line point cloud data of the target lane to obtain a lane line equation in the pixel coordinate system; calculating the distance between the point on the lane line and the lane line according to the lane line equation, and determining the point where the distance value change is greater than the distance threshold as the starting point of the ramp; taking the starting point of the ramp as the starting point, fitting the left lane line and the right lane line of the ramp respectively to obtain the fitting equation of the left lane line and the fitting equation of the right lane line; and determining the coordinates (u ) of the first point using the fitting equation of the left lane line and the fitting equation of the right lane line. i ,v i ) and the coordinates of the second point (u j ,v j ), wherein the first point is the intersection of the horizontal line on the ramp and the left lane line, and the second point is the intersection of the horizontal line on the ramp and the right lane line; the lane pixel width value at the horizontal line is obtained by using the coordinates of the first point and the coordinates of the second point; and the longitudinal distance at the horizontal line in the world coordinate system is obtained according to the lane pixel width value by using the pinhole imaging principle.

[0039] Optionally, after obtaining the longitudinal distance at the horizontal line in the world coordinate system according to the lane pixel width by using the pinhole imaging principle, the method further includes: mapping the calculated longitudinal distance to the pixel coordinate system through coordinate transformation to obtain the pixel coordinates corresponding to the first point; and mapping the mapped pixel coordinate value v x And the pixel coordinate value v obtained by detection i Subtract and get the coordinate difference v d .

[0040] Optionally, obtaining the height information of the first point according to the angle value corresponding to the longitudinal coordinate of the pixel point obtained from the road image and the longitudinal distance of the first point on the ramp in the world coordinate system includes: obtaining the height information of the first point according to the angle value radPerPixel corresponding to the longitudinal coordinate of the pixel point obtained from the road image and the longitudinal distance L of the first point on the ramp in the world coordinate system. x And the coordinate difference v d , get the height information H of the first point:

[0041] H=Lx *tan(θ), θ=v d *radPerPixel.

[0042] Optionally, according to the angle value radPerPixel corresponding to the longitudinal coordinate of the pixel point obtained from the road image, the longitudinal distance L of the first point on the ramp in the world coordinate system x And the coordinate difference v d After obtaining the height information H of the first point, the method further includes: calculating the lateral position information of the point on the ramp in the world coordinate system according to the longitudinal distance of the first point on the ramp in the world coordinate system and the distance difference between the point on the lane line and the center axis of the picture.

[0043] Through the above steps, a road image of the road on which the target vehicle is traveling is collected; based on the angle value corresponding to the longitudinal coordinate of the pixel point obtained from the road image and the longitudinal distance of the first point on the ramp in the world coordinate system, the height information of the first point is obtained, and slope recognition can be completed through a camera, which can solve the technical problem of high cost of slope recognition in related technologies.

[0044] As an optional embodiment, the technical solution of the present application is combined with Figure 2 And detailed steps are described in detail:

[0045] Step S1, calibrate the internal and external parameters of the monocular camera and collect lane line image information of the road surface.

[0046] Step S2, calculate the angle range of the camera in the pitch direction according to the resolution of the image, the formula is as follows:

[0047]

[0048] Among them, VFOV is the angular range of the pitch direction, imgH is the height of the image, and f is the focal length.

[0049] Step S3, according to the calculated angle range VFOV, the angle value corresponding to each vertical pixel coordinate is obtained, and the formula is as follows:

[0050] radPerPixel=VFOV / imgH.

[0051] Step S4: perform lane line detection on the road image using a lane line detection model based on the collected image information to obtain lane line point cloud data.

[0052] Step S5, take a lane line S1 of the lane, perform linear fitting on the lane line point cloud at the bottom of the image, and obtain the lane line equation in the pixel coordinate system.

[0053] Step S6, calculating the distance from the point on the lane line to the lane line according to the fitted lane line equation, and judging the point whose distance change is greater than the distance threshold as the starting point of the ramp through the change of the distance value.

[0054] Step S7, taking the starting point of the ramp as the starting point, respectively fitting the left and right lane lines to obtain a fitting equation.

[0055] Step S8, starting from the starting point, find the first point (u i ,v i ) and the intersection of the other lane line S2 (u j ,v j ) (i.e. the second point), and calculate the pixel width between the two points, which is the pixel width of the lane line at that point, L p ,like Figure 2 shown.

[0056] Step S9: Based on the pixel width obtained in step S8 and the pinhole imaging principle, the longitudinal distance information L in the world coordinate system can be obtained. x , the formula is as follows:

[0057]

[0058] Among them, L w is the actual width of the lane line.

[0059] Step S10, the calculated longitudinal distance is mapped to the pixel coordinate system through coordinate conversion to obtain the corresponding pixel coordinates (u x ,v x ), the formula is as follows:

[0060]

[0061] Among them, R is the rotation matrix, T is the translation matrix, X w , Y w , Z w is the position of the target in the world coordinate system, λ represents the scale factor, is the internal parameter matrix, f x It is the length of the focal length in the x-axis direction described in pixels, f y Pixels are used to describe the length of the focal length in the y-axis direction. (u0, v0) is the actual position of the principal point, and the unit is also pixel.

[0062] Step S11: map the pixel v x And the detected pixel v i Subtract and get the difference v d .

[0063] Step S12, based on the radPerPixel obtained in step S3, the L obtained in step S9 x , v obtained in step S11 d , the height information of the corresponding position can be obtained, the formula is as follows:

[0064] θ=v d radPerPixel,

[0065] H=L x tan(θ).

[0066] Step S13: In addition, according to the L obtained in step S9 x , the difference between the lane line point and the center axis of the screen, calculate the lateral position information x of the point on the ramp in the world coordinate system w , the formula is as follows:

[0067]

[0068] x p =u-0.5·imgW,

[0069] Among them, imgW is the image width.

[0070] By adopting the technical solution of the present application, only one monocular camera and less computing power are needed to obtain the slope information on a straight road in real time; in the actual driving process, the subsequent target detection and positioning can be corrected, thereby improving the perception ability of the monocular camera.

[0071] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0072] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0073] According to another aspect of an embodiment of the present application, a monocular camera-based height determination device for implementing the above-mentioned monocular camera-based height determination method is also provided. Figure 3 is a schematic diagram of an optional height determination device based on a monocular camera according to an embodiment of the present application, such as Figure 3 As shown, the device may include:

[0074] The acquisition unit 31 is used to acquire a road image of the road on which the target vehicle is traveling; the determination unit 33 is used to obtain the height information of the first point on the ramp based on the angle value corresponding to the longitudinal coordinate of the pixel point obtained from the road image and the longitudinal distance of the first point on the ramp in the world coordinate system.

[0075] Optionally, the determination unit is also used to: obtain the angle value corresponding to the longitudinal coordinate of the pixel point from the road image before obtaining the height information of the first point based on the angle value corresponding to the longitudinal coordinate of the pixel point obtained from the road image and the longitudinal distance of the first point on the ramp in the world coordinate system; and obtain the longitudinal distance of the first point on the ramp in the world coordinate system based on the road image.

[0076] Optionally, the determination unit is further used to: determine the angle range of the camera of the target vehicle in the pitch direction according to the resolution of the road image. Wherein, imgH is the pixel height of the road image, f is the focal length of the camera, and atan() is the inverse tangent function; according to the angle range VFOV, the angle value radPerPixel=VFOV / imgH corresponding to the longitudinal coordinate of the pixel in the road image is obtained.

[0077] Optionally, the determination unit is further used to: use a lane line detection model to perform lane line detection on the road image to obtain lane line point cloud data; perform linear fitting on the lane line point cloud data of the target lane to obtain a lane line equation in a pixel coordinate system; calculate the distance between a point on the lane line and the lane line according to the lane line equation, and determine the point where the distance value change is greater than a distance threshold as the starting point of the ramp; with the starting point of the ramp as the starting point, respectively fit the left lane line and the right lane line of the ramp to obtain a fitting equation for the left lane line and a fitting equation for the right lane line; and determine the coordinates of the first point (u ) using the fitting equation for the left lane line and the fitting equation for the right lane line. i ,v i ) and the coordinates of the second point (u j ,v j), wherein the first point is the intersection of the horizontal line on the ramp and the left lane line, and the second point is the intersection of the horizontal line on the ramp and the right lane line; the lane pixel width value at the horizontal line is obtained by using the coordinates of the first point and the coordinates of the second point; and the longitudinal distance at the horizontal line in the world coordinate system is obtained according to the lane pixel width value by using the pinhole imaging principle.

[0078] Optionally, the determination unit is further configured to: after obtaining the longitudinal distance at the horizontal line in the world coordinate system according to the lane pixel width value by using the pinhole imaging principle, map the calculated longitudinal distance to the pixel coordinate system through coordinate transformation to obtain the pixel coordinates corresponding to the first point; and map the mapped pixel coordinate value v x And the pixel coordinate value v obtained by detection i Subtract and get the coordinate difference v d .

[0079] Optionally, the determining unit is further configured to: determine the angle radPerPixel corresponding to the longitudinal coordinate of the pixel point obtained from the road image, the longitudinal distance L of the first point on the ramp in the world coordinate system, and the longitudinal distance L of the first point on the ramp in the world coordinate system. x And the coordinate difference v d , get the height information H of the first point:

[0080] H=L x *tan(θ), θ=v d *radPerPixel.

[0081] Optionally, the determining unit is further configured to: determine the angle value radPerPixel corresponding to the longitudinal coordinate of the pixel point obtained from the road image, the longitudinal distance L of the first point on the ramp in the world coordinate system, and the angle value radPerPixel corresponding to the longitudinal coordinate of the pixel point obtained from the road image. x And the coordinate difference v d After obtaining the height information H of the first point, the lateral position information of the point on the ramp in the world coordinate system is calculated according to the longitudinal distance of the first point on the ramp in the world coordinate system and the distance difference between the point on the lane line and the center axis of the picture.

[0082] According to another aspect of an embodiment of the present application, an electronic device is further provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the above method through the computer program.

[0083] According to one aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps of any one of the embodiments of the above method.

[0084] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments, and this embodiment will not be described in detail here.

[0085] Optionally, in this embodiment, the above-mentioned storage medium may include but is not limited to: a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk, and other media that can store program codes.

[0086] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0087] If the integrated units in the above embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in the above computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling one or more computer devices (which may be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application.

[0088] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0089] In the several embodiments provided in the present application, it should be understood that the disclosed client can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0090] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0091] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0092] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for determining height based on a monocular camera, characterized in that: include: Collecting a road image of the road on which the target vehicle is traveling; Obtaining the angle value corresponding to the longitudinal coordinate of the pixel point from the road image includes: Determine the pitch angle range of the target vehicle's camera based on the resolution of the road image Wherein, imgH is the pixel height of the road image, f is the focal length of the camera, and atan() is the inverse tangent function; According to the angle range VFOV, the angle value radPerPixel corresponding to the longitudinal coordinate of the pixel in the road image is obtained: VFOV / imgH; Obtaining a longitudinal distance of a first point on the ramp in a world coordinate system according to the road image; Obtaining height information of the first point on the ramp according to the angle value corresponding to the longitudinal coordinate of the pixel point obtained from the road image and the longitudinal distance of the first point on the ramp in the world coordinate system, including: According to the angle value radPerPixel corresponding to the longitudinal coordinate of the pixel point obtained from the road image, the longitudinal distance L of the first point on the ramp in the world coordinate system x And the coordinate difference v d , get the height information H of the first point: H=L x *tan(θ), θ=v d *radPerPixel; Obtaining the longitudinal distance of the first point on the ramp in the world coordinate system according to the road image includes: Using a lane line detection model to perform lane line detection on the road image to obtain lane line point cloud data; Perform linear fitting on the lane line point cloud data of the target lane to obtain the lane line equation in the pixel coordinate system; According to the lane line equation, the distance between the point on the lane line and the lane line equation is calculated, and the point where the distance value change is greater than the distance threshold is determined as the starting point of the ramp; Taking the starting point of the ramp as the starting point, fitting the left lane line and the right lane line of the ramp respectively to obtain a fitting equation for the left lane line and a fitting equation for the right lane line; Determine the first point (u) by using the fitting equation of the left lane line and the fitting equation of the right lane line i ,v i ) and the coordinates of the second point (u j ,v j ), wherein the first point (u i ,v i ) is the intersection of the horizontal line on the ramp and the left lane line, and the second point (u j ,v j ) is the intersection of the horizontal line on the ramp and the right lane line; Using the first point (u i ,v i ) and the coordinates of the second point (u j ,v j ) coordinates to obtain the lane pixel width value at the horizontal line; Using the pinhole imaging principle, the longitudinal distance at the horizontal line in the world coordinate system is obtained according to the lane pixel width value; After obtaining the longitudinal distance at the horizontal line in the world coordinate system according to the lane pixel width by using the pinhole imaging principle, the method further includes: mapping the calculated longitudinal distance to the pixel coordinate system through coordinate transformation to obtain the first point (u i ,v i ) corresponding to the pixel coordinates; the mapped pixel coordinate value v x And the pixel coordinate value v obtained by detection i Subtract and get the coordinate difference v d .

2. The method according to claim 1, characterized in that According to the angle value radPerPixel corresponding to the longitudinal coordinate of the pixel point obtained from the road image, the longitudinal distance L of the first point on the ramp in the world coordinate system x And the coordinate difference v d After obtaining the height information H of the first point, the method further includes: The lateral position information of the point on the ramp in the world coordinate system is calculated based on the longitudinal distance of the first point on the ramp in the world coordinate system and the distance difference between the point on the lane line and the center axis of the picture. The lateral position information x w The calculation formula is as follows: x p =u-0.5·imgW, Among them, imgW is the image width.

3. A height determination device based on a monocular camera, characterized in that: include: The acquisition unit is used to acquire a road image of the road on which the target vehicle is traveling, and obtain the angle value corresponding to the longitudinal coordinate of the pixel point from the road image, including: determining the angle range of the camera of the target vehicle in the pitch direction according to the resolution of the road image; Wherein, imgH is the pixel height of the road image, f is the focal length of the camera, and atan() is the inverse tangent function; according to the angle range VFOV, the angle value radPerPixel corresponding to the longitudinal coordinate of the pixel in the road image is obtained: VFOV / imgH; according to the road image, the longitudinal distance of the first point on the ramp in the world coordinate system is obtained; A determination unit, configured to obtain height information of the first point according to an angle value corresponding to the longitudinal coordinate of the pixel point obtained from the road image and a longitudinal distance of the first point on the ramp in a world coordinate system, comprising: determining the height information of the first point according to the angle value radPerPixel corresponding to the longitudinal coordinate of the pixel point obtained from the road image and the longitudinal distance L of the first point on the ramp in a world coordinate system; x And the coordinate difference v d , get the height information H of the first point: H=L x *tan(θ), θ=v d *radPerPixel; Obtaining the longitudinal distance of the first point on the ramp in the world coordinate system according to the road image includes: The lane line detection model is used to perform lane line detection on the road image to obtain lane line point cloud data; linear fitting is performed on the lane line point cloud data of the target lane to obtain a lane line equation in a pixel coordinate system; according to the lane line equation, the distance between the point on the lane line and the lane line equation is calculated, and the point whose distance value change is greater than the distance threshold is determined as the starting point of the ramp; with the starting point of the ramp as the starting point, the left lane line and the right lane line of the ramp are fitted respectively to obtain the fitting equation of the left lane line and the fitting equation of the right lane line; the first point (u i ,v i ) and the coordinates of the second point (u j ,v j ), wherein the first point (u i ,v i ) is the intersection of the horizontal line on the ramp and the left lane line, and the second point (u j ,v j ) is the intersection of the horizontal line on the ramp and the right lane line; using the first point (u i ,v i ) and the coordinates of the second point (u j ,v j ) coordinates to obtain the lane pixel width value at the horizontal line; using the pinhole imaging principle, according to the lane pixel width value, obtain the longitudinal distance at the horizontal line in the world coordinate system; after using the pinhole imaging principle, according to the lane pixel width value, obtain the longitudinal distance at the horizontal line in the world coordinate system, further comprising: mapping the calculated longitudinal distance to the pixel coordinate system through coordinate conversion, obtaining the pixel coordinates corresponding to the first point; and mapping the mapped pixel coordinate value v x And the pixel coordinate value v obtained by detection i Subtract and get the coordinate difference v d .

4. A storage medium, characterized in that: The storage medium includes a stored program, wherein the program executes the method described in any one of claims 1 to 2 when executed.

5. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: The processor executes the method described in any one of claims 1 to 2 through the computer program.

Citation Information

Patent Citations

  • Path detection method, related device and computer readable storage medium

    CN109074490A

  • Road slope calculation method and device based on monocular vision

    CN109900254A

  • Road height detection method and device, readable storage medium and electronic equipment

    CN112304293A

  • Road slope prediction method and device and storage medium

    CN112862890A

  • Unmanned vehicle position detection method and device, equipment, storage medium and vehicle

    CN113486796A