Image depth-of-field estimation method, apparatus, and storage medium

Abstract

The application relates to an image depth-of-field estimation method and device and a storage medium, the method comprising the following steps: determining a far-end intersection based on at least two lane lines in a target image, the far-end intersection being a visual intersection of the lane lines in the target image; the target image being obtained by a camera arranged above a lane; obtaining an intersection depression angle corresponding to the far-end intersection; the intersection depression angle being an angle between a line connecting the camera and the far-end intersection in a real scene and a plane where the lane is located; determining a depth-of-field estimation value of the far-end intersection based on the intersection depression angle and a maximum depression angle of the target image, the maximum depression angle being an angle between a line connecting the camera and a near end of the target image in the real scene and the plane where the lane is located; the depth-of-field estimation value of the image can be obtained only by simple lane line identification and geometric calculation, without consuming too many hardware resources, and the problem that a depth-of-field estimation method needs to occupy many hardware resources and affects the real-time performance of a monitoring system is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to an image depth-of-field estimation method, device and storage medium. BACKGROUND

[0002] In the field of intelligent traffic monitoring, it is necessary to detect and identify road conditions and possible traffic events through video images. The traffic events that need to be monitored can include road congestion, vehicle reverse driving, overspeed or underspeed, and U-turn driving. Events that need to associate distance information and speed information in the traffic monitoring scene all need to estimate the distance. In order to realize real-time monitoring and identification of traffic events, it is very important to provide a fast and efficient depth-of-field estimation method. However, since the image acquisition and processing device such as a monitoring camera itself does not have the function of obtaining the depth of field of the image, it is impossible to realize the depth-of-field estimation of the video image, which is a problem to be solved.

[0003] At present, there is no effective solution to the problem that the image acquisition and processing device cannot realize the depth-of-field estimation of the video image in the related art. SUMMARY

[0004] In the present embodiment, an image depth-of-field estimation method, device and storage medium are provided to solve the problem that the depth-of-field estimation method in the related art needs to occupy more hardware resources, affecting the real-time performance of the monitoring system.

[0005] In a first aspect, an image depth-of-field estimation method is provided in the present embodiment, and the method comprises:

[0006] Based on at least two lane lines in a target image, a far-end intersection is determined, the far-end intersection being a visual intersection of the lane lines in the target image; the target image being obtained by a camera arranged above the lane;

[0007] An intersection depression angle corresponding to the far-end intersection is obtained; the intersection depression angle being an angle between a line connecting the camera to the far-end intersection in a real scene and a plane on which the lane is located;

[0008] Based on the intersection depression angle and a maximum depression angle of the target image, a depth-of-field estimation value of the far-end intersection is determined, the maximum depression angle being an angle between a line connecting the camera to a near-end of the target image in the real scene and the plane on which the lane is located.

[0009] In some embodiments, the obtaining of the intersection depression angle corresponding to the far-end intersection comprises:

[0010] An amplification coefficient of a near-end of the target image relative to the far-end intersection is obtained;

[0011] Based on the magnification factor, the depression angle of the intersection point corresponding to the far end intersection point is obtained.

[0012] In some embodiments, obtaining the magnification factor of the near-end intersection point of the target image relative to the far-end intersection point includes:

[0013] Obtain the ratio of the near-end pixel width of the lane in the target image to the actual width of the lane;

[0014] The magnification factor is obtained based on the ratio and the actual distance between the distal intersection point and the proximal end. In some embodiments, obtaining the depression angle of the intersection point corresponding to the distal intersection point based on the magnification factor includes:

[0015]

[0016] Where θ0 is the angle of depression at the intersection point, scale dis The magnification factor is denoted as .

[0017] In some embodiments, determining the depth estimate of the far intersection point based on the depression angle of the intersection point and the maximum depression angle of the target image includes:

[0018] Using the depression angle at the intersection point as the lower limit of integration and the maximum depression angle as the upper limit of integration, the depression angle function of the camera is integrated to obtain the depth estimate of the far intersection point. The depression angle function is the reciprocal of the square of the depression angle tangent function of the camera.

[0019] In some embodiments, after determining the depth estimate of the far intersection point based on the depression angle of the intersection point and the maximum depression angle of the target image, the method further includes:

[0020] The depth-of-field estimate is adjusted based on the camera's set height and internal parameters.

[0021] In some embodiments, adjusting the depth-of-field estimate based on the camera's set height and internal parameters includes:

[0022] An adjustment factor is obtained based on the ratio of the camera's set height to its internal parameters;

[0023] The adjusted depth estimate is obtained by multiplying the adjustment factor by the depth estimate.

[0024] In some embodiments, determining the far intersection point based on at least two lane lines in the target image includes:

[0025] Identify at least two lane lines in the target image, wherein the at least two lane lines intersect the bottom boundary of the target image and are located on opposite sides of the target image.

[0026] Secondly, this embodiment provides an image depth estimation device, the image depth estimation device comprising:

[0027] The first determining module is used to determine the far intersection point based on at least two lane lines in the target image, wherein the far intersection point is the visual intersection point of the lane lines in the target image; the target image is obtained by a camera set above the lane;

[0028] The acquisition module is used to acquire the depression angle of the intersection point corresponding to the far intersection point; the depression angle of the intersection point is the angle between the line connecting the camera to the far intersection point in the real scene and the plane where the lane is located.

[0029] The second determining module is used to determine the depth estimate of the far intersection point based on the depression angle of the intersection point and the maximum depression angle of the target image, wherein the maximum depression angle is the angle between the line connecting the camera to the near end of the target image in the real scene and the plane where the lane is located.

[0030] Thirdly, this embodiment provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the image depth estimation method described in the first aspect above.

[0031] Compared with related technologies, the image depth estimation method provided in this embodiment determines the far-end intersection point based on at least two lane lines in the target image, thereby determining the far-end reference point in the target image used for depth estimation; it determines the relative position of the camera capturing the target image and the far-end intersection point by obtaining the intersection point depression angle corresponding to the far-end intersection point; and it determines the depth estimation value of the far-end intersection point based on the intersection point depression angle and the maximum depression angle of the target image, that is, it determines the relative position of the camera and the near end of the image by the maximum depression angle, and obtains the true distance between the two based on the scaling of the target image between the intersection point depression angle and the maximum depression angle, thereby determining the depth estimation value of the far-end intersection point. In other words, it obtains the image depth estimation value by performing lane line recognition and geometric calculation on the target image, thus solving the problem that image acquisition and processing devices cannot perform depth estimation on video images.

[0032] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0033] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0034] Figure 1 This is a hardware device block diagram of the image depth estimation method according to an embodiment of this application;

[0035] Figure 2 This is a flowchart of the image depth estimation method according to an embodiment of this application;

[0036] Figure 3 This is a schematic diagram of a target image captured by a camera according to an embodiment of this application;

[0037] Figure 4 This is a schematic diagram of the intersection angle and maximum angle of depression in an embodiment of this application;

[0038] Figure 5 This is a flowchart illustrating the process of obtaining the depression angle of the intersection point corresponding to the far intersection point according to an embodiment of this application.

[0039] Figure 6 This is a schematic diagram illustrating the principle of calculating image depth according to an embodiment of this application;

[0040] Figure 7 This is a flowchart of an image depth estimation method according to a preferred embodiment of this application;

[0041] Figure 8 This is a structural block diagram of an image depth estimation device according to an embodiment of this application. Detailed Implementation

[0042] To better understand the purpose, technical solution, and advantages of this application, the application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0043] Unless otherwise defined, the technical or scientific terms used in this application shall have the general meaning as understood by one of ordinary skill in the art to which this application pertains. Words such as “a,” “an,” “an,” “the,” “the,” and “these,” used in this application, do not indicate quantitative limitation and may be singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include steps or modules (units) not listed, or may include other steps or modules (units) inherent to such processes, methods, products, or devices. The terms “connected,” “linked,” and “coupled,” used in this application, are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. The term “multiple” used in this application refers to two or more. The "and / or" operator describes the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: A alone, A and B simultaneously, and B alone. Typically, the character " / " indicates that the objects before and after it are in an "or" relationship. The terms "first," "second," and "third," etc., used in this application are merely for distinguishing similar objects and do not represent a specific ordering of the objects.

[0044] The method embodiments provided in this example can be executed in traffic monitoring equipment, terminals, computers, servers, or similar computing devices. Traffic monitoring equipment can be traffic image acquisition devices such as surveillance cameras. Figure 1 This is a hardware device block diagram of the computer used for the image depth estimation method in this application embodiment. For example... Figure 1 As shown, a computer may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 and a memory 104 for storing data are also included. The processor 102 may be, but is not limited to, a microprocessor (MCU) or a programmable logic device (FPGA). The computer may also include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that… Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the computer described above. For example, the computer may also include components that are larger than... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown are illustrated.

[0045] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the image depth estimation method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, thereby implementing the aforementioned method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0046] Transmission device 106 is used to receive or send data via a network. A computer or traffic monitoring device performing an image depth estimation method can receive images sent by other devices through this network and send the results of the method's execution to other devices or a traffic monitoring platform via the network. The aforementioned network includes a wireless network provided by the computer's communication provider. In one example, transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, transmission device 106 can be a Radio Frequency (RF) module for wireless communication with the Internet.

[0047] This embodiment provides an image depth estimation method. Figure 2 This is a flowchart of the image depth estimation method in this embodiment, as follows: Figure 2 As shown, the process includes the following steps:

[0048] Step S201: Based on at least two lane lines in the target image, determine the far intersection point, which is the visual intersection point of the lane lines in the target image; the target image is obtained by a camera set above the lane.

[0049] In intelligent transportation applications, cameras capture images of vehicles on the road to assess and identify road conditions and traffic incidents. Cameras are typically mounted on brackets or crossbars above the lanes. Figure 3 This is a schematic diagram of the target image captured by the camera in this embodiment, such as... Figure 3As shown, the target image acquired by the camera usually contains lane lines 31, which can be used to help determine the far intersection point 32 of the image. In reality, lane lines 31 are parallel to each other, but in the captured target image, due to the extension of lane lines 31 in the depth direction of the image and the perspective phenomenon of near objects appearing larger and far objects appearing smaller, two or more lane lines 31 will intersect at a point, which is the far intersection point 32, also known as the vanishing point.

[0050] Before determining the far intersection point, lane lines in the target image can be marked, and the lane line information can be saved in the relevant device. Lane lines in the target image may be occluded or incomplete; marking can be done based on partial lane line segments. Real lane lines can be straight lines or curves; marked lane lines can be saved as straight lines or polylines. Straight lines can save the coordinates of the start and end points, while polylines can save the coordinates of multiple points from the start to the end point. If the target image contains two or more lane lines, any two parallel lane lines in the real scene can be selected. Based on the coordinates of these two lane lines, the position or coordinates of the far intersection point can be obtained. The position of the far intersection point can be located on or outside the target image. If the selected lane lines are saved as polylines, the lane line segments located on the left and right sides of the same lane at the bottom of the target image can be taken as the selected lane lines, and the corresponding far intersection points can be obtained.

[0051] Step S202: Obtain the depression angle of the intersection point corresponding to the far intersection point. The depression angle of the intersection point is the angle between the line connecting the camera to the far intersection point in the real scene and the plane where the lane is located.

[0052] Traffic monitoring cameras are typically installed above the road surface, taking downward images of the lanes. In this embodiment, the camera's downward angle can be fixed after installation. Based on the camera's own parameters, the downward angle, and the installation height, the length of the lane in the target image captured by the camera can be determined. If a line is drawn from the camera's location as the starting point to the location of a point in the target image in the real scene as the ending point, the angle between this line and the plane containing the lane is obtained, which is the downward angle corresponding to that point. Figure 4 This is a schematic diagram of the intersection point depression angle and the maximum depression angle in this embodiment, as shown below. Figure 4 As shown, the depression angle θ0 of the intersection point is the depression angle corresponding to the actual position of the far end intersection point, and the maximum depression angle θ n The angle of depression corresponding to the true position of the near end (i.e., the bottom region of the target image) of the target image. The distance between the true position of the far intersection point and the true position of the near end of the target image can be used as the depth estimate corresponding to that far intersection point.

[0053] Step S203: Based on the intersection angle and the maximum angle of the target image, determine the depth estimate of the far intersection point. The maximum angle of depression is the angle between the line connecting the camera to the near end of the target image in the real scene and the plane where the lane is located.

[0054] according to Figure 4 The definition of the mid-point depression angle allows us to establish a correspondence between depth estimates and depression angles. Based on the intersection depression angle, the maximum depression angle, and this correspondence, we can obtain the true distance between the near and far intersection points in the target image, thereby obtaining the depth estimate corresponding to the far intersection point.

[0055] The image depth estimation method provided in this embodiment determines a far-end intersection point based on at least two lane lines in the target image, thus establishing a far-end reference point in the target image for depth estimation. It then determines the relative position of the camera capturing the target image to the far-end intersection point by obtaining the intersection point's depression angle. Finally, it determines the depth estimation value of the far-end intersection point based on the intersection point's depression angle and the target image's maximum depression angle. Specifically, the maximum depression angle determines the relative position of the camera to the near end of the image. Based on the image scaling of the target image between the intersection point's depression angle and the maximum depression angle, the true distance between the two is obtained, and the depth estimation value of the far-end intersection point is determined. In essence, the depth estimation value of the image is obtained by performing lane line recognition and geometric calculations on the target image, solving the problem that image acquisition and processing devices cannot perform depth estimation on video images.

[0056] In some of these embodiments, Figure 5 This is a flowchart of obtaining the depression angle of the intersection point corresponding to the far intersection point in this embodiment, as shown below. Figure 5 As shown, the process includes the following steps:

[0057] Step S501: Obtain the magnification factor of the near end of the target image relative to the far end intersection point.

[0058] Based on features in the target image, such as lane lines or the dimensions of a lane between two lane lines, the magnification factor at the near and far intersections of the target image can be obtained. This magnification factor is the ratio of the actual size of a pixel at the far intersection to the actual size of a pixel at the near intersection. For example, if the actual width of 100 pixels at the near end is 1 meter, and the actual width of 1 pixel at the far intersection is also 1 meter, then the magnification factor is 100. The magnification factor can be calculated based on a pre-defined actual width corresponding to 1 pixel at the far intersection, the actual width of the lane, and its width at the near end of the target image. The width of the lane at the near end of the target image can be obtained using the coordinates of the lane lines at the near end, with the coordinates corresponding to the number of pixels.

[0059] Step S502: Based on the magnification factor, obtain the depression angle of the intersection point corresponding to the far intersection point.

[0060] There is a corresponding relationship between the magnification factor and the angle of depression at the intersection point. The value of the angle of depression at the intersection point can be calculated through this relationship.

[0061] The image depth estimation method provided in this embodiment obtains the magnification factor of the near end of the target image relative to the far end intersection point, and obtains the depression angle of the intersection point corresponding to the far end intersection point based on the magnification factor. That is, it only needs to obtain the near end size of the lane in the target image to calculate the value of the depression angle of the intersection point, without performing complex calculations that consume hardware resources. This provides a feasible way to determine the relative position of the camera that captures the target image and the far end intersection point based on the depression angle of the intersection point, and to obtain the depth estimation value corresponding to the far end intersection point.

[0062] In some embodiments, a specific procedure is involved in obtaining the magnification factor of the near-end intersection point of the target image relative to the far-end intersection point. This procedure includes the following steps:

[0063] Step S11: Obtain the ratio of the near-end pixel width of the lane to the actual width of the lane in the target image;

[0064] Step S12: Based on this ratio and the actual distance between the far end intersection and the near end, obtain the magnification factor. According to steps S11-S12 above, the magnification factor can be calculated using the following formula:

[0065] scale dis =S*(ΔX0 / W) world ) Formula 1

[0066] Where, scale dis Where S is the magnification factor, S is the true distance to the pixel where the far intersection point is located, ΔX0 is the near pixel width of the lane in the target image, and W is the magnification factor. world The actual width of the lane in the target image.

[0067] The magnification factor is the ratio of the true size of a pixel at the far intersection point to the true size of a pixel at the near end point in the target image. According to this definition, S is the true size of a pixel at the far intersection point, and ΔX0 is the near-end pixel width of the lane in the target image, i.e., the number of pixels in the lane along its near-end width. That is, scale. dis =(S / 1) / (W) world / ΔX0)=S*(ΔX0 / W world ).

[0068] The image depth estimation method provided in this embodiment obtains the magnification factor of the near end of the target image relative to the far end intersection point by combining the near end width of the lane in the target image with the actual width of the lane and the actual size corresponding to the preset far end intersection point through simple coordinate transformation and algebraic calculation. Based on this magnification factor, the depth estimation value is obtained, providing a simple method to achieve depth estimation without complex calculations.

[0069] In some embodiments, a specific method is involved in obtaining the angle of depression of the intersection point corresponding to the far intersection point based on a magnification factor. This method includes:

[0070]

[0071] Where θ0 is the angle of depression at the intersection point, scale dis This is the magnification factor. This is for rounding up.

[0072] There is a corresponding relationship between the angle of depression at the intersection point and the magnification factor, which can be described by Equation 2 above. According to Equation 2, the larger the magnification factor, the smaller the angle of depression at the intersection point. To simplify the calculation, the angle of depression at the intersection point is rounded for subsequent calculation of the depth-of-field estimate. It should be noted that the correspondence between the angle of depression at the intersection point and the magnification factor described in Equation 2 can be adjusted based on the fitting of the angle of depression at the intersection point and the magnification factor data obtained from actual tests. Other functions, such as polynomial functions and exponential functions, can also be used for fitting.

[0073] The image depth estimation method provided in this embodiment calculates the intersection point depression angle value through a functional relationship between the intersection point depression angle and the magnification factor, and obtains the intersection point depression angle through simple trigonometric function calculation. Based on the intersection point depression angle, a depth estimation value is obtained, providing a simple method to achieve depth estimation without complex calculations.

[0074] In some embodiments, a specific method relates to determining a depth estimate of a far intersection point based on the intersection's tilt angle and the maximum tilt angle of the target image. This method includes:

[0075] Using the angle of depression at the intersection point as the lower limit of integration and the maximum angle of depression as the upper limit of integration, the camera's angle of depression function is integrated to obtain the depth estimate at the far intersection point. This angle of depression function is the reciprocal of the square of the camera's angle of depression tangent function. The above method can be described by the following formula:

[0076]

[0077] Where D is the depth estimate, θ0 is the angle of depression at the intersection point, and θ n This is the maximum angle of depression.

[0078] Maximum tilt angle θ of the target image nIt can be determined based on the camera's mounting angle, representing the maximum downward angle within the camera's field of view, which is also the downward angle corresponding to the bottom (near end) of the target image.

[0079] Formula 3 above can be obtained through the following derivation process.

[0080] Figure 6 This is a schematic diagram illustrating the principle of calculating image depth of field based on intersection point tilt angle and maximum tilt angle in this embodiment. Figure 6 As shown, by using the differentiation method, the true distance corresponding to the intersection point from the bottom (near end) to the far end of the target image is first divided into countless small segments. Each distance segment is denoted as Δd, and the start and end points of the distance segment are A1 and B1, respectively. Let the camera's mounting height be h2, and θ1 and θ2 represent the depression angles corresponding to the distance segment Δd before and after it. Then Δd can be expressed as:

[0081]

[0082] Correspondingly, on the image, the points corresponding to A1 and B1 are A′1 and B′1, respectively. Let... Figure 6 In the simplified imaging model shown, the virtual distance from the camera lens to the image is represented by h0. It should be noted that the internal parameter h0 is not equivalent to the camera focal length, but rather a virtual distance within this imaging model. The length of A′1B′1 on the target image corresponding to the distance slice Δd, i.e., the number of pixels between A′1 and B′1 on the target image, can be calculated using trigonometric relationships, as follows:

[0083]

[0084] Next, the true distance dp corresponding to a pixel in the target image can be obtained:

[0085]

[0086] Since Δd is divided into infinitely small pieces, their corresponding depression angles can be considered to be similar. Therefore, the denominator terms of the formula can be combined to simplify the formula.

[0087] In real-world scenarios, after the intelligent traffic monitoring cameras are installed and debugged... The value of will no longer change. Therefore, according to the formula for dp, the actual distance represented by each dp is only related to the current corresponding tilt angle θ2 (or θ1). Thus, when calculating depth of field, we can integrate the actual distance corresponding to each pixel by dividing it into segments. The lower limit of integration is 0 (i.e., the bottom of the image), and the upper limit of integration is the number of pixels at the intersection of the bottom of the image and the far end, where Height is the number of pixels in the height direction of the target image, and Y... disHere is the ordinate of the farthest intersection point (coordinate axes from top to bottom). The depth estimate is expressed as:

[0088]

[0089] because Since it is a constant, it can be defined as the error coefficient γ. In the estimation process, it is set to 1 and omitted, thus obtaining Formula 3.

[0090] The image depth estimation method provided in this embodiment calculates the depth estimate by integrating the true distance between each pixel point corresponding to the far-end intersection point and the near-end intersection point in the target image. The depth estimate can be obtained based on the depression angle and maximum depression angle of the intersection point corresponding to the far-end intersection point. The calculation method is integration and trigonometric function operation, which does not require a lot of computing resources or preprocessing of the target image, thus saving computing time and hardware resources and improving the timeliness and efficiency of obtaining the depth estimate.

[0091] In some embodiments, after determining the depth estimate of the far intersection point based on the intersection angle and the maximum angle of depression of the target image, the image depth estimation method further includes:

[0092] The depth-of-field estimate is adjusted based on the camera's set height and internal parameters.

[0093] Furthermore, the specific process for adjusting the depth-of-field estimate includes the following steps:

[0094] Step S21: Obtain the adjustment factor based on the ratio of the camera's set height to its internal parameters;

[0095] Step S22: Obtain the adjusted depth estimate based on the product of the adjustment factor and the depth estimate.

[0096] Steps S21 to S22 above can be expressed by the following formula:

[0097]

[0098] Where D′ is the adjusted depth estimate, D is the depth estimate, h2 is the camera's set height, and h0 is the camera's internal parameter.

[0099] As shown in Formula 7, the depth-of-field estimate can be adjusted using the camera's mounting height h2 and internal parameter h0. Both the mounting height h2 and the internal parameter h0 have a certain impact on the depth of field of the captured image. When these two data points are available, adjusting the depth-of-field estimate based on this data can improve the accuracy of the depth-of-field estimate.

[0100] In some embodiments, a specific method relates to determining a far intersection point based on at least two lane lines in a target image. The method includes:

[0101] Identify at least two lane lines in the target image, which intersect the bottom boundary of the target image and are located on opposite sides of the target image.

[0102] Depending on the camera's location, lane lines in the target image may be located on one or both sides of the image, or in a corner. To obtain the lane width at the near end of the target image, two parallel lane lines intersecting the bottom boundary of the image can be selected. If these two lane lines are located on opposite sides of the target image, the lane width can be directly obtained from the coordinates of the lane lines.

[0103] The image depth estimation method provided in this embodiment determines the position of lane lines in the target image and selects lane lines that meet the conditions for depth estimation. This method can calculate the depth estimation value in a simpler way, saving computing resources and computing time, and improving the efficiency of obtaining the depth estimation value.

[0104] The present embodiment will now be described and illustrated through preferred embodiments.

[0105] Figure 7 This is a flowchart of the image depth estimation method according to a preferred embodiment. Figure 7 As shown, the process includes the following steps:

[0106] S701, mark the lane lines in the target image and save the lane line information in the form of straight lines or polylines;

[0107] S702, Select two lane lines from the lane line information; if the lane lines are stored in polyline form, select one lane and take one line segment from the left and right sides of the lane closest to the bottom of the target image.

[0108] S703, determine the visual vanishing point of the two lane lines, i.e., the far intersection point;

[0109] S704, assuming the ratio of the pixel coordinates at the vanishing point to the world coordinates is 1, i.e., 1 pixel = 1m, let the pixel width of the lane at the bottom of the screen be denoted as ΔX0, and the width of the lane in the traffic scene be denoted as W. world The magnification factor at the bottom near the vanishing point of the image is:

[0110] scale disappear ( / 1)=ΔX0 / W world Formula 9

[0111] S705, calculate the angle of depression θ0 of the intersection point corresponding to the visual vanishing point based on the magnification factor;

[0112]

[0113] S706 calculates the actual longitudinal distance D from the bottom of the target image to the vanishing point based on the intersection angle and the maximum angle of depression, which is the estimated depth information of the current scene. Maximum angle of depression θ n It is determined by the installation angle of the intelligent traffic camera, and it represents the maximum downward angle of the field of view, that is, the downward angle corresponding to the bottom of the captured image.

[0114]

[0115] Through the above steps S701 to S706, the lane lines in the target image are marked and the lane line information is saved to obtain the position of the lane lines in the target image; two lane lines are selected from the lane line information to obtain the lane lines that meet the depth estimation requirements, simplifying the calculation steps; the vanishing point is determined based on the lane lines to determine the far reference point in the target image used for depth estimation; the magnification factor between the near and far intersection points is obtained by calculating the near lane width in the target image, and the depression angle of the intersection point corresponding to the far intersection point is calculated based on the magnification factor to determine the relative position of the camera capturing the target image and the far intersection point; the depth estimation value of the far intersection point is determined by the depression angle and the maximum depression angle. The depth estimation value of the image can be obtained in a timely and efficient manner with only simple lane line recognition and geometric calculation. It does not require the use of high-cost equipment such as binocular cameras, nor does it require the use of deep learning algorithms for image information processing, which consumes hardware resources. This solves the problem that image acquisition and processing equipment cannot perform depth estimation on video images.

[0116] It should be noted that the steps shown in the above process or in the flowchart of the accompanying figures can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0117] In some embodiments, this application also provides an image depth estimation device for implementing the above embodiments and preferred embodiments, and details already described will not be repeated. The terms "module," "unit," "subunit," etc., used below refer to combinations of software and / or hardware that perform predetermined functions.

[0118] In some embodiments, Figure 8 This is a structural block diagram of the image depth estimation device in this embodiment, as shown below. Figure 7 As shown, the device includes:

[0119] The first determining module 81 is used to determine the far intersection point based on at least two lane lines in the target image, wherein the far intersection point is the visual intersection point of the lane lines in the target image; the target image is obtained by a camera set above the lane;

[0120] The acquisition module 82 is used to acquire the depression angle of the intersection point corresponding to the far intersection point; the depression angle of the intersection point is the angle between the line connecting the camera to the far intersection point in the real scene and the plane where the lane is located.

[0121] The second determining module 83 is used to determine the depth estimate of the far intersection point based on the intersection point depression angle and the maximum depression angle of the target image. The maximum depression angle is the angle between the line connecting the camera to the near end of the target image in the real scene and the plane where the lane is located.

[0122] In this embodiment, the image depth estimation device determines the far intersection point based on at least two lane lines in the target image using a first determining module 81, thus determining a far reference point in the target image used for depth estimation. The acquiring module 82 acquires the depression angle of the intersection point corresponding to the far intersection point, determining the relative position of the camera capturing the target image and the far intersection point. The second determining module 83 determines the depth estimation value of the far intersection point based on the depression angle of the intersection point and the maximum depression angle of the target image. Specifically, the maximum depression angle determines the relative position of the camera and the near end of the image. Based on the scaling of the target image between the depression angle of the intersection point and the maximum depression angle, the true distance between the two is obtained, and the depth estimation value of the far intersection point is determined. In other words, by performing lane line recognition and geometric calculations on the target image, the depth estimation value of the image is obtained, solving the problem that image acquisition and processing devices cannot perform depth estimation on video images.

[0123] In some embodiments, the acquisition module includes a first acquisition submodule and a second acquisition submodule. The first acquisition submodule is used to acquire the magnification factor of the near-end intersection point of the target image relative to the far-end intersection point, and the second acquisition submodule is used to acquire the depression angle of the intersection point corresponding to the far-end intersection point based on the magnification factor.

[0124] The image depth estimation device in this embodiment obtains the magnification factor of the near end of the target image relative to the far end intersection point through the first acquisition submodule, and obtains the intersection point depression angle corresponding to the far end intersection point based on the magnification factor through the second acquisition submodule. That is, it only needs to obtain the near end size of the lane in the target image to calculate the value of the intersection point depression angle, without performing complex calculations that consume hardware resources. This provides a feasible way to determine the relative position of the camera that captures the target image and the far end intersection point based on the intersection point depression angle, and to obtain the depth estimation value corresponding to the far end intersection point.

[0125] In some embodiments, the first acquisition submodule includes a first acquisition unit and a second acquisition unit. The first acquisition unit is used to acquire the ratio of the near-end pixel width of the lane to the actual width of the lane in the target image. The second acquisition unit is used to acquire the magnification factor based on the ratio and the actual distance between the far-end intersection and the near-end.

[0126] The image depth estimation device in this embodiment obtains the magnification factor of the near end of the target image relative to the far end intersection point by means of the magnification factor calculation unit based on the near end width of the lane in the target image, combined with the actual width of the lane and the actual size corresponding to the preset far end intersection point, through simple coordinate conversion and algebraic calculation. Based on the magnification factor, the depth estimation value is obtained, providing a simple method to achieve depth estimation without complex calculations.

[0127] In some embodiments, the second acquisition submodule includes an intersection depression angle calculation unit for calculating the intersection depression angle corresponding to the far intersection point based on the following formula:

[0128]

[0129] Where θ0 is the angle of depression at the intersection point, scale dis This is the magnification factor.

[0130] The image depth estimation device in this embodiment calculates the intersection depression angle value based on the functional relationship between the intersection depression angle and the magnification factor through the intersection depression angle calculation unit, and obtains the intersection depression angle through simple trigonometric function calculation. Based on the intersection depression angle, the depth estimation value is obtained, providing a simple method to achieve depth estimation without complex calculations.

[0131] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can reside in the same processor; or the above modules can be located in different processors in any combination.

[0132] This embodiment also provides an image acquisition device, including a lens, a memory, and a processor. The memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any of the above method embodiments to estimate the depth of field of the image acquired by the lens.

[0133] In this embodiment, the image acquisition device acquires a target image through a lens and then determines a far intersection point based on the target image. By obtaining the depression angle of the intersection point, the relative position between the image acquisition device and the far intersection point is determined. The depth estimate of the far intersection point is determined based on the depression angle of the intersection point and the maximum depression angle of the target image. By performing lane line recognition and simple mathematical calculations on the target image, depth estimation can be achieved without complex calculations or machine learning, thus meeting the needs of the image acquisition device for image depth estimation under limited hardware resources.

[0134] Optionally, the image acquisition device may further include a communication device and an input / output device, wherein the communication device is connected to the processor and the input / output device is connected to the processor.

[0135] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated in this embodiment.

[0136] Furthermore, in conjunction with the image depth estimation methods provided in the above embodiments, this embodiment can also provide a storage medium for implementation. This storage medium stores a computer program; when executed by a processor, the computer program implements any of the image depth estimation methods described in the above embodiments.

[0137] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated in this embodiment.

[0138] It should be understood that the specific embodiments described herein are merely illustrative of the application and not intended to limit it. All other embodiments derived by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0139] Obviously, the accompanying drawings are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar situations based on these drawings without any creative effort. Furthermore, it is understood that although the work done in this development process may be complex and lengthy, for those skilled in the art, certain design, manufacturing, or production modifications made based on the technical content disclosed in this application are merely conventional technical means and should not be considered as insufficient disclosure of this application.

[0140] The term "embodiment" in this application refers to a specific feature, structure, or characteristic described in connection with an embodiment that may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily imply the same embodiment, nor does it imply that it is mutually exclusive with or independent of other embodiments. It will be clearly or implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0141] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of patent protection. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the appended claims.

Claims

1. An image depth estimation method, characterized in that, The method includes: Based on at least two lane lines in the target image, a far intersection point is determined, wherein the far intersection point is the visual intersection point of the lane lines in the target image; the target image is obtained by a camera positioned above the lane. Obtain the depression angle of the intersection point corresponding to the far intersection point; the depression angle of the intersection point is the angle between the line connecting the camera to the far intersection point in the real scene and the plane where the lane is located; Based on the angle of depression at the intersection point and the maximum angle of depression of the target image, the depth estimate of the far intersection point is determined, wherein the maximum angle of depression is the angle between the line connecting the camera to the near end of the target image in the real scene and the plane where the lane is located. The process of determining the depth estimate of the far intersection point based on the depression angle of the intersection point and the maximum depression angle of the target image includes: Using the depression angle at the intersection point as the lower limit of integration and the maximum depression angle as the upper limit of integration, the depression angle function of the camera is integrated to obtain the depth estimate of the far intersection point. The depression angle function is the reciprocal of the square of the depression angle tangent function of the camera.

2. The method according to claim 1, characterized in that, The step of obtaining the depression angle of the intersection point corresponding to the far intersection point includes: Obtain the magnification factor of the near end of the target image relative to the far end intersection point; Based on the magnification factor, the depression angle of the intersection point corresponding to the far end intersection point is obtained.

3. The method according to claim 2, characterized in that, The magnification factor of the near end of the target image relative to the far end intersection point includes: Obtain the ratio of the near-end pixel width of the lane in the target image to the actual width of the lane; The magnification factor is obtained based on the ratio and the actual distance between the distal intersection and the proximal end.

4. The method according to claim 2, characterized in that, The step of obtaining the angle of depression of the intersection point corresponding to the far intersection point based on the magnification factor includes: Where θ0 is the angle of depression at the intersection point, scale dis The magnification factor is denoted as .

5. The method according to claim 1, characterized in that, After determining the depth estimate of the far intersection point based on the depression angle of the intersection point and the maximum depression angle of the target image, the method further includes: The depth-of-field estimate is adjusted based on the camera's set height and internal parameters.

6. The method according to claim 5, characterized in that, The adjustment of the depth-of-field estimate based on the camera's set height and internal parameters includes: An adjustment factor is obtained based on the ratio of the camera's set height to its internal parameters; The adjusted depth estimate is obtained by multiplying the adjustment factor by the depth estimate.

7. The method according to claim 1, characterized in that, Determining the far intersection point based on at least two lane lines in the target image includes: Identify at least two lane lines in the target image, wherein the at least two lane lines intersect the bottom boundary of the target image and are located on opposite sides of the target image.

8. An image depth estimation device, characterized in that, The device includes: The first determining module is used to determine the far intersection point based on at least two lane lines in the target image, wherein the far intersection point is the visual intersection point of the lane lines in the target image; the target image is obtained by a camera set above the lane; The acquisition module is used to acquire the depression angle of the intersection point corresponding to the far intersection point; the depression angle of the intersection point is the angle between the line connecting the camera to the far intersection point in the real scene and the plane where the lane is located. The second determining module is used to determine the depth estimate of the far intersection point based on the depression angle of the intersection point and the maximum depression angle of the target image, wherein the maximum depression angle is the angle between the line connecting the camera to the near end of the target image in the real scene and the plane where the lane is located. The process of determining the depth estimate of the far intersection point based on the depression angle of the intersection point and the maximum depression angle of the target image includes: Using the depression angle at the intersection point as the lower limit of integration and the maximum depression angle as the upper limit of integration, the depression angle function of the camera is integrated to obtain the depth estimate of the far intersection point. The depression angle function is the reciprocal of the square of the depression angle tangent function of the camera.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the image depth estimation method according to any one of claims 1 to 7.

Images