A method, device, and storage medium for road surface anomaly detection based on inverse depth maps

Abstract

This invention relates to a method, apparatus, and storage medium for road anomaly detection based on inverse depth maps. The method includes the following steps: acquiring a dense depth map of the road surface based on a binocular stereo matching algorithm or a monocular depth estimation network, and taking the reciprocal of the depth value of each point in the dense depth map to obtain a dense inverse depth map; estimating the roll angle of a depth camera using the coordinates and inverse depth values ​​of each point in the inverse depth map; estimating road surface projection model parameters using the coordinates, inverse depth values, and roll angle of each point in the inverse depth map; and transforming the inverse depth map using the coordinates, inverse depth values, roll angle of the depth camera, and road surface projection model parameters to obtain a road anomaly detection result map. Compared with existing technologies, this invention has advantages such as high computational efficiency and wide applicability.

Description

Technical Field

[0001] This invention relates to the field of robot vision, and in particular to a method, apparatus and storage medium for road surface anomaly detection based on inverse depth maps. Background Technology

[0002] In recent years, digital twin technology has been frequently used to build smart cities, creating related urban simulation systems for unified deployment and control of urban transportation, smart healthcare, and engineering scheduling. Road construction is not only significant for urban planning, economic development, and transformation, but also has a huge impact on people's daily lives. Smart systems that use digital twin technology to model and manage transportation hubs have entered the public eye. Road safety information is transmitted to the cloud in real time, allowing city managers to arrange road maintenance, road monitoring, and traffic flow management based on this information. Meanwhile, the public can not only choose routes and plan navigation routes in real time based on cloud-based twin information, but also upload information and participate in city management. The entire city achieves real-time information sharing, making digital road traffic information readily available. In the context of smart cities, road environment perception technology provides the primary support for road inspection and has become a focal point of attention in the field of autonomous driving in recent years.

[0003] Roads are the carriers of urban transportation and play a vital role in people's daily lives. However, with economic development, the pressure on road surfaces is increasing, leading to problems such as potholes and cracks. These road surface anomalies (small obstacles like bricks or road damage such as potholes and cracks) affect road conditions, reduce the lifespan of vehicles and roads, and even threaten traffic safety. Currently, fields such as mobile robots, intelligent wheelchairs, and autonomous driving focus primarily on detecting larger targets, such as pedestrians, lane markings, and signs. However, detecting small road obstacles is also crucial, as it can affect the intelligent sensing of small robots' movement direction; and for intelligent wheelchairs and autonomous vehicles, small obstacles can easily cause accidents, not only damaging the vehicle itself but also threatening life and property. Summary of the Invention

[0004] The purpose of this invention is to provide a road surface anomaly detection method, device, and storage medium based on inverse depth maps, which can efficiently, quickly, and accurately detect road surface anomalies.

[0005] The objective of this invention can be achieved through the following technical solutions:

[0006] A road surface anomaly detection method based on inverse depth maps includes the following steps:

[0007] S1. Obtain a dense depth map of the road surface based on a binocular stereo matching algorithm or a monocular depth estimation network, and take the reciprocal of the depth value of each point in the dense depth map to obtain a dense inverse depth map.

[0008] S2. Estimate the roll angle of the depth camera using the coordinates of each point in the inverse depth map and the inverse depth value;

[0009] S3. Estimate the road surface projection model parameters using the coordinates of each point in the inverse depth map, the inverse depth value, and the roll angle of the depth camera.

[0010] S4. Transform the inverse depth map using the coordinates of each point in the inverse depth map, the inverse depth value, the roll angle of the depth camera, and the road surface projection model parameters to obtain the road surface anomaly detection result map.

[0011] The stereo matching algorithms include traditional methods such as Normalized Cross-Correlation (NCC), SAD (Sum of Absolute Difference), SGM (Semi-Global Matching), Graph Cuts, and Belief Propagation, as well as stereo matching networks: PSMNet (an end-to-end stereo matching network based on pyramids), GANet (a global association network based on key points), and RAFT-Stereo (a multi-level cyclic field transformation stereo matching network).

[0012] The monocular depth estimation networks include MonoDepth and MonoViT.

[0013] The obtained dense depth map is denoted as Z, with height H and width W. The coordinates of a given point P are (x, y), where x is the x-coordinate of point P, y is the y-coordinate of point P, and the depth value of point P is z. Then, the inverse depth map D is obtained by taking the reciprocal of the depth value of each point. If the value of z is 0 or ∞, the inverse depth value of the corresponding position is set to NaN.

[0014] S2 includes the following steps:

[0015] S21. Calculate the model parameters, where N is the number of points in the inverse depth map D whose value is not NaN, and P... k Let k be the k-th non-NaN point from left to right and top to bottom in the inverse depth map, with coordinates (x...). k ,y k Its inverse depth value is d. k :

[0016]

[0017]

[0018]

[0019]

[0020]

[0021]

[0022]

[0023] S22, Calculate the roll angle Angle, energy equation in and When it reaches an extreme state:

[0024]

[0025]

[0026] in,

[0027]

[0028] Ψ = e1(e4-e6)-e2(e3-e5)

[0029] Θ=e7(e3-e5+e6-e4)+e8(e2-e1)

[0030] S23, respectively and Substitute into the energy equation:

[0031]

[0032] S24. Solving the roll angle of the depth camera based on the energy equation.

[0033]

[0034] Specifically, S3 is:

[0035]

[0036] C1 and C2 are the parameters of the road surface projection model.

[0037] The method for transforming the inverse depth map in S4 is as follows:

[0038]

[0039] Where T is the transformed inverse depth map, and ω is the transformation constant.

[0040] The transformation constant ω is:

[0041]

[0042] A road surface anomaly detection device based on inverse depth map includes a memory, a processor, and a program stored in the memory. When the processor executes the program, it implements the method described above.

[0043] A storage medium having a program stored thereon, which, when executed, implements the method described above.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] (1) In the road surface anomaly detection result diagram obtained by the present invention, the outline of the road surface anomaly is clear and the detection effect is good.

[0046] (2) The method used in this invention is simple, has low computational complexity, and can quickly calculate the transformation results, thereby improving detection efficiency.

[0047] (3) The method of the present invention has strong applicability and can cope with the detection of most road surface abnormalities. Attached Figure Description

[0048] Figure 1 This is a flowchart of the method of the present invention;

[0049] Figure 2 The diagram shows the results of road surface anomaly detection, where (a) is the original image and (b) is the transformed detection result. Detailed Implementation

[0050] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0051] This embodiment provides a road surface anomaly detection method based on inverse depth maps, such as... Figure 1 As shown, it includes the following steps:

[0052] S1. Obtain a dense depth map of the road surface based on a binocular stereo matching algorithm or a monocular depth estimation network, and take the reciprocal of the depth value of each point in the dense depth map to obtain a dense inverse depth map.

[0053] S11. Obtain dense depth information using a binocular stereo matching algorithm or a monocular depth estimation network.

[0054] In this embodiment, dense depth information is obtained through binocular stereo matching algorithms, such as traditional methods like Normalized Cross-Correlation (NCC), SAD (Sum of Absolute Difference), SGM (Semi-Global Matching), Graph Cuts, and Belief Propagation, as well as stereo matching networks such as Pyramid-based end-to-end stereo matching network (PSMNet), keypoint-based global association network (GANet), and multi-level cyclic field transformation stereo matching network (RAFT-Stereo).

[0055] S12. Take the reciprocal of the depth value of each point in the dense depth map to obtain the dense inverse depth map: The obtained dense depth map is denoted as Z. Its height is H and its width is W. The coordinates of a given point P are (x, y), where x is the x-coordinate of point P and y is the y-coordinate. The depth value of point P is z. Taking the reciprocal of the depth value of each point yields an inverse depth map D (if z is 0 or ∞, the corresponding inverse depth value is set to NaN). The number of points in inverse depth map D that are not NaN is N. The k-th point from left to right and from top to bottom that is not NaN is denoted as P. k Its coordinates are (x k ,y k Its inverse depth value is denoted as d. k .

[0056] S2. Estimate the roll angle of the depth camera using the coordinates of each point in the inverse depth map and the inverse depth value.

[0057] S21. Calculate the model parameters, where N is the number of points in the inverse depth map D whose value is not NaN, and P... k Let k be the k-th non-NaN point from left to right and top to bottom in the inverse depth map, with coordinates (x...). k ,y k Its inverse depth value is d. k :

[0058]

[0059] S22, Calculate the roll angle Angle, energy equation in and When it reaches an extreme state:

[0060]

[0061]

[0062] in,

[0063]

[0064] Ψ = e1(e4-e6)-e2(e3-e5)

[0065] Θ=e7(e3-e5+e6-e4)+e8(e2-e1)

[0066] S23, respectively and Substitute into the energy equation:

[0067]

[0068] S24. Solving the roll angle of the depth camera based on the energy equation.

[0069]

[0070] S3. Estimate the road surface projection model parameters using the coordinates of each point in the inverse depth map, the inverse depth value, and the roll angle of the depth camera.

[0071] Obtain the roll angle of the depth camera Then, the road surface projection model parameters C1 and C2 can be calculated using the following formula:

[0072]

[0073] S4 transforms the inverse depth map using the coordinates of each point in the inverse depth map, the inverse depth value, the roll angle of the depth camera, and the parameters of the road projection model.

[0074] The inverse depth map transformation can be achieved using the following formula:

[0075]

[0076] Where T is the transformed inverse depth map, and ω is a constant, which can be calculated using the following formula:

[0077]

[0078] like Figure 2 As shown, potholes and abnormal protrusions are deepened in outline after transformation using the method described in this invention, making them easier to distinguish. That is, road surface anomalies (such as small obstacles or damaged areas) are clearly visible in the transformed inverse depth map, enabling road surface anomaly detection.

[0079] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A road surface anomaly detection method based on inverse depth maps, characterized in that, Includes the following steps: S1. Obtain a dense depth map of the road surface based on a binocular stereo matching algorithm or a monocular depth estimation network, and take the reciprocal of the depth value of each point in the dense depth map to obtain a dense inverse depth map. S2. Estimate the roll angle of the depth camera using the coordinates of each point in the inverse depth map and the inverse depth value; S3. Estimate the road surface projection model parameters using the coordinates of each point in the inverse depth map, the inverse depth value, and the roll angle of the depth camera. S4. Transform the inverse depth map using the coordinates of each point in the inverse depth map, the inverse depth value, the roll angle of the depth camera, and the road surface projection model parameters to obtain the road surface anomaly detection result map; The obtained dense depth map is denoted as Z, with height H and width W. The coordinates of a given point P are (x, y), where x is the x-coordinate of point P, y is the y-coordinate of point P, and the depth value of point P is z. Then, the inverse depth map D is obtained by taking the reciprocal of the depth value of each point. If the value of z is 0 or ∞, the inverse depth value at the corresponding position is set to NaN. S2 includes the following steps: S21. Calculate the model parameters, where N is the number of points in the inverse depth map D whose value is not NaN, and P... k Let k be the k-th non-NaN point from left to right and top to bottom in the inverse depth map, with coordinates (x...). k ,y k Its inverse depth value is d. k : S22, Calculate the roll angle Angle, energy equation in and When it reaches an extreme state: in, Ψ = e1(e4-e6)-e2(e3-e5) Θ=e7(e3-e5+e6-e4)+e8(e2-e1) S23, respectively and Substitute into the energy equation: S24. Solving the roll angle of the depth camera based on the energy equation.

2. The road surface anomaly detection method based on inverse depth map according to claim 1, characterized in that, The stereo matching algorithms include traditional methods such as Normalized Cross-Correlation (NCC), SAD, SGM, Graph Cut, and Belief Propagation, as well as stereo matching networks: Pyramid-based end-to-end stereo matching network PSMNet, Keypoint-based Global Association Network (GANet), and Multi-level Cyclic Field Transform Stereo matching network RAFT-Stereo.

3. The road surface anomaly detection method based on inverse depth map according to claim 1, characterized in that, The monocular depth estimation networks include MonoDepth and MonoViT.

4. The road surface anomaly detection method based on inverse depth map according to claim 1, characterized in that, Specifically, S3 is: C1 and C2 are the parameters of the road surface projection model.

5. The road surface anomaly detection method based on inverse depth map according to claim 4, characterized in that, The method for transforming the inverse depth map in S4 is as follows: Where T is the transformed inverse depth map, and ω is the transformation constant.

6. The road surface anomaly detection method based on inverse depth map according to claim 5, characterized in that, The transformation constant ω is:

7. A road surface anomaly detection device based on inverse depth maps, comprising a memory, a processor, and a program stored in the memory, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1-6.

8. A storage medium having a program stored thereon, characterized in that, When the program is executed, it implements the method as described in any one of claims 1-6.

Images