Automated pavement distress inspection method

Abstract

This invention relates to an automated method for inspecting road pavement damage. It involves continuously acquiring multiple frames of road images of a real road, directly using a deep learning model to identify the pavement damage areas and their pixel coordinates in each frame. Based on continuously acquired camera exterior orientation parameters, a road reference plane in a global coordinate system is estimated. The pixel coordinates of the pavement damage areas in these road image frames are then converted to global coordinates and projected onto the road reference plane, thus forming a complete road pavement damage image. Therefore, this invention directly identifies the pavement damage areas from the original resolution road image and projects only the pavement damage areas onto the road reference plane. This avoids dense point cloud operations and significantly reduces data processing time compared to projecting the road image first and then identifying the pavement damage areas.

Claims

1. An automated road pavement damage inspection method, executed by at least one computer device with computing power, wherein the automated road pavement damage inspection method includes the following steps: (a) continuously acquiring multiple frames of road images of a real road and camera exterior orientation parameters; (b) identifying the pavement damage area and its pixel coordinates in each frame of road images using a deep learning model; (c) estimating a road reference plane in a world coordinate system based on the continuously acquired camera exterior orientation parameters, and establishing a conversion relationship between pixel coordinates and world coordinates; and (d) converting the pixel coordinates of the pavement damage area in the frames of road images in step (b) to world coordinates based on the conversion relationship between pixel coordinates and world coordinates, and projecting them onto the road reference plane to form a complete road pavement damage image.

2. The automated road pavement damage inspection method as described in request item 1, wherein: The camera orientation parameters in step (a) include the camera orientation and three-dimensional coordinates; and step (c) further includes: (c1) obtaining the elevation of the three-dimensional coordinates of each frame of road image; (c2) shifting the obtained elevation downwards according to a preset camera height above the ground to estimate the ground point corresponding to the captured frame of road image; (c3) assuming the pavement is a flat and continuous surface, constructing a smooth and continuous elevation curve by interpolation of the multiple ground points estimated from the continuously acquired road images in step (c2); and (c4) expanding the continuous elevation curve to obtain an approximate numerical elevation model (DEM) and using it as the reference surface for the road.

3. The automated road pavement damage inspection method as described in claim 2, wherein the camera exterior orientation parameters in step (a) are obtained by the following steps: (a1) obtaining the initial camera exterior orientation parameters and the camera interior orientation parameters obtained after calibration, and reconstructing the three-dimensional scene; (a2) re-estimating the initial camera exterior orientation parameters using a structure for motion inference (SfM) technique, and calculating the relative attitude by matching the features between multiple overlapping images; and (a3) ​​converting the camera attitude calculated in step (a2) to the absolute orientation in world coordinates using GPS coordinates with geometric constraints.

4. An automated road pavement damage inspection method as described in any one of claims 1 to 3, wherein: In step (b), the deep learning model identifies the image pixels of the pavement damage area in each frame of road imagery and generates a binary image accordingly; and in step (d), the binary images are projected onto the road reference plane after coordinate transformation.

5. The automated road pavement damage inspection method as described in claim 4 further includes step (e): calculating the physical dimensions of the pavement damage area in the full road pavement damage image.

6. The automated road pavement damage inspection method as described in claim 5, wherein the physical dimension is the actual area or structural geometric feature.

7. The automated road pavement damage inspection method as described in claim 6, wherein the pavement damage areas in the full road pavement damage image in step (e) are calculated based on vector data processing, and then their actual area or structural geometric features are calculated.

8. The automated road pavement damage inspection method as described in claim 7, wherein: The deep learning model in step (b) is trained to include a patch semantic segmentation model, a crack semantic segmentation model, and a longitudinal and transverse crack semantic segmentation model, and generates a patch mask, a crack mask, and a longitudinal and transverse crack respectively; wherein the deep learning model identifies the pavement damage area in each frame of road imagery by the following steps: (b1) using the patch mask, the crack mask, and the longitudinal and transverse crack mask, extracting the patch pixels, crack pixels, and longitudinal and transverse crack pixels in the frames of road imagery, and generating a patch binary image, a crack binary image, and a longitudinal and transverse crack binary image accordingly; and (b2) A patch outline is obtained from the patch binary image and a patch outline binary image is generated; a crack outline is obtained from the crack binary image and a crack outline binary image is generated; the longitudinal and transverse crack binary images and the crack binary image are differentially combined, and the difference result is framed to obtain a single-pixel-wide longitudinal and transverse crack binary image; and step (d) further includes the following steps: (d1) the pixel-converted coordinates of the patch outline binary image and the crack outline binary image are projected onto the road reference surface according to the camera exterior orientation parameters to form independent patch and crack polygons, and then these independent patch and crack polygons are combined; and (d2) after sorting the pixels of the longitudinal and transverse crack binary image, the pixel-converted coordinates are projected onto the road reference surface again according to the camera exterior orientation parameters, and duplicate longitudinal and transverse cracks are deleted.

9. The automated road pavement damage inspection method as described in claim 8, wherein step (d2) involves covering the longitudinal and transverse crack pixel projections of the binary images of longitudinal and transverse cracks in two adjacent frames, and then deleting the uncovered pixels.

10. The automated road pavement damage inspection method as described in claim 9, wherein step (e) is to calculate the area of ​​the patch and the area of ​​the crack after the union of step (d1), and to calculate the length of the longitudinal and transverse cracks obtained in step (d2).

Images