A disease space-time tracing method, product, medium and equipment based on pavement disease alignment

Abstract

The application discloses a disease space-time tracing method, product, medium and equipment based on pavement disease alignment, relates to the technical field of road disease detection, and comprises the following steps: acquiring original pavement images and image information of a road to be detected and saving the original pavement images and the image information in an original pavement image information database; performing pavement disease identification on the original pavement images, obtaining disease information, and saving the disease information, the corresponding original pavement images and the image information in a disease database; calculating disease center point coordinates based on the disease information and determining original disease images; determining disease overhead images according to the original disease images; rotating the azimuth angle of the disease overhead images to obtain disease alignment images after pavement disease alignment; performing disease matching according to the disease alignment images, and performing disease space-time tracing based on the disease matching results. The disease space-time tracing method, product, medium and equipment based on pavement disease alignment can effectively improve the disease space-time tracing efficiency and accuracy.

Description

Technical Field

[0001] This invention relates to the field of road defect detection technology, and in particular to a method, product, medium and equipment for spatiotemporal tracing of road defects based on pavement defect alignment. Background Technology

[0002] With the rapid development of road and transportation construction, a vast network of municipal, expressway, and medium-low-grade roads has been established. As this network expands rapidly, research on road maintenance is becoming increasingly in-depth. Currently, intelligent devices are commonly used to collect data on pavement health conditions to promptly detect and address pavement defects. In repairing and treating pavement defects, in addition to timely superficial repairs, researchers often need to trace the historical context of the defects, analyze their causes, and tailor more effective repair plans based on these causes, providing more reliable data support for subsequent pavement construction. Furthermore, it is hoped that defect tracing can be used to monitor changes in road defects.

[0003] Spatiotemporal tracing of road defects refers to the process of combining images of the same defect taken at different times and from different angles to clearly demonstrate its evolution. This helps researchers analyze the causes and severity of the defect, and develop targeted and effective maintenance plans. Besides traditional manual observation methods, which involve manually combining images of the same defect taken at different times, with different equipment, and from different angles, it is also possible to extract defect features through defect feature matching and automatically combine images of the same defect taken at different times, with different equipment, and from different angles based on these features.

[0004] Traditional methods for tracing the spatiotemporal origin of plant diseases involve manually matching data to search for all historically similar diseases near the target disease based on location information and disease name. These are then manually verified and compared one by one to identify all historically collected information for the target disease, thus completing the spatiotemporal tracing. However, this method is labor-intensive, time-consuming, cannot be automated, and is difficult to apply to all diseases.

[0005] Currently, a popular method for spatiotemporal tracing of road damage involves matching disease features. This method identifies historical similar damage in the vicinity of the target damage, extracts damage features from images of the damage area using a disease feature extraction algorithm, and then calculates the similarity between the target damage's features and those of similar historical damage, automatically matching based on the similarity. This method requires no manual intervention and performs the matching automatically. However, this method heavily relies on the quality of the extracted disease features. Because the same damage can appear differently in images taken from different angles and locations, directly extracting disease features from the original road surface image for matching is difficult, resulting in low accuracy and consequently low accuracy and efficiency in spatiotemporal tracing of road damage. Summary of the Invention

[0006] The purpose of this invention is to provide a method, product, medium, and equipment for spatiotemporal tracing of road surface defects based on road defect alignment, which can effectively improve the efficiency and accuracy of spatiotemporal tracing of road defects.

[0007] To achieve the above objectives, the present invention provides the following solution:

[0008] A method for spatiotemporal tracing of road surface defects based on road defect alignment includes:

[0009] The original road surface image and image information of the road to be detected are acquired and stored in the original road surface image information database; the image information includes image name, acquisition time and GPS positioning information; the GPS positioning information includes longitude, latitude and azimuth.

[0010] The original pavement image is subjected to pavement defect identification to obtain defect information, and the defect information, along with the corresponding original pavement image and image information, is stored in the defect database; the defect information includes defect coordinates, defect name, and defect ID;

[0011] Calculate the coordinates of the disease center point based on disease information and determine the original disease image;

[0012] Determine the top-view image of the disease based on the original disease image;

[0013] Rotate the top view image of the road surface defects to obtain the aligned image of the road surface defects;

[0014] Disease matching is performed based on disease-aligned images, and spatiotemporal tracing of diseases is performed based on the disease matching results.

[0015] Optionally, before calculating the coordinates of the disease center point based on disease information and determining the original disease image, the method further includes:

[0016] The coordinates of the corner points of the checkerboard pattern of known size on the original road surface image and the coordinates of the corner points of the checkerboard pattern of known size on the top view image are collected. The projection mapping matrix Pmat from the original road surface image to the top view image is obtained by using the getPerspectiveTransform method in OpenCV.

[0017] Optionally, before calculating the coordinates of the disease center point based on disease information and determining the original disease image, the method further includes:

[0018] The inverse of the projection mapping matrix Pmat from the original road surface image to the top view image is calculated using the np.linalg.inv method in NumPy, thus obtaining the mapping matrix InvPmat from the top view image to the original image; or, based on the coordinates of the checkerboard corner points of known size on the top view image and the coordinates of the checkerboard corner points of known size on the original road surface image, the mapping matrix InvPmat from the top view image to the original image is obtained using the getPerspectiveTransform method in OpenCV.

[0019] Optionally, the step of calculating the coordinates of the disease center point and determining the original disease image based on disease information specifically includes:

[0020] Calculate the coordinates of the disease center point (point_center) based on the disease coordinates (box) in the disease information;

[0021] Based on the coordinates of the disease center point point_center and the projection mapping matrix Pmat, the coordinates of the disease center point on the top view image are obtained by the cv2.perspectiveTransform method in OpenCV.

[0022] On the top-view image, define a square region bev_box with a side length of 2 meters, centered on the center point of the lesion.

[0023] Based on the square region bev_box and the inverse transformation matrix InvPmat, the cv2.perspectiveTransform method in OpenCV is used to obtain the region mbox of the square region on the original road surface image.

[0024] The region mbox is cropped from the original road surface image to obtain the original defect image boxImage.

[0025] Optionally, determining the top-view image of the lesion based on the original lesion image specifically includes:

[0026] Based on the coordinates of the top left, top right, bottom left and bottom right corners of the original disease image boxImage and the coordinates of the top left, top right, bottom left and bottom right corners of the top view image after the original disease image boxImage is transformed, the projection mapping matrix newPmat of the original road surface image to the top view image of the disease area is calculated using the cv2.getPerspectiveTransform method in OpenCV.

[0027] Based on the projection mapping matrix newPmat and the original disease image boxImage, the top view image boxImagebev of the disease is obtained using the cv2.warpPerspective method.

[0028] Optionally, rotating the top view image of the road surface defects to obtain an aligned image of the road surface defects specifically includes:

[0029] Based on the azimuth angle Azimuth in the GPS positioning information corresponding to the disease top view image boxImagebev, the rotation matrix Tmat is obtained using the cv2.getRotationMatrix2D(image_center, 0-Azimuth) method in OpenCV; where image_center is the coordinate of the center point of the disease top view image boxImagebev;

[0030] Based on the top-view image of the lesion boxImagebev and the rotation matrix Tmat, the top-view image of the lesion with an azimuth angle of 0 degrees is calculated using the cv2.warpAffine method in OpenCV and used as the lesion alignment image Damage_Align_Image.

[0031] Optionally, the step of matching diseases based on the disease-aligned image specifically includes:

[0032] A disease feature extraction algorithm is used to extract disease features from the disease-aligned image Damage_Align_Image to obtain a disease feature vector.

[0033] Store the disease ID and disease feature Vector into the disease feature library DamageVectorLib;

[0034] Based on the current GPS location information of the disease, query the DamageDatabase to search for the presence of the same type of disease in nearby historical diseases;

[0035] If there are no diseases of the same type in the nearby historical diseases, a new disease attribution ID will be generated for the current disease.

[0036] If there are diseases of the same type in the nearby historical diseases, the corresponding disease features are extracted from the disease feature library DamageVectorLib according to different disease IDs.

[0037] Calculate the similarity between the extracted disease features and the current disease features. If the maximum similarity among the various similarities is less than the similarity threshold, a new disease attribution ID is generated for the current disease. If the maximum similarity is greater than or equal to the similarity threshold, it is determined that the current disease and the disease corresponding to the maximum similarity belong to the same disease. The disease attribution ID of the disease corresponding to the maximum similarity is used as the disease attribution ID of the current disease, and the disease attribution ID is stored in the disease database DamageDatabase.

[0038] A computer program product includes a computer program that, when executed by a processor, implements the spatiotemporal tracing method for road surface distress alignment.

[0039] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the spatiotemporal tracing method for road surface defects based on alignment.

[0040] A computer device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the spatiotemporal tracing method for road surface defects based on alignment.

[0041] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0042] This invention provides a method, product, medium, and equipment for spatiotemporal tracing of road surface defects based on defect alignment. It incorporates the concept of defect alignment, similar to face alignment in face matching. Before extracting defect features, defect areas captured from different locations and angles are aligned to the same viewpoint. Aligned defect images reduce the impact of shooting angle and position on defect imaging, resulting in higher similarity for images of the same defect. This improves the accuracy of defect matching during feature extraction and matching. Therefore, the spatiotemporal tracing method, product, medium, and equipment based on road surface defect alignment provided by this invention can effectively improve the efficiency and accuracy of spatiotemporal defect tracing. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0044] Figure 1 A schematic diagram of the spatiotemporal tracing method for road surface defects based on alignment provided by the present invention;

[0045] Figure 2 This is a schematic diagram of the original road surface image provided by the present invention;

[0046] Figure 3 A schematic diagram of a top-view image after conversion of the original road surface image provided by the present invention;

[0047] Figure 4 A schematic diagram of the structure of the high-frequency intelligent road surface defect inspection system provided by the present invention. Detailed Implementation

[0048] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0049] Current popular methods for attributing road surface defects based on their characteristics heavily rely on these characteristics. Specifically, they depend on the similarity of images of the same defect taken at different times, with different equipment, and from different angles. Typically, defect features are extracted directly from the defect areas in the original road surface image. However, this process is significantly affected by the shooting angle and position, resulting in low accuracy for defect matching based solely on feature extraction from the original road surface image. Therefore, the purpose of this invention is to provide a method, product, medium, and equipment for spatiotemporal tracing of road surface defects based on defect alignment, which can effectively improve the efficiency and accuracy of spatiotemporal tracing.

[0050] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0051] like Figure 1 As shown, the spatiotemporal tracing method for road surface defects based on alignment disclosed in this invention includes:

[0052] Step 1: Obtain the original road surface image and image information of the road to be detected, and save it in the original road surface image information database.

[0053] The patrol vehicle conducts multiple patrols on the road to be inspected, collecting raw road surface images (ImageData) and recording the GPS location information, image name (ImageName), and collection time (CollectionTime) for each image. The GPS location information includes longitude, latitude, and azimuth. The image information, including collectionTime, image name, longitude, latitude, and azimuth, is stored in the cloud-based raw road surface image database (SrcImageDatabase).

[0054] Step 2: Identify road surface defects in the original road surface image to obtain defect information, and save the defect information together with the corresponding original road surface image and image information in the defect database.

[0055] The system queries the original pavement image and image information in the original pavement image information database (SrcImageDatabase) to obtain image information such as collection time, image name, longitude, latitude, and azimuth. It then reads the original pavement image containing the defect images and inputs it into a defect detection algorithm (generally a pre-trained deep learning neural network algorithm). Using this algorithm, it identifies defects on each original pavement image, obtaining defect coordinates (box(x1, y1, x2, y2)) and defect name (Label). A defect ID (a unique identifier for each defect) is generated. The defect information, including the defect coordinates (box(x1, y1, x2, y2), defect name (Label), and defect ID), along with the image information, is stored in the defect database (DamageDatabase). Here, (x1, y1) represents the pixel coordinates of the top-left corner of the defect rectangle on the original image; (x2, y2) represents the pixel coordinates of the bottom-right corner of the defect rectangle on the original image. The disease ID is a unique identifier for a disease; it is the only ID that can be identified.

[0056] Before aligning road surface defects, the following preparatory work is required:

[0057] A positioning module and an image acquisition module are installed on the patrol vehicle that collects images. The image acquisition module can be a camera.

[0058] By using the coordinates of known points on the original road surface image acquired by the camera to the coordinates of known points on the top view image, the projection mapping matrix Pmat from the original road surface image acquired by the camera to the top view image is calculated.

[0059]

[0060] Where Pmat is the projection mapping matrix from the original image pixel space to the top view image pixel space, a 11 a 12 a 21 a 22 All are linear transformation elements, a 13 a 23 All are elements that undergo translational transformation, a 31 a 32 a 33 All are perspective transformation elements.

[0061]

[0062] Where X and Y are the pixel coordinates in the top view image, and x and y are the pixel coordinates in the original road surface image.

[0063] Specifically, using a chessboard of known size, the coordinates of the corner points of the chessboard grid of known size on the original road surface image and the coordinates of the corner points of the chessboard grid of known size on the top view image are substituted into formula (2) to calculate the projection mapping matrix Pmat. Figure 2As shown, the pixel coordinates of the four corner points of the checkerboard pattern in the original road surface image are [(xtl, ytl), (xtr, ytr), (xbl, ybl), (xbr, ybr)]. Since the checkerboard pattern does not deform in the top-view image, we assume that the length and width of the checkerboard pattern are both 2 meters, and let the distance between each two pixels in the top-view image represent 1 millimeter. Thus, we obtain the coordinates of the four corner points of the checkerboard pattern in the top-view image [(Xtl, Ytl), (Xtl+2000, Ytl), (Xtl, Ytl+2000), (Xtl+2000, Ytl+2000)]. Among them, (Xtl, Ytl) represents the pixel coordinates of the upper left corner of the checkerboard pattern in the top-view image. It can be set as needed. Generally, the checkerboard pattern is placed in the lower center of the top-view image and set to Xtl = IMGW / 2 - 1000 and Ytl = IMGH - 2000. Where IMGW is the width of the top-view image; IMGH is the height of the top-view image. Alternatively, the projection mapping matrix Pmat can be quickly obtained using the `getPerspectiveTransform` method in OpenCV: input the coordinates of the checkerboard corner points in the original road surface image and the checkerboard corner points in the top-view image, and it can be calculated quickly. The inverse matrix of the projection mapping matrix Pmat is then obtained as the inverse transformation matrix InvPmat from the top-view image to the original road surface image. The inverse transformation matrix InvPmat can also be calculated using the `np.linalg.inv` method in NumPy. The inverse transformation matrix InvPmat is mainly used later to transform the coordinates under the top-view image to the coordinates under the original road surface image. Like the projection mapping matrix Pmat, the inverse transformation matrix InvPmat is a 3×3 matrix. Using the `cv2.warpPerspective` method, inputting the projection mapping matrix Pmat and the original road surface image, a top-view image of the original road surface image can be obtained. Figure 3 As shown, in the top-view image, the distance between every two pixels represents 1 millimeter. Therefore, by converting the bounding box of the original road surface image into the top-view image, the length, width, height, area, and other dimensional information of the bounding box in the original road surface image can be obtained based on the pixel information in the top-view image.

[0064] When aligning disease lesions, this invention uses the top-view image of the area extending 2 meters outward from the center point of the lesion, rotated to an azimuth angle of 0 degrees, as the aligned image. Since the size of the lesion frame varies depending on the time and location of the image, to reduce the impact of different sized frames on the lesion features, images within a 2-meter radius extending outward from the center point of the lesion are uniformly used as the original lesion image. There are two methods for obtaining the top-view image of the original lesion: Method one is to transform the original lesion image into a top-view image, and then crop the lesion area from the top-view image; Method two is to calculate the mapping transformation matrix from the original lesion image to the top-view image, and only perform a top-view transformation on the lesion area to obtain the top-view image of the lesion. Since directly converting the original lesion image to the top-view image involves a large computational load and is slow, this invention uses a method that first calculates the transformation matrix of the lesion area and then converts the lesion area into a top-view image, as shown in steps 3 to 5.

[0065] Step 3: Calculate the coordinates of the disease center point based on the disease information and determine the original disease image.

[0066] Based on the disease coordinates box(x1, y1, x2, y2) on the original pavement image obtained in step 2, calculate the coordinates point_center(x_center, y_center) of the disease center point on the original pavement image. Then, calculate the coordinates bev_point_center(bev_x_center, bev_y_center) of the disease center point on the top-view image. This can be quickly obtained using the cv2.perspectiveTransform method in OpenCV, by inputting the disease center point coordinates point_center and the projection mapping matrix Pmat. On the top-view image, determine a square region bev_box(bev_x_center-1000, bev_y_center-1000, bev_x_center+1000, bev_y_center+1000) with a side length of 2 meters, centered at the center point of the road defect. Calculate the region mbox(xm1, ym1, xm2, ym2) of this square region on the original road surface image using the inverse transform matrix InvPmat and the square region bev_box. This can also be achieved using the cv2.perspectiveTransform method in OpenCV, inputting the square region bev_box(bev_x_center-1000, bev_y_center-1000, bev_x_center+1000, bev_y_center+1000) and the inverse transform matrix InvPmat. Crop the region mbox from the original road surface image to obtain the original defect image boxImage.

[0067] Step 4: Determine the top view image of the disease based on the original disease image.

[0068] First, the top-view transformation matrix of the cropped original pavement image (boxImage) needs to be calculated. Based on the coordinates of the top-left, top-right, bottom-left, and bottom-right corners of the original pavement image (boxImage) and the top-left, top-right, bottom-left, and bottom-right corners of the transformed top-view image (boxImage), the projection mapping matrix newPmat from the original pavement image to the top-view image in OpenCV is calculated using the cv2.getPerspectiveTransform method. This is the top-view transformation matrix, specifically calculated using the cropped original pavement image (boxImage). The coordinates of the top-left corner P1(0,0), top-right corner P2(xm2-xm1,0), bottom-left corner P3(0,ym2-ym1), and bottom-right corner P4(xm2-xm1,ym2-ym1) of the original image `boxImage` are calculated, along with the coordinates of the top-left corner TP1(0,0), top-right corner TP2(2000,0), bottom-left corner TP3(0,2000), and bottom-right corner TP4(2000,2000) of the transformed top-view image. Then, based on the projection mapping matrix `newPmat` and the original disease image `boxImage`, the top-view image `boxImagebev` is obtained using the `cv2.warpPerspective` method.

[0069] Step 5: Rotate the azimuth angle of the top view image of the road surface defects to obtain the aligned image of the road surface defects.

[0070] Based on the azimuth angle (Azimuth) in the GPS positioning information corresponding to the disease top-view image (boxImagebev), the disease top-view image (boxImagebev) is rotated to a uniform angle, generally rotating to the top view with an azimuth angle of 0 degrees. The rotation matrix Tmat is obtained using the `cv2.getRotationMatrix2D(image_center, 0-Azimuth)` method in OpenCV; where `image_center` is the coordinate of the center point of the disease top-view image (boxImagebev), i.e., (1000, 1000). Based on the disease top-view image (boxImagebev) and the rotation matrix Tmat, the disease top-view image with an azimuth angle of 0 degrees is calculated using the `cv2.warpAffine` method in OpenCV and used as the disease alignment image (Damage_Align_Image).

[0071] Step 6: Match diseases according to the disease alignment image, and trace the spatiotemporal path of diseases based on the disease matching results.

[0072] Step 6 is divided into two parts: disease feature extraction and disease matching. First, disease features are extracted based on the disease-aligned image Damage_Align_Image. Specifically, the Damage_Align_Image is input into a disease feature extraction algorithm (generally a trained deep learning neural network algorithm) to extract disease features, thus obtaining the aligned disease feature Vector. The disease ID and the disease feature Vector are then stored in the disease feature library DamageVectorLib.

[0073] Subsequently, disease matching is performed based on the disease-aligned images. Diseases that are the same as the current disease in the detected historical data are grouped together and represented by the same disease attribution ID. That is, one disease ID includes a corresponding disease image, a disease feature, and a disease attribution ID. However, one disease attribution ID can correspond to multiple disease IDs. Specifically: based on the current disease's GPS location information, the disease database (DamageDatabase) is queried to search for nearby historical diseases of the same category. If no similar disease exists in nearby historical data, a new disease attribution ID is generated for the current disease. If similar diseases exist in nearby historical data, the corresponding disease features are extracted from the disease feature database (DamageVectorLib) based on the different disease IDs. Calculate the similarity between the extracted disease features and the current disease features. If the maximum similarity among the various similarities is less than the similarity threshold, a new disease attribution ID is generated for the current disease. If the maximum similarity is greater than or equal to the similarity threshold, it is determined that the current disease and the disease corresponding to the maximum similarity belong to the same disease. The disease attribution ID of the disease corresponding to the maximum similarity is used as the disease attribution ID of the current disease, and the disease attribution ID is stored in the disease database DamageDatabase.

[0074] After matching the disease-aligned images, spatiotemporal tracing of the disease is performed based on the matching results. Specifically, the disease database (DamageDatabase) is queried for diseases with the same disease attribution ID. This allows us to find images of the same disease taken at different times and locations. Spatiotemporal tracing of the disease is then performed on images of the same disease taken at different times and locations to analyze the causes of the disease. Based on the causes of the disease, more effective maintenance plans are customized, and more reliable data support is provided for subsequent road construction.

[0075] The spatiotemporal tracing method for pavement defects based on pavement defect alignment provided in this invention needs to be used in conjunction with a high-frequency intelligent pavement defect inspection system, which generally includes an edge acquisition and control module and a cloud-based intelligent analysis module. The edge acquisition and control module is as follows: Figure 4As shown, the patrol vehicle is equipped with a vehicle-mounted camera and a central industrial control computer to collect raw road surface images. The central industrial control computer is powered by the vehicle's battery, and the vehicle-mounted camera is connected to it. The central industrial control computer sends image acquisition commands and pushes image information, acquisition time, and GPS location information to a cloud server. After receiving the data, the cloud server... Figure 1 The process shown completes disease detection and disease attribution.

[0076] The implementation process can be summarized as follows:

[0077] 1) Establish an intelligent analysis module on the cloud server, including intelligent road surface defect identification algorithm, defect feature extraction algorithm and feature matching algorithm.

[0078] 2) Install edge acquisition modules on the patrol lanes and calibrate them.

[0079] 3) Start the device to collect images and GPS positioning information, and push them to the cloud server in real time.

[0080] 4) After receiving the data, the cloud server, according to... Figure 1 The steps shown complete the detection and alignment of road defects. Using the image information after defect alignment and the defect feature extraction algorithm, defect features are extracted. Defect IDs are matched according to the defect features, and the results are written into the database for query and display.

[0081] In summary, the pavement defect alignment-based spatiotemporal tracing method provided by this invention can be deployed on a cloud server and combined with intelligent inspection equipment to achieve rapid, accurate, and real-time defect matching, supporting spatiotemporal defect tracing and providing researchers with clearer data to analyze the causes of defects and formulate maintenance plans. Compared with existing spatiotemporal defect tracing methods, the pavement defect alignment-based spatiotemporal tracing method of this invention has the following advantages:

[0082] (1) Compared with the currently popular method of directly extracting disease features from the disease area on the original road surface image, the present invention extracts disease features based on the disease-aligned image, which effectively reduces the impact caused by disease imaging at different locations in the image and disease boxes of different sizes, thereby effectively improving the accuracy of disease matching.

[0083] (2) The disease alignment concept introduced in this invention can align disease images taken at different locations and angles to the same viewpoint, effectively improving the similarity of the same disease imaging, reducing the difficulty of disease feature matching, and thus improving the accuracy of disease feature matching.

[0084] (3) The present invention proposes to obtain the original disease image based on a square area 2 meters outside the center point of the disease. For large diseases, it can reduce the influence of different sizes of boxes obtained from different angles on disease matching. For small diseases, it can introduce information around the disease for disease feature matching, which effectively improves the accuracy of disease matching.

[0085] (4) Since the computational workload of directly converting the original road surface image into a top view image is large, the present invention does not require top view transformation of the original road surface image to obtain a top view image of a square area 2 meters outside the center of the disease, which has the advantages of small computational workload and fast calculation speed.

[0086] In addition, the present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of a spatiotemporal tracing method for road surface defects based on road defect alignment.

[0087] Furthermore, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a method for spatiotemporal tracing of road surface defects based on road defect alignment.

[0088] Furthermore, the present invention also provides a computer device, including a processor, a memory, an input / output interface (I / O), a communication interface, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the aforementioned method for spatiotemporal tracing of road surface defects based on alignment.

[0089] The processor, memory, and input / output interface are connected via a system bus, and the communication interface is also connected to the system bus via the input / output interface. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores pending transactions. The input / output interface allows the processor to exchange information with external devices. The communication interface allows communication with external terminals via a network connection. When the computer program is executed by the processor, it implements the spatiotemporal tracing method for pavement distress alignment.

[0090] It should be noted that the object information (including but not limited to object device information, object personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this invention are all information and data authorized by the object or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0091] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided by this invention can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided by this invention may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided by this invention may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0092] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0093] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for spatiotemporal tracing of road surface defects based on road defect alignment, characterized in that, include: The original road surface image and image information of the road to be detected are acquired and stored in the original road surface image information database; the image information includes image name, acquisition time and GPS positioning information; the GPS positioning information includes longitude, latitude and azimuth. The original pavement image is subjected to pavement defect identification to obtain defect information, and the defect information, along with the corresponding original pavement image and image information, is stored in the defect database; the defect information includes defect coordinates, defect name, and defect ID; The coordinate information of the corner points of the checkerboard with known dimensions on the original road surface image and the coordinate information of the corner points of the checkerboard with known dimensions on the top view image are collected. The projection mapping matrix Pmat from the original road surface image to the top view image is obtained by using the getPerspectiveTransform method in OpenCV. The inverse of the projection mapping matrix Pmat from the original road surface image to the top view image is calculated using the np.linalg.inv method in NumPy, thus obtaining the mapping matrix InvPmat from the top view image to the original image; or, based on the coordinates of the corner points of the checkerboard grid of known size on the top view image and the coordinates of the corner points of the checkerboard grid of known size on the original road surface image, the mapping matrix InvPma from the top view image to the original image is obtained using the getPerspectiveTransform method in OpenCV. The calculation of the disease center point coordinates and determination of the original disease image based on disease information includes: calculating the disease center point coordinates (point_center) based on the disease coordinates (box) in the disease information; obtaining the disease center point coordinates (bev_point_center) on the top view image using the cv2.perspectiveTransform method in OpenCV based on the disease center point coordinates (point_center) and the projection mapping matrix Pmat; determining a square region (bev_box) with a side length of 2 meters centered on the disease center point on the top view image; obtaining the region (mbox) of the square region (bev_box) on the original road surface image using the cv2.perspectiveTransform method in OpenCV based on the square region (bev_box) and the inverse transformation matrix InvPmat; and cropping the region (mbox) from the original road surface image to obtain the original disease image (boxImage). Determine the top-view image of the disease based on the original disease image; Rotate the top view image of the road surface defects to obtain the aligned image of the road surface defects; Disease matching is performed based on disease-aligned images, and spatiotemporal tracing of diseases is performed based on the disease matching results.

2. The method for spatiotemporal tracing of road surface defects based on road defect alignment according to claim 1, characterized in that, The process of determining the top-view image of the lesion based on the original lesion image specifically includes: Based on the coordinates of the top left, top right, bottom left and bottom right corners of the original disease image boxImage and the coordinates of the top left, top right, bottom left and bottom right corners of the top view image after the original disease image boxImage is transformed, the projection mapping matrix newPmat of the original road surface image to the top view image of the disease area is calculated using the cv2.getPerspectiveTransform method in OpenCV. Based on the projection mapping matrix newPmat and the original disease image boxImage, the top view image boxImagebev of the disease is obtained using the cv2.warpPerspective method.

3. The spatiotemporal tracing method for road surface defects based on defect alignment according to claim 2, characterized in that, The azimuth angle of the rotated top view image of the road surface defects is used to obtain the aligned image of the road surface defects, specifically including: Based on the azimuth angle Azimuth in the GPS positioning information corresponding to the disease top view image boxImagebev, the rotation matrix Tmat is obtained using the cv2.getRotationMatrix2D(image_center, 0-Azimuth) method in OpenCV; where image_center is the coordinate of the center point of the disease top view image boxImagebev; Based on the top-view image of the lesion boxImagebev and the rotation matrix Tmat, the top-view image of the lesion with an azimuth angle of 0 degrees is calculated using the cv2.warpAffine method in OpenCV and used as the lesion alignment image Damage_Align_Image.

4. The spatiotemporal tracing method for road surface defects based on defect alignment according to claim 3, characterized in that, The process of matching diseases based on disease-aligned images specifically includes: A disease feature extraction algorithm is used to extract disease features from the disease-aligned image Damage_Align_Image to obtain a disease feature vector. Store the disease ID and disease feature Vector into the disease feature library DamageVectorLib; Based on the current GPS location information of the disease, query the DamageDatabase to search for the presence of the same type of disease in nearby historical diseases; If there are no diseases of the same type in the nearby historical diseases, a new disease attribution ID will be generated for the current disease. If there are diseases of the same type in the nearby historical diseases, the corresponding disease features are extracted from the disease feature library DamageVectorLib according to different disease IDs. Calculate the similarity between the extracted disease features and the current disease features. If the maximum similarity among the various similarities is less than the similarity threshold, a new disease attribution ID is generated for the current disease. If the maximum similarity is greater than or equal to the similarity threshold, it is determined that the current disease and the disease corresponding to the maximum similarity belong to the same disease. The disease attribution ID of the disease corresponding to the maximum similarity is used as the disease attribution ID of the current disease, and the disease attribution ID is stored in the disease database DamageDatabase.

5. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the spatiotemporal tracing method for road surface defects based on alignment as described in any one of claims 1-4.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the spatiotemporal tracing method for road surface defects based on alignment as described in any one of claims 1-4.

7. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the spatiotemporal tracing method for pavement distress alignment based on any one of claims 1-4.

Images