Large-scale pavement performance degradation tracking method based on logistics vehicle data

Abstract

The invention relates to a large-scale pavement performance degradation tracking method based on logistics vehicle data, and the method comprises the following steps: carrying out the precise recognition of a plurality of types of pavement diseases in an image based on the image data of an automobile data recorder of a logistics vehicle and recorded GPS position data; evaluating the minimum detection frequency of the road section by adopting a Bernoulli function; pavement performance degradation automatic evaluation is realized based on a three-stage disease matching algorithm, in the first stage, GPS position data is utilized to perform preliminary screening on pavement disease recognition result pictures, and the minimum detection frequency is utilized to perform picture sampling to screen out pavement pictures with diseases at the same position; in the second stage, it is ensured that the images belong to the same scene or similar view angles through background matching, and in the third stage, an adjacent local area matching algorithm is used for matching specific pavement diseases, and pavement performance degradation automatic evaluation is achieved. Compared with the prior art, large-scale pavement damage conditions can be updated at high frequency, and rapid detection and matching of pavement diseases are realized.

Description

Technical Field

[0001] The present invention relates to the field of road infrastructure performance detection and evaluation, and in particular to a large-scale road performance degradation tracking method based on logistics vehicle data. Background Art

[0002] In recent years, the field of road infrastructure has developed rapidly, and the high stock of in-service roads has led to unprecedented maintenance pressure. In order to ensure the effective implementation of daily road maintenance operations and preventive maintenance measures, scientific data support is indispensable. Therefore, rapid and accurate monitoring of pavement performance and in-depth analysis of performance degradation trends are core needs in research and practice in the field of road maintenance. This ability to perceive and analyze performance trends not only helps to identify current road conditions, but also predict future performance changes, thereby providing data-driven support for maintenance decisions. Not only that, the development tracking and deterioration analysis of pavement diseases are also crucial to the optimization of road facility distribution, traffic load regulation, pavement performance prediction, and the rational design of road structure materials. Therefore, real-time detection and evaluation of road infrastructure performance are required.

[0003] Existing pavement defect detection methods often only focus on the accuracy and efficiency of detection, do not consider historical test results, lack the ability to track detected defects, and lack the detection of road infrastructure performance. Summary of the invention

[0004] The purpose of the present invention is to provide a large-scale pavement performance degradation tracking method based on logistics vehicle data, which inputs road images and GPS location data, determines the type of disease through a computer vision algorithm, deduplicates the diseases and analyzes the deterioration through a disease matching method, outputs the disease type, disease number and disease degradation condition, and realizes high-precision large-scale pavement performance degradation tracking.

[0005] The purpose of the present invention can be achieved by the following technical solutions:

[0006] A large-scale road performance degradation tracking method based on logistics vehicle data includes the following steps:

[0007] S1, based on the dashcam image data and recorded GPS location data of logistics vehicles, uses artificial intelligence image detection technology to analyze and extract the collected images, combines the two-layer routing attention mechanism and the P2 small target layer, and optimizes the bounding box regression loss function to accurately identify multiple types of road surface diseases in the image;

[0008] S2, based on the detection accuracy of the pavement damage identification result in step S1, the Bernoulli function is used to evaluate the minimum detection frequency of the road section to ensure the adequacy and representativeness of the pavement damage data;

[0009] S3, based on the three-stage disease matching algorithm, realizes automatic evaluation of pavement performance degradation, effectively eliminates repeated detection and continuously tracks the deterioration of the disease. The first stage of the three-stage disease matching algorithm uses GPS location data to preliminarily screen the pavement disease identification result pictures of step S1, and uses the minimum detection frequency determined in step S2 to sample pictures, and screen out the road surface pictures with diseases at the same location. The second stage ensures that the images screened in the first stage belong to the same scene or similar perspective through background matching to reduce mismatching. The third stage uses the adjacent local area matching algorithm to match the specific pavement diseases in the pictures processed in the second stage, ensuring the accuracy and continuity of the disease data, and realizing automatic evaluation of pavement performance degradation.

[0010] The types of road surface diseases include cracks, cracks, potholes, crack repairs, crack repairs, and pothole repairs.

[0011] The artificial intelligence image detection technology adopts the real-time detection model YOLOv8 in the field of computer vision, and its training process is as follows:

[0012] S11, establish a pavement disease damage data set based on the disease history data;

[0013] S12, preprocessing the images in the pavement disease damage dataset to improve the image quality and adapt to the input requirements of the YOLOv8 model;

[0014] S13, based on the YOLOv8 model, combines the two-layer routing attention mechanism and the P2 small target layer to focus on road surface damage and municipal facility damage of different scales to improve the accuracy of disease identification;

[0015] S14, post-processing the output results of the YOLOv8 model, including non-maximum suppression (NMS) and bounding box regression loss function optimization, to reduce positioning errors and improve the recognition accuracy of road surface defects;

[0016] S15, use the loss function to optimize the convolution kernel of the YOLOv8 model and prune the trained model to reduce the number of model parameters so that it can be quickly deployed on edge devices.

[0017] The preprocessing includes image denoising, resizing and normalization.

[0018] Based on the YOLOv8 model, the two-layer routing attention mechanism and the P2 small target layer are specifically:

[0019] The improved SFBPPF module is introduced into the last layer of the backbone network of the YOLOv8 model. The SFBPPF module consists of a CBS module, an RFB module, and a maximum pooling layer. The input data of the SFBPPF module is processed by the first CBS module, and then passes through the RFB module and three maximum pooling layers in sequence. The outputs of the three maximum pooling layers and the output of the first CBS module are spliced ​​and input into the second CBS module. The output of the CBS module is the output of the SFBPPF module.

[0020] A P2 small target layer is introduced in the detection head of the YOLOv8 model. The P2 small target layer includes an upsampling layer, a splicing operation, a C2fBi module and a CBS module. The C2fBi module introduces a double-layer routing attention mechanism and a depthwise separable convolution on the basis of the C2f module. The input data is processed by the depthwise separable convolution and then added to the input data to obtain a first feature. The first feature is sequentially processed by layer normalization and the output after the double-layer routing attention is added to the first feature to obtain a second feature. The second feature is sequentially processed by layer normalization and the output after the MLP layer is added to the second feature to obtain the output of the C2fBi module.

[0021] The calculation formula of the optimized bounding box regression loss function is as follows:

[0022] L CDIoUv1 =L IoU +R CDIoU

[0023]

[0024] Among them, L CDIoUv1 is the optimized bounding box regression loss function, L IoU is the bounding box regression loss function, R CDIoU is a penalty function that measures the regression error between the predicted box and the true box. represents the left distance vector from the anchor box to the target box, represents the right distance vector from the anchor box to the target box, is the diagonal vector of the minimum bounding box.

[0025] The calculation formula of the Bernoulli function is:

[0026]

[0027] P(x≥1)=1-P(x=0)=1-(1-p c ) n ≥P TH

[0028] Where P(x=k) is the probability of an event occurring k times, n represents the minimum detection frequency, and pc is the minimum detection accuracy of various diseases in step S1, is the combination number formula, which means the number of combinations of k elements selected from n elements, P TH is the preset detection accuracy threshold.

[0029] The use of GPS location data to preliminarily screen the pavement disease identification result image of step S1 is specifically as follows:

[0030] Using the GPS location data recorded by logistics vehicles, different clustering radii are selected in different road scenes, and the images of adjacent points are screened through clustering algorithms. Direct clustering of GPS may cause the images of opposite lanes to be matched together or the images of intersecting lanes to be matched together. Further secondary clustering is performed using the azimuth data in the GPS location data, where the azimuth value ranges from 0 to 360, where 0 represents north and 90 represents east.

[0031] The specific steps of the background matching include:

[0032] The images in the same cluster are grouped in pairs, and the SuperPoint algorithm is used to detect key points and extract descriptors for the pair of images to be matched. The key points and descriptors of the extracted pair of images are sent to the SuperGlue algorithm. Through the graph attention mechanism of SuperGlue, the similarities between the key points are learned, and the extracted key points and descriptors are matched one by one to establish a reliable matching correspondence. Based on the matching results, the similarity between the two images is evaluated, and the sinkhorn algorithm is used to find the optimal pairing to achieve background matching.

[0033] The specific steps of the adjacent local area matching algorithm include:

[0034] Determine whether there is a corresponding matching quadrilateral for a pair of images to be matched, the pair of images comprising a first image and a second image;

[0035] If there is a matching quadrilateral, for the pavement disease in the first image and the second image, extract their bounding box coordinates respectively, and determine the set of matching pixel points MA in the first image and the second image; select four pixel points closest to the bounding box of the pavement disease in the first image in the set MA, determine the matching pair of the four pixel points in the second image, and form a matching quadrilateral; determine whether the bounding box of the pavement disease in the second image is located in the corresponding matching quadrilateral, and if it is located in the matching quadrilateral, it is determined that the pavement disease in the first image and the second image are the same pavement disease;

[0036] If there is no matching quadrilateral, use UFLD to detect the lane line edges in the first image and the second image respectively, and determine the lane line intersection point I respectively.a ,I b and the intersection point O of the lane line and the edge of the image a , O b ; In the first image, draw auxiliary lines through O a , so that it is in F a Intersecting with another lane line, ensure that I a F a =I a O a , to estimate the lane line direction, in the second image, draw an auxiliary line through O b , so that it is in F b Intersecting with another lane line, ensure that I b F b =I b O b , to estimate the lane line direction; select two matching points M on the road surface disease of the first image a and N a , through M a and N a The parallel line of F a O a Parallel, and determine their projection point M on the lane centerline ta and N ta ; Select two matching points M on the road surface damage of the second image b and N b , through M b and N b The parallel line of F b O b Parallel, and determine their projection point M on the lane centerline tb and N tb ; Based on the projection point M ta With N ta 、M tb With N tb The distance between ta N ta )、h(M tb N tb ) Calculate the projection ratio of the two images α = h(M ta N ta ) / h(M tb N tb );Determine the midpoint D of the pavement defect boundary box in the first image and the second image a and D b , the midpoint D a With N ta The distance h(D a N ta ) and midpoint D b With Ntb The distance h(D b N tb ) is within the preset upper and lower limits of the projection ratio, it is determined that the pavement defects in the first image and the second image are the same pavement defect.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] The present invention relates to a large-scale pavement performance degradation tracking method based on logistics vehicle data. The method uses the driving recorder of the logistics vehicle to collect street view images, and realizes accurate identification of multiple types of pavement diseases by constructing a deep convolutional neural network and combining a double-layer routing attention mechanism. At the same time, the bounding box regression loss function is optimized to improve the recognition accuracy of common pavement defects. In addition, the Bernoulli function is used to evaluate the minimum detection frequency of the road section, so that image sampling can be performed reasonably during the pavement performance degradation tracking process. Finally, a three-stage disease matching algorithm is designed, which combines the attention graph neural network and the adjacent local area matching algorithm to effectively eliminate repeated detection and continuously track the deterioration of the disease, thereby realizing pavement performance degradation tracking. Compared with the existing pavement performance degradation tracking method, the present invention can update large-scale pavement damage at a high frequency, realize rapid detection and matching of pavement diseases, and can analyze the road disease degradation. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 is a flow chart of the method of the present invention;

[0040] Figure 2 This is the architecture diagram of the YOLOv8 model after optimization of the present invention;

[0041] Figure 3 Schematic diagram of the bounding box regression loss function of the present invention;

[0042] Figure 4 This is a three-stage disease matching algorithm architecture diagram of the present invention;

[0043] Figure 5 It is a background matching algorithm architecture diagram of the present invention;

[0044] Figure 6 This is a diagram of the adjacent local area matching algorithm architecture of the present invention. DETAILED DESCRIPTION

[0045] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0046] Example 1

[0047] like Figure 1 As shown, this embodiment provides a large-scale road performance degradation tracking method based on logistics vehicle data, including the following steps:

[0048] S1, based on the dashcam image data and recorded GPS location data of logistics vehicles, uses artificial intelligence image detection technology to analyze and extract the collected images, combines the two-layer routing attention mechanism and the P2 small target layer, and optimizes the bounding box regression loss function to accurately identify multiple types of road surface diseases in the image.

[0049] In order to achieve high-frequency collection of road defects, this embodiment installs an industrial camera in front of the logistics vehicle. When the logistics vehicle is driving, it will automatically collect street view images on both sides of the road and record the geographical location, shooting time and other information of the image. The collected street view images may have problems such as blur, backlighting, and overexposure, which will affect the accuracy of subsequent defect identification. Therefore, it is necessary to screen the images. The specific screening process is as follows:

[0050] (1) Improve image quality through image preprocessing techniques such as dehazing and contrast enhancement.

[0051] (2) Remove images taken in rainy days or other situations where the camera lens is not functioning.

[0052] (3) Sample the images according to vehicle speed and remove duplicate images and images with high similarity.

[0053] To evaluate the deterioration of pavement diseases is to analyze the temporal changes and spatial distribution of pavement diseases. For example, on a temporal scale, cracks will appear on paved asphalt roads over time, and sudden potholes may appear on the road surface after a natural disaster; on a spatial scale, pavement roads with heavy loads are often more prone to pavement diseases. To this end, it is first necessary to automatically identify pavement diseases in images. That is, to identify and locate specific targets in the image, assign a bounding box and a corresponding category label to each target. This process can usually be performed using target detection. Target detection is a core task in the field of computer vision. It aims to locate and identify targets of interest in an image, assigning a bounding box and a corresponding category label to each target. Through deep learning neural networks, computers can autonomously learn, analyze and extract key features of targets from massive amounts of labeled data. Based on deep learning target detection algorithms, computers can automatically search for targets in images, accurately locate and identify targets, and thus provide effective technical support for various application scenarios. The artificial intelligence image detection technology of this embodiment adopts the real-time detection model YOLOv8 in the field of computer vision, with a structure such as Figure 2 As shown, the training process is as follows:

[0054] S11, establishing a pavement disease damage data set based on the disease history data.

[0055] S12, performing preprocessing such as image denoising, size adjustment and normalization on the images in the pavement disease damage dataset.

[0056] Specifically, a large number of road images collected above are used, and representative images containing road diseases are selected from them, and data enhancement including mirroring, cropping, adding noise and brightness transformation are performed on them. Then, the open source image annotation tool Labelme is used to annotate various types of diseases in each road landscape image after data enhancement. The annotated elements include cracks, cracks, potholes, crack repairs, crack repairs, pothole repairs, etc., so as to establish a road disease damage dataset.

[0057] S13, based on the YOLOv8 model, combines the two-layer routing attention mechanism and the P2 small target layer to focus on road surface diseases and municipal facility damage of different scales.

[0058] like Figure 2 As shown in the figure, the improved SFBPPF module is introduced in the last layer of the backbone network of the YOLOv8 model. This module can better extract the features of small targets by introducing more scale information. The SFBPPF module consists of a CBS module, an RFB module, and a maximum pooling layer. The input data of the SFBPPF module is processed by the first CBS module, and then passes through the RFB module and three maximum pooling layers in turn. The outputs of the three maximum pooling layers and the output of the first CBS module are spliced ​​and input into the second CBS module. The output of the CBS module is the output of the SFBPPF module.

[0059] A P2 small target layer is introduced in the detection head of the YOLOv8 model. The P2 small target layer includes an upsampling layer, a splicing operation, a C2fBi module and a CBS module. The C2fBi module introduces a double-layer routing attention mechanism and a depthwise separable convolution on the basis of the C2f module. The input data is processed by the depthwise separable convolution and then added to the input data to obtain a first feature. The first feature is sequentially processed by layer normalization and the output after the double-layer routing attention is added to the first feature to obtain a second feature. The second feature is sequentially processed by layer normalization and the output after the MLP layer is added to the second feature to obtain the output of the C2fBi module.

[0060] The dual-layer routing attention mechanism is adopted and integrated with the P2 small target detection layer. The dual-layer routing attention mechanism can effectively improve the computational efficiency of the network and enhance the network's detection ability for small targets. The P2 small target detection layer is a layer specifically used to detect small targets. Combining it with the dual-layer routing attention mechanism can further improve the detection effect of small targets. The depthwise separable convolution is a lightweight convolution operation that can effectively reduce the number of network parameters and the amount of computation. The C2f module is a module used to enhance the feature transfer capability of the network. Combining it with the depthwise separable convolution can reduce the amount of network computation while maintaining the detection effect of the network.

[0061] S14, post-processing the output results of the YOLOv8 model, including non-maximum suppression and bounding box regression loss function optimization.

[0062] The present invention optimizes the evaluation index IoU. Since cracks usually have a small actual area, their annotation quality in the image is often low, which causes the traditional bounding box regression loss function to produce unstable gradients during the training process, thereby affecting the generalization ability of the model. Low-quality annotation of cracks is inevitable, and geometric factors will significantly increase the penalty for low-quality examples. In order to effectively weaken the penalty of geometric factors and improve the generalization ability of the model, the present invention constructs a multi-cost fused CDIoU. Figure 3 As shown, the calculation formula of the optimized bounding box regression loss function is as follows:

[0063] L CDIoUv1 =L IoU +R CDIoU

[0064]

[0065] Among them, L CDIoUv1 is the optimized bounding box regression loss function, L IoU is the bounding box regression loss function, R CDIoU is a penalty function that measures the regression error between the predicted box and the true box. represents the left distance vector from the anchor box to the target box, represents the right distance vector from the anchor box to the target box, is the diagonal vector of the minimum bounding box.

[0066] S15, use the loss function to optimize the convolution kernel of the YOLOv8 model and prune the trained model.

[0067] S2, based on the detection accuracy of the road damage identification result in step S1, the Bernoulli function is used to evaluate the minimum detection frequency of the road section.

[0068] Although computer vision algorithms can effectively detect pavement defects, the accuracy of a single inspection is limited due to the complex environment. It is necessary to combine the results of multiple inspections to improve the detection rate. Assuming that the successful detection of each pavement defect is independent, which can be regarded as a Bernoulli trial, the number of successful disease detections follows a binomial distribution. The binomial distribution describes the probability of a successful or failed result in multiple repeated inspections. For each pavement defect, its detection frequency is given by the average accuracy p of the computer vision algorithm. c Estimate, where c represents the disease type. Since x~B(N,Pc), the detection rate of k inspections is as follows:

[0069]

[0070] To ensure that more than 90% of pavement defects can be detected, the minimum detection frequency is determined according to the following formula:

[0071] P(x≥1)=1-P(x=0)=1-(1-p c ) n ≥P TH

[0072] Where P(x=k) is the probability of an event occurring k times, n represents the minimum detection frequency, and p c is the minimum detection accuracy of various diseases in step S1, is the combination number formula, which means the number of combinations of k elements selected from n elements, P TH is a preset detection accuracy threshold, which is set to 0.90 in this embodiment.

[0073] S3, automatic assessment of pavement performance degradation based on a three-stage disease matching algorithm.

[0074] The architecture of the three-stage disease matching algorithm is as follows Figure 4 As shown, it includes the following three stages:

[0075] Phase 1: Use GPS location data to preliminarily screen the pavement disease identification result images of step S1, and use the minimum detection frequency determined in step S2 to sample the images to screen out the images of the pavement with diseases at the same location.

[0076] Taking into account the obstruction of urban buildings and the positioning error of GPS on multi-lane roads, the GPS location data recorded by the logistics vehicles is first used to select 5 meters, 10 meters, and 15 meters as the clustering radius in different road scenes. The images of adjacent points are screened by the clustering algorithm, and secondary clustering is performed according to the azimuth in the GPS location data, where the azimuth ranges from 0 to 360, where 0 represents north and 90 represents east. This embodiment adopts an improved K-means algorithm, which adds a filtering mechanism for outlier data in the clustering process. This improved algorithm can better handle outliers in GPS data, thereby improving the accuracy and efficiency of clustering. By filtering out outliers, the algorithm can more accurately determine the cluster center, so that the clustering results are more in line with the actual situation.

[0077] Stage 2: Background matching is used to ensure that the images screened in the first stage belong to the same scene or similar perspectives.

[0078] The invention uses the deep learning method of SuperPoint+SuperGlue to further screen the road images in the same cluster. SuperPoint is a general interest point detection and description algorithm that can generate key points with good characteristics and describe these key points for matching between different images. SuperPoint is trained through a deep learning network and can maintain stable key point detection performance under various scenarios and transformations. SuperGlue is an efficient feature matching algorithm that has achieved advanced results in feature matching tasks in outdoor environments. It is an end-to-end model composed of CNN feature extraction and graph neural network (GNN) feature matching. It learns the geometric transformation and regularity priors of the 3D world through self-attention and cross-attention mechanisms. End-to-end training enables it to surpass traditional hand-designed functions such as SIFT. Its architecture is as follows Figure 5 As shown in Figure 2, SuperGlue can establish reliable matching correspondences between different images by learning the relationship between feature points.

[0079] Specifically, the images in the same cluster are grouped in pairs, and the SuperPoint algorithm is used to detect key points and extract descriptors for the pair of images to be matched. The key points and descriptors of the extracted pair of images are sent to the SuperGlue algorithm. Through the graph attention mechanism of SuperGlue, the similarities between the key points are learned, and the extracted key points and descriptors are matched one by one to establish a reliable matching correspondence. Based on the matching results, the similarity between the two images is evaluated, and the sinkhorn algorithm is used to find the optimal pairing to achieve background matching.

[0080] Phase 3: Use the adjacent local area matching algorithm to match specific pavement defects in the images processed in Phase 2 to achieve automatic assessment of pavement performance degradation.

[0081] The SuperGlue network provides matching pixels between the two images, while the image recognition algorithm draws the bounding box of each defect. If the pixels in the two bounding boxes are matched, then there are repeated road defects in the two images. However, due to different shooting angles, unclear features or abnormal weather, the same defects in the two images may not share matching features. Repaired defects and potholes are easy to match because their features are more obvious, but the original defects are difficult to match directly through Superglue because their features are not obvious to the entire image. Therefore, this embodiment designs an adjacent local area matching algorithm by extracting local areas around the defects and then using a feature matching algorithm to compare the similarities of these local areas. By calculating the relative position and direction of the defects in the local area, it is determined whether the two defects are the same. This method can accurately match defects in two images, even if they are of different shapes and sizes. Specifically, if Figure 6 As shown, for a pair of images to be matched (including the first image A and the second image B), first determine whether there is a corresponding matching quadrilateral:

[0082] If there is a matching quadrilateral, extract the bounding box coordinates of the road surface damage in the first image and the second image respectively. The bounding box coordinates of image A are (ax1, ay1), (ax2, ay2), (ax3, ay3), (ax4, ay4), and the bounding box coordinates of image B are (bx1, by1), (bx2, by2), (bx3, by3), (bx4, by4). Determine the set MA of matching pixels in the first image A and the second image B; select the four pixels (nax1, nay1), (nax2, nay2), (nax3, nay3), (nax4, nay4) ∈ MA that are closest to the bounding box of the road surface damage in the first image A in the set MA, and determine the matching pairs (nbx1, nby1), (nbx2, nby2), (nbx3, nby3), (nbx4, nby4) of the four pixels in the second image B to form a matching quadrilateral. It is determined whether the bounding boxes (bx1, by1), (bx2, by2), (bx3, by3), and (bx4, by4) of the pavement defect in the second image B are located within the corresponding matching quadrilateral. If so, it is determined that the pavement defects in the first image A and the second image B are the same pavement defect.

[0083] If there is no matching quadrilateral, an affine triangle is used to match the road surface defect. Specifically, UFLD (Ultra-Fast Structure-Aware Deep Lane Detection) is used to detect the lane line edges in the first image A and the second image B respectively, and the lane line intersection points I are determined respectively. a ,I b and the intersection point O of the lane line and the edge of the image a , O b ; In the first image A, draw auxiliary lines through O a , so that it is in F a Intersecting with another lane line, ensure that I a F a =I a O a , F a O a to I a The vertical line of is used to approximate the direction of the lane line. In the second image B, the auxiliary line is drawn through O b , so that it is in F b Intersecting with another lane line, ensure that I b F b =I b O b , F b O b to I b The vertical line of is used to approximate the direction of the lane line. Since the projection of any object photographed by the vehicle on the center line of the lane should be equal, two matching points M are selected on the road surface defect of the first image A. a and N a , through M a and N a The parallel line of F a O a Parallel, and determine their projection point M on the lane centerline ta and N ta Similarly, two matching points M are selected on the road surface damage of the second image B. b and N b , through M b and N b The parallel line of F b O b Parallel, and determine their projection point M on the lane centerline tb and N tb . Due to M ta N ta and M tb N tb The actual distance is the same, so the projection point M ta With N ta The distance between taN ta )、M tb With N tb The distance between tb N tb ) to calculate the projection ratio of the two images α = h(M ta N ta ) / h(M tb N tb ). Determine the midpoint D of the pavement defect bounding box in the first image and the second image. a and D b , the midpoint D a With N ta The distance h(D a N ta ) and midpoint D b With N tb The distance h(D b N tb ) is within the preset upper and lower limits of the projection ratio, that is, h(D a N ta )∈[0.8*α*h(D b N tb ),1.2*α*h(D b N tb )], it is determined that the pavement defects in the first image and the second image are the same pavement defect.

[0084] After obtaining the tracking of each disease, a reid is assigned to each disease through the disease matching algorithm. The same reid represents the same disease, thereby achieving long-term tracking of pavement performance degradation.

[0085] Example 2

[0086] This embodiment verifies the large-scale road performance degradation tracking method based on logistics vehicle data described in Embodiment 1.

[0087] (1) Collect road images using industrial cameras mounted on logistics vehicles

[0088] In order to collect images containing road surface defects, this embodiment installs an industrial camera on a logistics vehicle, and uses the movement of the logistics vehicle to collect road data in a certain area for 6 months, and manually annotates various defects for subsequent analysis.

[0089] (2) Detect road surface defects using deep learning image detection algorithms

[0090] The improved YOLOv8 model is used to identify the collected road images, and the average recognition accuracy of six types of road diseases, namely cracks, potholes, network cracks, repaired cracks, repaired potholes, and repaired network cracks, is calculated as follows:

[0091] Road damage types mAP50 mAP50-95 crack 69.7 51.3 Potholes 65.1 48.7 Web-like cracks 74.8 55.4 Repairing cracks 80.2 58.1 Pothole repair 89.3 62.2 Repairing mesh cracks 95.2 65.4

[0092] (3) Using the Bernoulli function to evaluate the minimum detection frequency of a road section

[0093] For each disease, mAP50 is used for calculation and rounded up to ensure that the disease is sampled by enough logistics vehicles. The minimum number of sampling times (i.e., minimum detection frequency) required for each disease is as follows:

[0094] mAP50 Sampling times crack 69.7 2 Potholes 65.1 3 Web-like cracks 74.8 2 Repairing cracks 80.2 2 Pothole repair 89.3 2 Repairing mesh cracks 95.2 1

[0095] (4) Using the three-stage disease matching algorithm to screen road diseases and track pavement performance degradation

[0096] For GPS positioning, in order to balance accuracy and false negative rate, a clustering threshold of 10 meters was used, and the collected images were divided into 161 cluster units. In each cluster unit, the SuperPoint algorithm was used to detect interest points and extract descriptors. The SuperPoint algorithm was set to detect 500 key points of each image and generate corresponding descriptors. The extracted key points and descriptors were input into the SuperGlue algorithm for matching. The SuperGlue algorithm successfully established a reliable matching correspondence between images in the same cluster unit through its graph attention mechanism. The similarity between images was evaluated through the matching results, and the sinkhorn algorithm was used to screen out image pairs belonging to the same scene or similar perspectives.

[0097] For the selected image pairs, the bounding box of each defect drawn by the image recognition algorithm is extracted. Since some defects may not be matched directly by the SuperGlue algorithm due to the shooting angle or unclear features, the adjacent local area matching algorithm is used to further match the defects. For each matched defect, the same reid is assigned to them for numbering, so as to track the deterioration of the pavement defects.

[0098] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning, or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A large-scale road performance degradation tracking method based on logistics vehicle data, characterized in that: The following steps are involved: S1, based on the dashcam image data and recorded GPS location data of logistics vehicles, uses artificial intelligence image detection technology to analyze and extract the collected images, combines the two-layer routing attention mechanism and the P2 small target layer, and optimizes the bounding box regression loss function to accurately identify multiple types of road surface diseases in the image; S2, based on the detection accuracy of the road surface disease identification result in step S1, the Bernoulli function is used to evaluate the minimum detection frequency of the road section; S3, based on the three-stage disease matching algorithm, realizes the automatic evaluation of pavement performance degradation. The first stage of the three-stage disease matching algorithm uses GPS location data to preliminarily screen the pavement disease identification result image of step S1, and uses the minimum detection frequency determined in step S2 to sample the image, and screens out the road surface pictures with diseases at the same location. The second stage ensures that the images screened in the first stage belong to the same scene or similar perspective through background matching. The third stage uses the adjacent local area matching algorithm to match the specific pavement diseases in the image processed by the second stage, so as to realize the automatic evaluation of pavement performance degradation.

2. A large-scale road performance degradation tracking method based on logistics vehicle data according to claim 1, characterized in that: The types of pavement diseases include cracks, cracks, potholes, crack repairs, crack repairs, and pothole repairs.

3. The large-scale road performance degradation tracking method based on logistics vehicle data according to claim 1 is characterized in that: The artificial intelligence image detection technology adopts the real-time detection model YOLOv8 in the field of computer vision, and its training process is as follows: S11, establishing a pavement disease damage data set based on disease history data; S12, preprocessing images in the pavement disease damage dataset; S13, based on the YOLOv8 model, combines the two-layer routing attention mechanism and the P2 small target layer to focus on road surface diseases and municipal facility damage at different scales; S14, post-processing the output results of the YOLOv8 model, including non-maximum suppression and bounding box regression loss function optimization; S15, use the loss function to optimize the convolution kernel of the YOLOv8 model and prune the trained model.

4. The large-scale road performance degradation tracking method based on logistics vehicle data according to claim 3 is characterized in that: The preprocessing includes image denoising, resizing and normalization.

5. The large-scale road performance degradation tracking method based on logistics vehicle data according to claim 3 is characterized in that: Based on the YOLOv8 model, the two-layer routing attention mechanism and the P2 small target layer are specifically: The improved SFBPPF module is introduced into the last layer of the backbone network of the YOLOv8 model. The SFBPPF module consists of a CBS module, an RFB module, and a maximum pooling layer. The input data of the SFBPPF module is processed by the first CBS module, and then passes through the RFB module and three maximum pooling layers in sequence. The outputs of the three maximum pooling layers and the output of the first CBS module are spliced ​​and input into the second CBS module. The output of the CBS module is the output of the SFBPPF module. A P2 small target layer is introduced in the detection head of the YOLOv8 model. The P2 small target layer includes an upsampling layer, a splicing operation, a C2fBi module and a CBS module. The C2fBi module introduces a double-layer routing attention mechanism and a depthwise separable convolution on the basis of the C2f module. The input data is processed by the depthwise separable convolution and then added to the input data to obtain a first feature. The first feature is sequentially processed by layer normalization and the output after the double-layer routing attention is added to the first feature to obtain a second feature. The second feature is sequentially processed by layer normalization and the output after the MLP layer is added to the second feature to obtain the output of the C2fBi module.

6. The large-scale road performance degradation tracking method based on logistics vehicle data according to claim 1 is characterized in that: The calculation formula of the optimized bounding box regression loss function is as follows: L CDI oh U v1=L I oh U +R CDI oh U Among them, L CDI o U v1 is the optimized bounding box regression loss function, L I o U is the bounding box regression loss function, R CDI o U is a penalty function that measures the regression error between the predicted box and the true box. represents the left distance vector from the anchor box to the target box, represents the right distance vector from the anchor box to the target box, is the diagonal vector of the minimum bounding box.

7. The large-scale road performance degradation tracking method based on logistics vehicle data according to claim 1 is characterized in that: The calculation formula of the Bernoulli function is: P(x≥1)=1-P(x=0)=1-(1-pc)n≥P TH Where P(x=k) is the probability of an event occurring k times, n represents the minimum detection frequency, and p c is the minimum detection accuracy of various diseases in step S1, is the combination number formula, which means the number of combinations of k elements selected from n elements, P TH is the preset detection accuracy threshold.

8. The large-scale road performance degradation tracking method based on logistics vehicle data according to claim 1 is characterized in that: The use of GPS location data to preliminarily screen the pavement disease identification result image of step S1 is specifically as follows: Using the GPS location data recorded by logistics vehicles, different clustering radii are selected in different road scenes. The images of nearby points are screened through the clustering algorithm, and secondary clustering is performed according to the azimuth in the GPS location data. The azimuth ranges from 0 to 360, where 0 represents north and 90 represents east.

9. The large-scale road performance degradation tracking method based on logistics vehicle data according to claim 1 is characterized in that: The specific steps of the background matching include: The images in the same cluster are grouped in pairs, and the SuperPoint algorithm is used to detect key points and extract descriptors for the pair of images to be matched. The key points and descriptors of the extracted pair of images are sent to the SuperGlue algorithm. Through the graph attention mechanism of SuperGlue, the similarities between the key points are learned, and the extracted key points and descriptors are matched one by one to establish a reliable matching correspondence. Based on the matching results, the similarity between the two images is evaluated, and the sinkhorn algorithm is used to find the optimal pairing to achieve background matching.

10. The large-scale road performance degradation tracking method based on logistics vehicle data according to claim 1 is characterized in that: The specific steps of the adjacent local area matching algorithm include: Determine whether there is a corresponding matching quadrilateral for a pair of images to be matched, the pair of images comprising a first image and a second image; If there is a matching quadrilateral, for the pavement disease in the first image and the second image, extract their bounding box coordinates respectively, and determine the set of matching pixel points MA in the first image and the second image; select four pixel points closest to the bounding box of the pavement disease in the first image in the set MA, determine the matching pair of the four pixel points in the second image, and form a matching quadrilateral; determine whether the bounding box of the pavement disease in the second image is located in the corresponding matching quadrilateral, and if it is located in the matching quadrilateral, it is determined that the pavement disease in the first image and the second image are the same pavement disease; If there is no matching quadrilateral, use UFLD to detect the lane line edges in the first image and the second image respectively, and determine the lane line intersection point I respectively. a ,I b and the intersection point O of the lane line and the edge of the image a , O b ; In the first image, draw auxiliary lines through O a , so that it is in F a Intersecting with another lane line, ensure that I a F a =I a O a , to estimate the lane line direction, in the second image, draw an auxiliary line through O b , so that it is in F b Intersecting with another lane line, ensure that I b F b =I b O b , to estimate the lane line direction; select two matching points M on the road surface disease of the first image a and N a , through M a and N a The parallel line of F a O a Parallel, and determine their projection point M on the lane centerline ta and N ta ; Select two matching points M on the road surface damage of the second image b and N b , through M b and N b The parallel line of F b O b Parallel, and determine their projection point M on the lane centerline tb and N tb ; Based on the projection point M ta With N ta 、M tb With N tb The distance between ta N ta )、h(M tb N tb ) Calculate the projection ratio of the two images α = h(M ta N ta ) / h(M tb N tb );Determine the midpoint D of the pavement defect boundary box in the first image and the second image a and D b , the midpoint D a With N ta The distance h(D a N ta ) and midpoint D b With N tb The distance h(D b N tb ) is within the preset upper and lower limits of the projection ratio, it is determined that the pavement defects in the first image and the second image are the same pavement defect.

Images