A large-scale road performance degradation tracking method based on logistics vehicle data

Abstract

The present invention relates to a large-scale pavement performance degradation tracking method based on logistics vehicle data. The method comprises the following steps: accurately identifying multiple types of pavement defects in images based on dashcam image data and recorded GPS location data from logistics vehicles; employing a Bernoulli function to assess the minimum detection frequency of road sections; and automatically assessing pavement performance degradation using a three-stage defect matching algorithm. In the first stage, images of pavement defect identification results are preliminarily screened using GPS location data, and image sampling is performed using the minimum detection frequency to select images of pavement with defects in the same location. In the second stage, background matching is used to ensure that images belong to the same scene or similar perspectives. In the third stage, an adjacent local area matching algorithm is used to match specific pavement defects, thereby automatically assessing pavement performance degradation. Compared with existing technologies, the present invention can update large-scale pavement damage information at a high frequency, achieving rapid detection and matching of pavement defects.

Description

Technical Field

[0001] The present invention relates to the field of road infrastructure performance detection and evaluation, and in particular to a large-scale road performance degradation tracking method based on logistics vehicle data. Background Art

[0002] In recent years, the field of road infrastructure has rapidly developed, and the large inventory of in-service roads has led to unprecedented maintenance pressure. Scientific data support is indispensable to ensure the effective implementation of daily road maintenance operations and preventive maintenance measures. Therefore, rapid and accurate monitoring of pavement performance and in-depth analysis of performance degradation trends are core requirements in road maintenance research and practice. This ability to perceive and analyze performance trends not only helps identify current road conditions but also predicts future performance changes, providing data-driven support for maintenance decisions. Furthermore, tracking the development of pavement defects and analyzing their deterioration are also crucial for optimizing road facility distribution, regulating traffic loads, predicting pavement performance, and rationally designing road structural materials. Therefore, real-time monitoring and evaluation of road infrastructure performance are necessary.

[0003] Existing pavement defect detection methods often only focus on detection accuracy and efficiency, do not consider historical detection results, lack the ability to track detected defects, and lack the detection of road infrastructure performance. Summary of the Invention

[0004] The purpose of the present invention is to provide a large-scale pavement performance degradation tracking method based on logistics vehicle data. The method inputs road images and GPS location data, determines the type of disease through a computer vision algorithm, deduplicates the diseases and analyzes the deterioration condition through a disease matching method, outputs the disease type, disease number and disease degradation condition, and realizes high-precision large-scale pavement performance degradation tracking.

[0005] The purpose of the present invention can be achieved by the following technical solutions:

[0006] A large-scale road performance degradation tracking method based on logistics vehicle data includes the following steps:

[0007] S1, based on the dashcam image data and recorded GPS location data of logistics vehicles, uses artificial intelligence image detection technology to analyze and extract the collected images. It combines a two-layer routing attention mechanism and a P2 small object layer, and optimizes the bounding box regression loss function to accurately identify multiple types of road surface defects in the images;

[0008] S2, based on the detection accuracy of the pavement defect identification results in step S1, the Bernoulli function is used to evaluate the minimum detection frequency of the road section to ensure the adequacy and representativeness of the pavement defect data;

[0009] S3, based on a three-stage defect matching algorithm, automatically assesses pavement performance degradation, effectively eliminating duplicate detection and continuously tracking the deterioration of defects. The first stage of the three-stage defect matching algorithm uses GPS location data to perform a preliminary screening of the pavement defect identification results images from step S1. Image sampling is performed using the minimum detection frequency determined in step S2 to select images of pavement with defects at the same location. The second stage uses background matching to ensure that the images screened in the first stage belong to the same scene or similar perspective, reducing mismatches. The third stage uses an adjacent local area matching algorithm to match specific pavement defects in the images processed in the second stage, ensuring the accuracy and continuity of the defect data and enabling automatic assessment of pavement performance degradation.

[0010] The types of pavement diseases include cracks, cracks, potholes, crack repairs, cracks repairs, and pothole repairs.

[0011] The artificial intelligence image detection technology uses the real-time detection model YOLOv8 in the field of computer vision. Its training process is as follows:

[0012] S11, establishing a pavement disease damage dataset based on historical disease data;

[0013] S12, preprocessing the images in the pavement disease damage dataset to improve image quality and adapt to the input requirements of the YOLOv8 model;

[0014] S13, based on the YOLOv8 model, combines a two-layer routing attention mechanism and a P2 small target layer to focus on road surface damage and municipal facility damage at different scales, improving the accuracy of damage identification;

[0015] S14, post-processing the output of the YOLOv8 model, including non-maximum suppression (NMS) and bounding box regression loss function optimization, to reduce positioning errors and improve the recognition accuracy of road surface defects;

[0016] S15 uses the loss function to optimize the convolution kernel of the YOLOv8 model and prunes the trained model to reduce the number of model parameters so that it can be quickly deployed on edge devices.

[0017] The preprocessing includes image denoising, size adjustment and normalization.

[0018] Based on the YOLOv8 model, the dual-layer routing attention mechanism and the P2 small target layer are specifically:

[0019] The improved SFBPPF module is introduced into the last layer of the backbone network of the YOLOv8 model. The SFBPPF module consists of a CBS module, an RFB module, and a maximum pooling layer. The input data of the SFBPPF module is processed by the first CBS module, and then passes through the RFB module and three maximum pooling layers in sequence. The outputs of the three maximum pooling layers and the output of the first CBS module are concatenated and input into the second CBS module. The output of the CBS module is the output of the SFBPPF module.

[0020] A P2 small target layer is introduced in the detection head of the YOLOv8 model. The P2 small target layer includes an upsampling layer, a splicing operation, a C2fBi module and a CBS module. The C2fBi module introduces a double-layer routing attention mechanism and a depthwise separable convolution on the basis of the C2f module. The input data is processed by the depthwise separable convolution and then added to the input data to obtain the first feature. The first feature is sequentially processed by layer normalization and the output after the double-layer routing attention is added to the first feature to obtain the second feature. The second feature is sequentially processed by layer normalization and the output after the MLP layer is added to the second feature to obtain the output of the C2fBi module.

[0021] The calculation formula of the optimized bounding box regression loss function is as follows:

[0022] L CDIoUv1 =L IoU +R CDIoU

[0023]

[0024] Among them, L CDIoUv1 is the optimized bounding box regression loss function, L IoU is the bounding box regression loss function, R CDIoU is a penalty function that measures the regression error between the predicted box and the true box. represents the left distance vector from the anchor box to the target box, represents the right distance vector from the anchor box to the target box, is the diagonal vector of the minimum bounding box.

[0025] The calculation formula of the Bernoulli function is:

[0026]

[0027] P(x≥1)=1-P(x=0)=1-(1-p c ) n ≥P TH

[0028] Where P(x=k) is the probability of an event occurring k times, n represents the minimum detection frequency, and pc is the minimum detection accuracy of various diseases in step S1, is the combination number formula, which means the number of combinations of k elements selected from n elements, P TH is the preset detection accuracy threshold.

[0029] The GPS location data is used to perform preliminary screening of the pavement damage identification result image in step S1, specifically:

[0030] Using GPS location data recorded by logistics vehicles, we select different cluster radii for different road scenarios and use a clustering algorithm to filter images of nearby locations. Direct GPS clustering can result in images from opposing lanes or intersecting lanes being matched together. We then perform secondary clustering using the azimuth angle data from the GPS location data. Azimuth angles range from 0 to 360 degrees, with 0 representing north and 90 representing east.

[0031] The specific steps of the background matching include:

[0032] Images in the same cluster are grouped in pairs. The SuperPoint algorithm is used to detect key points and extract descriptors for the matching pair of images. The extracted key points and descriptors of the pair of images are fed into the SuperGlue algorithm. Through SuperGlue's graph attention mechanism, the similarities between the key points are learned, and the extracted key points and descriptors are matched one by one to establish a reliable matching correspondence. Based on the matching results, the similarity between the two images is evaluated, and the sinkhorn algorithm is used to find the optimal pairing to achieve background matching.

[0033] The specific steps of the adjacent local area matching algorithm include:

[0034] Determining whether a corresponding matching quadrilateral exists for a pair of images to be matched, the pair of images comprising a first image and a second image;

[0035] If a matching quadrilateral exists, the bounding box coordinates of the pavement defect in the first image and the second image are extracted, respectively, to determine a set MA of matching pixels in the first image and the second image; four pixels in the set MA that are closest to the pavement defect bounding box in the first image are selected, and matching pairs of these four pixels in the second image are determined to form a matching quadrilateral; a determination is made as to whether the bounding box of the pavement defect in the second image is within the corresponding matching quadrilateral; if so, the pavement defects in the first and second images are deemed to be the same pavement defect;

[0036] If there is no matching quadrilateral, use UFLD to detect the lane line edges in the first image and the second image respectively, and determine the lane line intersection point I respectively.a , I b and the intersection point O of the lane line and the image edge a , O b ; In the first image, draw auxiliary lines through O a , so that it is in F a Intersect with another lane line, make sure I a F a =I a O a To estimate the lane line direction, in the second image, draw an auxiliary line through O b , so that it is in F b Intersect with another lane line, make sure I b F b =I b O b , to estimate the lane line direction; select two matching points M on the road surface disease of the first image a and N a , through M a and N a The parallel line of F a O a Parallel, and determine their projection point M on the lane centerline ta and N ta ; Select two matching points M on the road surface damage in the second image b and N b , through M b and N b The parallel line of F b O b Parallel, and determine their projection point M on the lane centerline tb and N tb ; Based on the projection point M ta With N ta 、M tb With N tb The distance between h(M ta N ta )、h(M tb N tb ) Calculate the projection ratio of the two images α=h(M ta N ta ) / h(M tb N tb ) ; Determine the midpoint D of the pavement defect bounding box in the first image and the second image a and D b , midpoint D a With N ta The distance h(D a N ta ) and midpoint D b With Ntb The distance h(D b N tb ) is within the preset upper and lower limits of the projection ratio, it is determined that the pavement defects in the first image and the second image are the same pavement defect.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] The present invention relates to a large-scale pavement performance degradation tracking method based on logistics vehicle data. The method uses the driving recorder of the logistics vehicle to collect street view images, and realizes the accurate identification of multiple types of pavement diseases by constructing a deep convolutional neural network and combining it with a two-layer routing attention mechanism. At the same time, the bounding box regression loss function is optimized to improve the recognition accuracy of common pavement defects. In addition, the Bernoulli function is used to evaluate the minimum detection frequency of the road section, so that image sampling can be performed reasonably during the pavement performance degradation tracking process. Finally, a three-stage disease matching algorithm is designed, which combines the attention graph neural network and the adjacent local area matching algorithm to effectively eliminate repeated detection and continuously track the deterioration of the disease, thereby realizing pavement performance degradation tracking. Compared with the existing pavement performance degradation tracking method, the present invention can update large-scale pavement damage at a high frequency, realizes rapid detection and matching of pavement diseases, and can analyze the deterioration of road diseases. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 is a flow chart of the method of the present invention;

[0040] Figure 2 This is the YOLOv8 model architecture diagram after optimization by the present invention;

[0041] Figure 3 Schematic diagram of the bounding box regression loss function of the present invention;

[0042] Figure 4 This is a diagram of the three-stage disease matching algorithm architecture of the present invention;

[0043] Figure 5 This is a diagram of the background matching algorithm architecture of the present invention;

[0044] Figure 6 This is a diagram of the adjacent local area matching algorithm architecture of the present invention. DETAILED DESCRIPTION

[0045] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0046] Example 1

[0047] like Figure 1 As shown, this embodiment provides a large-scale road performance degradation tracking method based on logistics vehicle data, including the following steps:

[0048] S1, based on the dashcam image data and recorded GPS location data of logistics vehicles, uses artificial intelligence image detection technology to analyze and extract the collected images, combines a two-layer routing attention mechanism and a P2 small target layer, and optimizes the bounding box regression loss function to accurately identify multiple types of road surface defects in the images.

[0049] To achieve high-frequency detection of road defects, this embodiment installs an industrial camera in front of a logistics vehicle. While the vehicle is in motion, it automatically collects street view images from both sides of the road and records information such as the image's geographic location and capture time. The collected street view images may be blurry, backlit, or overexposed, affecting the accuracy of subsequent defect identification. Therefore, image screening is necessary. The specific screening process is as follows:

[0050] (1) Improve image quality through image preprocessing techniques such as dehazing and contrast enhancement.

[0051] (2) Remove pictures taken in rainy days or other conditions where the camera lens is disabled.

[0052] (3) Sample the images according to vehicle speed and remove duplicate and highly similar images.

[0053] To evaluate the deterioration of pavement defects is to analyze the temporal changes and spatial distribution of pavement defects. For example, on a temporal scale, cracks will appear on paved asphalt roads over time, and after a natural disaster, sudden potholes may appear on the road surface; on a spatial scale, roads with heavy loads are often more prone to pavement defects. To this end, it is first necessary to automatically identify pavement defects in images. That is, to identify and locate specific targets in the image, assign a bounding box and a corresponding category label to each target. This process can usually be performed using target detection. Target detection is a core task in the field of computer vision. It aims to locate and identify targets of interest in an image, and assign a bounding box and a corresponding category label to each target. Through deep learning neural networks, computers can autonomously learn, analyze and extract key features of targets from massive amounts of labeled data. Based on deep learning target detection algorithms, computers can automatically find targets in images, accurately locate and identify targets, and thus provide effective technical support for various application scenarios. The artificial intelligence image detection technology of this embodiment adopts the real-time detection model YOLOv8 in the field of computer vision, and its structure is as follows: Figure 2 As shown, the training process is as follows:

[0054] S11, establishing a pavement disease damage dataset based on historical disease data.

[0055] S12, performing preprocessing such as image denoising, size adjustment and normalization on the images in the pavement disease damage dataset.

[0056] Specifically, we used the large number of road images collected above to select representative images containing road surface defects. We then performed data enhancement on these images, including mirroring, cropping, adding noise, and brightness transformation. We then used the open-source image annotation tool Labelme to annotate each enhanced roadscape image for various defects, including cracks, cracks, potholes, repaired cracks, repaired cracks, and repaired potholes. This approach resulted in the creation of a road surface damage dataset.

[0057] S13, based on the YOLOv8 model, combines a two-layer routing attention mechanism and a P2 small target layer to focus on road surface diseases and municipal facility damage at different scales.

[0058] like Figure 2 As shown in the figure, the improved SFBPPF module is introduced in the last layer of the YOLOv8 model's backbone network. By incorporating more scale information, this module can better extract features of small objects. The SFBPPF module consists of a CBS module, an RFB module, and a max pooling layer. After being processed by the first CBS module, the input data of the SFBPPF module passes through the RFB module and three max pooling layers. The outputs of the three max pooling layers and the output of the first CBS module are concatenated and input into the second CBS module. The output of the CBS module becomes the output of the SFBPPF module.

[0059] A P2 small target layer is introduced in the detection head of the YOLOv8 model. The P2 small target layer includes an upsampling layer, a splicing operation, a C2fBi module and a CBS module. The C2fBi module introduces a double-layer routing attention mechanism and a depthwise separable convolution on the basis of the C2f module. The input data is processed by the depthwise separable convolution and then added to the input data to obtain the first feature. The first feature is sequentially processed by layer normalization and the output after the double-layer routing attention is added to the first feature to obtain the second feature. The second feature is sequentially processed by layer normalization and the output after the MLP layer is added to the second feature to obtain the output of the C2fBi module.

[0060] The dual-layer routing attention mechanism is integrated with the P2 small target detection layer. This dual-layer routing attention mechanism can effectively improve the network's computational efficiency while also enhancing the network's ability to detect small targets. The P2 small target detection layer is a layer specifically designed for detecting small targets. Combining it with the dual-layer routing attention mechanism can further improve small target detection. Depthwise separable convolution is a lightweight convolution operation that can effectively reduce the number of network parameters and computation. The C2f module is a module used to enhance the network's feature transfer capabilities. Combining it with depthwise separable convolution can reduce the network's computational load while maintaining the network's detection effectiveness.

[0061] S14, post-processing the output results of the YOLOv8 model, including non-maximum suppression and bounding box regression loss function optimization.

[0062] The present invention optimizes the evaluation index IoU. Since cracks usually have a small actual area, their annotation quality in the image is often low, which causes the traditional bounding box regression loss function to produce unstable gradients during training, thereby affecting the generalization ability of the model. Low-quality annotation of cracks is inevitable, and geometric factors will significantly increase the penalty for low-quality examples. In order to effectively weaken the penalty of geometric factors and improve the generalization ability of the model, the present invention constructs a multi-cost fusion CDIoU. Figure 3 As shown, the calculation formula of the optimized bounding box regression loss function is as follows:

[0063] L CDIoUv1 =L IoU +R CDIoU

[0064]

[0065] Among them, L CDIoUv1 is the optimized bounding box regression loss function, L IoU is the bounding box regression loss function, R CDIoU is a penalty function that measures the regression error between the predicted box and the true box. represents the left distance vector from the anchor box to the target box, represents the right distance vector from the anchor box to the target box, is the diagonal vector of the minimum bounding box.

[0066] S15, use the loss function to optimize the convolution kernel of the YOLOv8 model and prune the trained model.

[0067] S2, based on the detection accuracy of the road surface disease identification result in step S1, the Bernoulli function is used to evaluate the minimum detection frequency of the road section.

[0068] Although computer vision algorithms can effectively detect pavement defects, the accuracy of a single inspection is limited due to the complex environment. It is necessary to combine the results of multiple inspections to improve the detection rate. Assuming that the successful detection of each pavement defect is independent, which can be regarded as a Bernoulli trial, the number of successful defect detections follows a binomial distribution. The binomial distribution describes the probability of success or failure in multiple repeated inspections. For each pavement defect, its detection frequency is given by the average precision p of the computer vision algorithm. c Estimate, where c represents the disease type. Since x ~ B(N, Pc), the detection rate of k inspections is as follows:

[0069]

[0070] To ensure that more than 90% of pavement defects can be detected, the minimum detection frequency is determined according to the following formula:

[0071] P(x≥1)=1-P(x=0)=1-(1-p c ) n ≥P TH

[0072] Where P(x=k) is the probability of an event occurring k times, n represents the minimum detection frequency, and p c is the minimum detection accuracy of various diseases in step S1, is the combination number formula, which means the number of combinations of k elements selected from n elements, P TH is a preset detection accuracy threshold, which is set to 0.90 in this embodiment.

[0073] S3, automatic assessment of pavement performance degradation based on a three-stage disease matching algorithm.

[0074] The architecture of the three-stage disease matching algorithm is as follows Figure 4 As shown, it includes the following three stages:

[0075] Phase 1: Use GPS location data to preliminarily screen the pavement defect identification result images from step S1, and use the minimum detection frequency determined in step S2 to sample images to screen out images of pavement with defects at the same location.

[0076] Taking into account the obstruction of urban buildings and the positioning error of GPS on multi-lane roads, we first use the GPS location data recorded by logistics vehicles, and select 5 meters, 10 meters, and 15 meters as the clustering radius in different road scenes. The images of adjacent points are screened by the clustering algorithm, and secondary clustering is performed according to the azimuth in the GPS location data, where the azimuth value range is 0 to 360, where 0 represents north and 90 represents east. This embodiment adopts an improved K-means algorithm, which adds a filtering mechanism for outlier data in the clustering process. This improved algorithm can better handle outliers in GPS data, thereby improving the accuracy and efficiency of clustering. By filtering out outliers, the algorithm can more accurately determine the cluster center, so that the clustering results are more in line with the actual situation.

[0077] Phase 2: Background matching is used to ensure that the images selected in the first phase belong to the same scene or similar perspectives.

[0078] The invention uses the deep learning method of SuperPoint+SuperGlue to further screen the road images in the same cluster. SuperPoint is a general point of interest detection and description algorithm that can generate key points with good characteristics and describe these key points for matching between different images. SuperPoint is trained through a deep learning network and can maintain stable key point detection performance under various scenarios and transformations. SuperGlue is an efficient feature matching algorithm that has achieved advanced results in feature matching tasks in outdoor environments. It is an end-to-end model composed of CNN feature extraction and graph neural network (GNN) feature matching. It learns the geometric transformation and regularity priors of the 3D world through self-attention and cross-attention mechanisms. End-to-end training enables it to surpass traditional hand-designed functions such as SIFT. Its architecture is as follows Figure 5 As shown in Figure 2, SuperGlue can establish reliable matching correspondences between different images by learning the relationship between feature points.

[0079] Specifically, the images in the same cluster are grouped in pairs, and the SuperPoint algorithm is used to detect key points and extract descriptors for the pair of images to be matched. The key points and descriptors of the extracted pair of images are sent to the SuperGlue algorithm. Through the graph attention mechanism of SuperGlue, the similarity between the key points is learned, and the extracted key points and descriptors are matched one by one to establish a reliable matching correspondence. Based on the matching results, the similarity between the two images is evaluated, and the sinkhorn algorithm is used to find the optimal pairing to achieve background matching.

[0080] Phase 3: Use the adjacent local area matching algorithm to match specific pavement defects in the images processed in the second phase to achieve automatic assessment of pavement performance degradation.

[0081] The SuperGlue network provides matching pixels between the two images, while the image recognition algorithm draws a bounding box for each defect. If the pixels in the two bounding boxes are matched, then there are duplicate road surface defects in the two images. However, due to different shooting angles, unclear features or abnormal weather, the same defects in the two images may not share matching features. Repaired defects and potholes are easy to match because their features are more obvious, but the original defects are difficult to match directly through Superglue because their features are not obvious to the entire image. Therefore, this embodiment designs an adjacent local area matching algorithm by extracting local areas around the defects and then using a feature matching algorithm to compare the similarity of these local areas. By calculating the relative position and direction of the defects in the local area, it is determined whether the two defects are the same. This method can accurately match defects in two images even if they are different in shape and size. Specifically, if Figure 6 As shown, for a pair of images to be matched (including the first image A and the second image B), first determine whether there is a corresponding matching quadrilateral:

[0082] If a matching quadrilateral exists, extract the bounding box coordinates of the pavement damage in the first and second images, respectively. The bounding box coordinates for image A are (ax1,ay1), (ax2,ay2), (ax3,ay3), (ax4,ay4), and the bounding box coordinates for image B are (bx1,by1), (bx2,by2), (bx3,by3), (bx4,by4). Determine a set MA of matching pixels in the first and second images A and B. Select the four pixels (nax1,nay1), (nax2,nay2), (nax3,nay3), (nax4,nay4)∈MA that are closest to the pavement damage bounding box in the first image A from the set MA. Determine the matching pairs (nbx1,nby1), (nbx2,nby2), (nbx3,nby3), (nbx4,nby4) of these four pixels in the second image B to form a matching quadrilateral. Determine whether the bounding boxes (bx1, by1), (bx2, by2), (bx3, by3), and (bx4, by4) of the pavement defect in the second image B are within the corresponding matching quadrilateral. If so, it is determined that the pavement defects in the first image A and the second image B are the same pavement defect.

[0083] If there is no matching quadrilateral, an affine triangle is used to match the road surface defect. Specifically, UFLD (Ultra-Fast Structure-Aware Deep Lane Detection) is used to detect the lane line edges in the first image A and the second image B, and the lane line intersection points I are determined respectively. a , I b and the intersection point O of the lane line and the image edge a , O b ; In the first image A, draw auxiliary lines through O a , so that it is in F a Intersect with another lane line, make sure I a F a =I a O a , F a O a to I a The vertical line of is used to approximate the direction of the lane line. In the second image B, the auxiliary line is drawn through O b , so that it is in F b Intersect with another lane line, make sure I b F b =I b O b , F b O b to I b The vertical line of is used to approximate the direction of the lane line. Since the projection of any object photographed by the vehicle on the lane centerline should be equal, two matching points M are selected on the road surface defect of the first image A. a and N a , through M a and N a The parallel line of F a O a Parallel, and determine their projection point M on the lane centerline ta and N ta Similarly, two matching points M are selected on the road surface damage of the second image B. b and N b , through M b and N b The parallel line of F b O b Parallel, and determine their projection point M on the lane centerline tb and N tb . Due to M ta N ta and M tb N tb The actual distance is the same, so the projection point M ta With N ta The distance between h(M taN ta ), M tb With N tb The distance between h(M tb N tb ) to calculate the projection ratio of the two images α=h(M ta N ta ) / h(M tb N tb ). Determine the midpoint D of the pavement defect bounding box in the first image and the second image. a and D b , midpoint D a With N ta The distance h(D a N ta ) and midpoint D b With N tb The distance h(D b N tb ) is within the preset upper and lower limits of the projection ratio, that is, h(D a N ta )∈[0.8*α*h(D b N tb ),1.2*α*h(D b N tb )], it is determined that the pavement defects in the first image and the second image are the same pavement defect.

[0084] After obtaining the tracking of each disease, a reid is assigned to each disease through the disease matching algorithm. The same reid represents the same disease, thereby achieving long-term tracking of pavement performance degradation.

[0085] Example 2

[0086] This embodiment verifies the large-scale road performance degradation tracking method based on logistics vehicle data described in Example 1.

[0087] (1) Using industrial cameras mounted on logistics vehicles to collect road images

[0088] In order to collect images containing road surface defects, this embodiment installs an industrial camera on a logistics vehicle, and uses the movement of the logistics vehicle to collect road data in a certain area for 6 months, and manually annotates various defects for subsequent analysis.

[0089] (2) Detecting road surface defects using deep learning image detection algorithms

[0090] The improved YOLOv8 model was used to identify the collected road images. The average recognition accuracy of six types of road defects, namely cracks, potholes, network cracks, repaired cracks, repaired potholes, and repaired network cracks, was calculated as follows:

[0091] Road damage types mAP50 mAP50-95 crack 69.7 51.3 potholes 65.1 48.7 Network cracks 74.8 55.4 Repair cracks 80.2 58.1 Pothole repair 89.3 62.2 Repairing network cracks 95.2 65.4

[0092] (3) Using Bernoulli function to evaluate the minimum detection frequency of road sections

[0093] For each disease, mAP50 is used for calculation and rounded up to ensure that the disease is sampled by a sufficient number of logistics vehicles. The minimum number of sampling times (i.e., minimum detection frequency) required for each disease is as follows:

[0094] mAP50 Sampling times crack 69.7 2 potholes 65.1 3 Network cracks 74.8 2 Repair cracks 80.2 2 Pothole repair 89.3 2 Repairing network cracks 95.2 1

[0095] (4) Using a three-stage disease matching algorithm to screen road diseases and track pavement performance degradation

[0096] For GPS positioning, to balance accuracy and false negatives, a clustering threshold of 10 meters was used. The collected images were divided into 161 cluster units. Within each cluster unit, the SuperPoint algorithm was used to detect points of interest and extract descriptors. The SuperPoint algorithm was set to detect 500 key points in each image and generate corresponding descriptors. The extracted key points and descriptors were input into the SuperGlue algorithm for matching. Using its graph attention mechanism, the SuperGlue algorithm successfully established reliable matching relationships between images in the same cluster unit. The matching results were used to evaluate the similarity between the images, and the Sinkhorn algorithm was used to select image pairs that belong to the same scene or have similar viewpoints.

[0097] For each selected image pair, the bounding box of each defect is extracted using the image recognition algorithm. Because some defects may not be directly matched using the SuperGlue algorithm due to the shooting angle or unclear features, the adjacent local region matching algorithm is used to further match the defects. Each matched defect is assigned a unique ReID number to track the deterioration of the pavement.

[0098] The above describes in detail the preferred embodiments of the present invention. It should be understood that those skilled in the art can make numerous modifications and variations based on the concepts of the present invention without inventive effort. Therefore, any technical solutions that can be derived by those skilled in the art through logical analysis, reasoning, or limited experimentation based on the concepts of the present invention and the prior art should be within the scope of protection defined by the claims.

Claims

1. A large-scale road performance degradation tracking method based on logistics vehicle data, characterized by: The following steps are involved: S1, based on the dashcam image data and recorded GPS location data of logistics vehicles, uses artificial intelligence image detection technology to analyze and extract the collected images. It combines a two-layer routing attention mechanism and a P2 small object layer, and optimizes the bounding box regression loss function to accurately identify multiple types of road surface defects in the images; S2, based on the detection accuracy of the road surface disease identification results in step S1, the Bernoulli function is used to evaluate the minimum detection frequency of the road section; S3, automatic assessment of pavement performance degradation is achieved based on a three-stage defect matching algorithm. The first stage of the three-stage defect matching algorithm uses GPS location data to perform preliminary screening of the pavement defect identification result images from step S1, and uses the minimum detection frequency determined in step S2 to sample images to screen out images of pavement with defects at the same location. The second stage uses background matching to ensure that the images screened in the first stage belong to the same scene or similar perspective. The third stage uses an adjacent local area matching algorithm to match specific pavement defects in the images processed in the second stage, thereby achieving automatic assessment of pavement performance degradation. The combination of the two-layer routing attention mechanism and the P2 small target layer is specifically as follows: The improved SFBPPF module is introduced into the last layer of the backbone network of the YOLOv8 model. The SFBPPF module consists of a CBS module, an RFB module, and a maximum pooling layer. The input data of the SFBPPF module is processed by the first CBS module, and then passes through the RFB module and three maximum pooling layers in sequence. The outputs of the three maximum pooling layers and the output of the first CBS module are concatenated and input into the second CBS module. The output of the CBS module is the output of the SFBPPF module. A P2 small target layer is introduced into the detection head of the YOLOv8 model. The P2 small target layer includes an upsampling layer, a splicing operation, a C2fBi module, and a CBS module. The C2fBi module introduces a double-layer routing attention mechanism and a depthwise separable convolution on the basis of the C2f module. The input data is processed by the depthwise separable convolution and then added to the input data to obtain the first feature. The first feature is sequentially processed by layer normalization and the output after the double-layer routing attention is added to the first feature to obtain the second feature. The second feature is sequentially processed by layer normalization and the output after the MLP layer is added to the second feature to obtain the output of the C2fBi module. The calculation formula of the optimized bounding box regression loss function is as follows: in, is the optimized bounding box regression loss function, is the bounding box regression loss function, is a penalty function that measures the regression error between the predicted box and the true box. represents the left distance vector from the anchor box to the target box, represents the right distance vector from the anchor box to the target box, is the diagonal vector of the minimum bounding box; The calculation formula of the Bernoulli function is: in, P ( x = k ) is the event k The probability of times n Indicates the minimum detection frequency, is the minimum detection accuracy of various diseases in step S1, is the combination number formula, which means the number of combinations of k elements selected from n elements. is the preset detection accuracy threshold.

2. The large-scale road performance degradation tracking method based on logistics vehicle data according to claim 1 is characterized in that: The types of pavement diseases include cracks, cracks, potholes, crack repairs, cracks repairs, and pothole repairs.

3. The large-scale road performance degradation tracking method based on logistics vehicle data according to claim 1 is characterized in that: The artificial intelligence image detection technology uses the real-time detection model YOLOv8 in the field of computer vision. Its training process is as follows: S11, establishing a pavement disease damage dataset based on historical disease data; S12, preprocessing images in the pavement disease damage dataset; S13, based on the YOLOv8 model, combines a two-layer routing attention mechanism and a P2 small target layer to focus on road surface diseases and municipal facility damage at different scales; S14, post-processing the output results of the YOLOv8 model, including non-maximum suppression and bounding box regression loss function optimization; S15, use the loss function to optimize the convolution kernel of the YOLOv8 model and prune the trained model.

4. The large-scale road performance degradation tracking method based on logistics vehicle data according to claim 3 is characterized in that: The preprocessing includes image denoising, size adjustment and normalization.

5. The large-scale road performance degradation tracking method based on logistics vehicle data according to claim 1 is characterized in that: The GPS location data is used to perform preliminary screening of the pavement damage identification result image in step S1, specifically: Using the GPS location data recorded by logistics vehicles, different clustering radii are selected in different road scenes. The images of nearby points are screened through a clustering algorithm, and secondary clustering is performed according to the azimuth in the GPS location data. The azimuth ranges from 0 to 360, where 0 represents north and 90 represents east.

6. The large-scale road performance degradation tracking method based on logistics vehicle data according to claim 1 is characterized in that: The specific steps of the background matching include: Images in the same cluster are grouped in pairs. The SuperPoint algorithm is used to detect key points and extract descriptors for the matching pair of images. The extracted key points and descriptors of the pair of images are fed into the SuperGlue algorithm. Through SuperGlue's graph attention mechanism, the similarities between the key points are learned, and the extracted key points and descriptors are matched one by one to establish a reliable matching correspondence. Based on the matching results, the similarity between the two images is evaluated, and the sinkhorn algorithm is used to find the optimal pairing to achieve background matching.

7. The large-scale road performance degradation tracking method based on logistics vehicle data according to claim 1 is characterized in that: The specific steps of the adjacent local area matching algorithm include: Determining whether a corresponding matching quadrilateral exists for a pair of images to be matched, the pair of images comprising a first image and a second image; If a matching quadrilateral exists, the bounding box coordinates of the pavement defect in the first image and the second image are extracted, respectively, to determine a set MA of matching pixels in the first image and the second image; four pixels in the set MA that are closest to the pavement defect bounding box in the first image are selected, and matching pairs of these four pixels in the second image are determined to form a matching quadrilateral; a determination is made as to whether the bounding box of the pavement defect in the second image is within the corresponding matching quadrilateral; if so, the pavement defects in the first and second images are deemed to be the same pavement defect; If there is no matching quadrilateral, use UFLD to detect the lane line edges in the first image and the second image respectively, and determine the lane line intersection points respectively. I a 、 I b and the intersection of the lane line and the edge of the image O a 、 O b ; In the first image, draw auxiliary lines by O a , so that it F a Intersecting with another lane line, ensure I a F a = I a O a , to estimate the lane line direction, in the second image, draw auxiliary lines through O b , so that it F b Intersecting with another lane line, ensure I b F b = I b O b , to estimate the lane line direction; select two matching points on the road surface disease of the first image M a and N a , pass through M a and N a parallel lines, and F a O a Parallel and determine their projection points on the lane centerline M ta and N ta ; Select two matching points on the road surface damage in the second image M b and N b , pass through M b and N b parallel lines, and F b O b Parallel and determine their projection points on the lane centerline M tb and N tb ; Based on the projection point M ta and N ta 、 M tb and N tb The distance between h ( M ta N ta ), h ( M tb N tb )Calculate the projection ratio of the two images α = h ( M ta N ta ) / h ( M tb N tb ); Determine the midpoint of the pavement defect bounding box in the first image and the second image D a and D b , at the midpoint D a and N ta The distance between h ( D a N ta ) and the midpoint D b and N tb The distance between h ( D b N tb ) is within the preset upper and lower limits of the projection ratio, it is determined that the pavement defects in the first image and the second image are the same pavement defect.

Images