Intelligent detection and evaluation method for road surface disease based on multi-scale data mining

Abstract

The application provides a pavement disease intelligent detection and evaluation method based on multi-scale data mining, and belongs to the technical field of pavement disease detection. The application constructs a disease segmentation network by fusing multi-scale expansion convolution and double attention mechanism, greatly improves the pixel-level segmentation accuracy of complex background and small target disease, simultaneously builds a complete scientific link from pixel-level segmentation, multi-dimensional feature quantization to radial basis probability neural network classification, forms an end-to-end automatic processing flow from original image input to PCI index calculation, GIS visualization display, introduces spatiotemporal big data analysis into the pavement evaluation field to realize disease evolution prediction, matches an interactive correction mechanism to guarantee the accuracy and reliability of the detection and evaluation results, adopts a loosely coupled interface design between modules to support flexible deployment of edges and clouds, solves the defects of the prior art, and provides strong data support for highway scientific maintenance decision-making.

Description

Technical Field

[0001] This invention relates to the field of pavement distress detection technology, specifically to a method for intelligent detection and evaluation of pavement distress based on multi-scale data mining. Background Technology

[0002] Accurate detection and scientific evaluation of road surface defects (such as cracks, potholes, and repairs) are the core basis for highway maintenance decisions. Traditional manual inspection methods suffer from low efficiency, strong subjectivity, and significant safety hazards.

[0003] Accurate detection and scientific evaluation of pavement defects are core bases for highway maintenance decisions. Traditional manual inspection methods suffer from low efficiency, strong subjectivity, significant safety hazards, and low data standardization, making them unsuitable for the large-scale, routine maintenance needs of modern highways. Existing automatic detection technologies based on image processing and deep learning still face limitations, including a lack of high-quality benchmark datasets, insufficient robustness in segmenting complex background noise and small-target defects, a lack of systematic quantitative feature engineering for defect classification and severity assessment, the absence of an end-to-end automated process from image input to PCI calculation and GIS visualization, lack of ability to predict the spatiotemporal evolution of pavement technical conditions, and a lack of interactive correction mechanisms and flexible deployment solutions at the edge and cloud levels. These limitations hinder their ability to fully support scientific decision-making in highway maintenance. Therefore, this paper proposes an intelligent detection and evaluation method for pavement defects based on multi-scale data mining. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides an intelligent detection and evaluation method for pavement defects based on multi-scale data mining. By integrating multi-scale dilated convolution and dual attention mechanisms, it improves the segmentation accuracy of defects in complex backgrounds and small targets, providing data support for scientific maintenance decisions.

[0005] The technical solution adopted in this invention is as follows: A method for intelligent detection and evaluation of pavement defects based on multi-scale data mining includes the following steps: Step 1: Collect road surface image data and preprocess the road surface image data to obtain standardized input image data; Step 2: Input the standardized input image data into a pre-constructed pavement disease segmentation network based on a deep convolutional neural network. The pavement disease segmentation network performs pixel-level classification on the input image data by fusing multi-scale expanded attention features and feature maps of different levels, and outputs a binarized feature map of pavement disease as the disease extraction result. Step 3: Based on the binarized feature map of the pavement distress, perform morphological operations on the extracted distress areas to connect the fracture areas, and construct a low-dimensional feature vector set using fractal dimension, regional pixel distribution density, and geometric centroid features. Input the low-dimensional feature vector set into the radial basis function probabilistic neural network classification model, and output the distress classification results and distress parameters. The distress parameters include the length and average width of linear cracks, as well as the area of ​​network cracks, potholes, and repairs. Step 4: Based on the disease classification results and disease parameters, automatically calculate the pavement damage index to obtain the pavement technical condition evaluation result; Step 5: Based on the geographic information system, the road surface technical condition evaluation results are correlated with traffic network data and visualized.

[0006] Preferably, the pavement defect segmentation network in step 2 specifically includes a multi-scale expanded attention feature extraction module and a feature fusion upsampling module; The multi-scale expanded attention feature extraction module is used to extract multi-scale semantic features of the input image data under different receptive fields, and to weight the multi-scale semantic features through an attention mechanism to generate a weighted multi-scale feature map. The feature fusion upsampling module is used to connect and fuse high-resolution feature maps in shallow networks with low-resolution feature maps in deep networks, and then apply the attention mechanism again to refine the fused features, finally upsampling to the original resolution of the input image data, and outputting the binarized feature map of the road surface defects.

[0007] Preferably, the multi-scale expanded attention feature extraction module uses a ResNet pre-trained model with an expanded network strategy as the backbone network, and captures the multi-scale semantic information through multiple dilated convolutional layers with different expansion rates; the attention mechanism constructs an attention feature vector for feature recalibration by performing max pooling, sigmoid activation and normalization on the feature map.

[0008] Preferably, in step 3: The fractal dimension is calculated using the box counting method and is used to distinguish between unidirectional cracks, network cracks, pits, and repair-related defects. The pixel distribution density of the region is calculated by the ratio of the number of pixels in the diseased area within the smallest bounding rectangle to the total number of pixels, and is used to distinguish between cracks and blocky cracks. The geometric centroid is used to determine the minimum bounding rectangle of a unidirectional crack, and the structural features of transverse and longitudinal cracks are extracted in combination with the azimuth angle.

[0009] Preferably, step 4, calculating the pavement damage index, further includes: assessing the severity level of each defect based on the average width of the linear cracks and the area of ​​the network cracks, potholes, or repairs; and automatically calculating the pavement damage index based on the defect classification results and severity levels.

[0010] Preferably, it also includes a spatiotemporal evolution analysis step of road surface technical condition: Obtain the results of multiple phases of pavement technical condition evaluation, and integrate historical inspection and evaluation data, design and construction data, geographical environment data, and operation and management data of the road; Based on pavement evaluation units, a spatiotemporal big data analysis model is constructed, and a spatial autoregressive model is used to analyze the spatiotemporal distribution patterns and evolution trends of pavement technical conditions. A pavement distress evolution prediction model is established based on a combination of multiple regression model and grey theory, and the prediction results of future pavement technical conditions are output.

[0011] Preferably, after step 1, an interactive correction step is also included: The binarized feature map of the road surface defects, the defect classification results, and the defect parameters are loaded into the interactive drawing software interface. Receive manual operation instructions to add, delete, or modify the identification and classification results of diseases; The disease parameters are updated based on the corrected results, and the road damage index calculation in step 4 is retried.

[0012] Preferably, the road surface image data is collected by a road surface image acquisition device installed on a mobile acquisition vehicle, the preprocessing, segmentation, classification, and evaluation steps are performed by an edge AI computing device or a cloud server, and the visualization step is performed by a maintenance status visualization system deployed in the cloud.

[0013] This invention also proposes an intelligent detection and evaluation system for pavement defects based on multi-scale data mining, comprising: The image acquisition and preprocessing module is used to acquire road surface image data and perform preprocessing to obtain standardized input image data; The multi-scale disease segmentation module includes a pavement disease segmentation network based on a deep convolutional neural network, which receives the standardized input image data, performs pixel-level classification by fusing multi-scale expanded attention features and feature maps of different levels, and outputs a binarized feature map of pavement disease as the disease extraction result. The disease classification and parameter calculation module is used to receive the binary feature map of the pavement disease, perform morphological operations to connect the fracture areas, and construct a low-dimensional feature vector set using fractal dimension, regional pixel distribution density and geometric centroid features. It outputs the disease classification results and disease parameters through a radial basis probability neural network classification model. The disease parameters include the length and average width of linear cracks, as well as the area of ​​network cracks, potholes and repairs. The pavement condition evaluation module is used to automatically calculate the pavement damage index based on the disease classification results and disease parameters, and obtain the pavement technical condition evaluation results. The visualization and interaction module is used to associate and visualize the pavement technical condition evaluation results with traffic network data based on the geographic information system, and to provide an interactive interface to accept corrections to the disease classification results and parameters.

[0014] Preferably, the intelligent pavement distress detection and evaluation system based on multi-scale data mining also includes: The spatiotemporal evolution analysis module is used to obtain the evaluation results of pavement technical condition in multiple periods, integrate multi-source and multi-dimensional data, construct a spatiotemporal big data analysis model and a combined prediction model based on evaluation units, and output the spatiotemporal evolution law of pavement distress and the prediction results of future technical condition. The system deployment architecture is as follows: the image acquisition and preprocessing module is deployed on a road image acquisition vehicle; the multi-scale disease segmentation module and the disease classification and parameter calculation module are deployed on an edge AI computing device; and the road condition evaluation module, visualization and interaction module, and spatiotemporal evolution analysis module are deployed on a cloud server. The edge AI computing device and the cloud server communicate via 4G or 5G networks.

[0015] The beneficial effects of this invention are: This invention constructs a disease segmentation network by integrating multi-scale dilated convolution and dual attention mechanisms, significantly improving the pixel-level segmentation accuracy of diseases in complex backgrounds and small targets. It also establishes a complete scientific chain from pixel-level segmentation and multi-dimensional feature quantization to radial basis function probabilistic neural network classification, forming an end-to-end automated processing flow from raw image input to PCI index calculation and GIS visualization. Furthermore, it introduces spatiotemporal big data analysis into the field of pavement evaluation to predict disease evolution. An interactive correction mechanism ensures accurate and reliable detection and evaluation results. The loosely coupled interface design between modules supports flexible deployment at the edge and in the cloud, addressing the shortcomings of existing technologies and providing strong data support for scientific highway maintenance decisions. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the database structure according to an embodiment of the present invention; Figure 2This is a schematic diagram of a multi-type pavement distress benchmark dataset according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the visualization results of disease feature mapping at different levels according to an embodiment of the present invention; Figure 4 This is a schematic diagram of a pavement distress extraction network structure according to an embodiment of the present invention; Figure 5 This is a schematic diagram illustrating the distinguishability of disease features according to an embodiment of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Example 1 like Figures 1-5 As shown in the figure, the intelligent detection and evaluation method for road surface defects based on multi-scale data mining according to an embodiment of the present invention includes the following steps: Step 1: Collect road surface image data and preprocess the road surface image data to obtain standardized input image data; Step 2: Input the standardized input image data into the pre-constructed pavement disease segmentation network based on deep convolutional neural network. The pavement disease segmentation network performs pixel-level classification of the input image data by fusing multi-scale expanded attention features and feature maps of different levels, and outputs the binarized feature map of pavement disease as the disease extraction result. Step 3: Based on the binarized feature map of pavement distress, perform morphological operations on the extracted distress areas to connect fracture areas, and construct a low-dimensional feature vector set using fractal dimension, regional pixel distribution density, and geometric centroid features. Input the low-dimensional feature vector set into the radial basis function probabilistic neural network classification model to output the distress classification results and distress parameters. The distress parameters include the length and average width of linear cracks, as well as the area of ​​network cracks, potholes, and repairs. Step 4: Based on the disease classification results and disease parameters, automatically calculate the pavement damage index to obtain the pavement technical condition evaluation results; Step 5: Based on the geographic information system, link and visualize the road surface technical condition evaluation results with traffic network data.

[0019] Preferably, the pavement distress segmentation network in step 2 specifically includes a multi-scale expanded attention feature extraction module and a feature fusion upsampling module; The multi-scale expanded attention feature extraction module is used to extract multi-scale semantic features of input image data under different receptive fields, and to weight the multi-scale semantic features through an attention mechanism to generate a weighted multi-scale feature map. The feature fusion upsampling module is used to connect and fuse high-resolution feature maps in shallow networks with low-resolution feature maps in deep networks, and then apply an attention mechanism to refine the fused features. Finally, it upsamples the data to the original resolution of the input image data and outputs a binarized feature map of road surface defects.

[0020] Preferably, the multi-scale expanded attention feature extraction module uses a ResNet pre-trained model with an expanded network strategy as the backbone network, and captures multi-scale semantic information through multiple dilated convolutional layers with different expansion rates; the attention mechanism constructs attention feature vectors for feature recalibration by performing max pooling, sigmoid activation and normalization on the feature map.

[0021] Preferably, in step 3: Fractal dimension is calculated using the box counting method and is used to distinguish between unidirectional cracks, network cracks, pits, and repair-related defects. The regional pixel distribution density is calculated by the ratio of the number of pixels in the diseased area within the smallest bounding rectangle to the total number of pixels, and is used to distinguish between cracks and blocky cracks. The geometric centroid is used to determine the minimum bounding rectangle of a unidirectional crack, and the structural features of transverse and longitudinal cracks are extracted by combining the azimuth angle.

[0022] Preferably, step 4, calculating the pavement damage index, further includes: assessing the severity level of each defect based on the average width of linear cracks and the area of ​​network cracks, potholes, or repairs; and automatically calculating the pavement damage index based on the defect classification results and severity levels.

[0023] Preferably, it also includes a spatiotemporal evolution analysis step of road surface technical condition: Obtain the results of multiple phases of pavement technical condition evaluation, and integrate historical inspection and evaluation data, design and construction data, geographical environment data, and operation and management data of the road; Based on pavement evaluation units, a spatiotemporal big data analysis model is constructed, and a spatial autoregressive model is used to analyze the spatiotemporal distribution patterns and evolution trends of pavement technical conditions. A pavement distress evolution prediction model is established based on a combination of multiple regression model and grey theory, and the prediction results of future pavement technical conditions are output.

[0024] Preferably, after step 1, an interactive correction step is also included: Load the binary feature map of road surface defects, defect classification results and defect parameters into the interactive delineation software interface; Receive manual operation instructions to add, delete, or modify the identification and classification results of diseases; Update the damage parameters based on the corrected results and re-trigger the road damage index calculation in step 4.

[0025] Preferably, the road surface image data is collected by a road surface image acquisition device installed on a mobile acquisition vehicle, the preprocessing, segmentation, classification, and evaluation steps are performed by an edge AI computing device or a cloud server, and the visualization step is performed by a maintenance status visualization system deployed in the cloud.

[0026] Raw image data of highway surfaces is acquired using road surface image acquisition equipment installed on a mobile acquisition vehicle. This acquisition process needs to cover diverse real-world scenarios: road surface types include asphalt and cement pavements; quality levels include high and low quality; and nine typical scenarios must be considered, including wet, dry, strong light, weak light, blurry, clear, stained, and shadowed conditions. This diverse acquisition strategy aims to construct a high-quality benchmark dataset that balances intra-class diversity and inter-class differences, laying the foundation for the training and evaluation of subsequent deep learning models. The acquired raw images often contain complex background noise such as uneven lighting and low contrast, which severely interferes with the extraction of disease features. Therefore, this invention performs an adaptive uneven lighting compensation algorithm and a low-contrast image enhancement algorithm on the raw images to eliminate or suppress background interference and highlight disease targets. After the above preprocessing, standardized input image data is obtained, denoted as set. ,in The total number of images included in a single detection task, and each image All images are grayscale or color digital images with normalized dimensions and pixel value ranges (e.g., 0-255). At this point, the data stream has completed its mapping from the physical world to the digital space, providing high-quality input for subsequent pixel-level analysis.

[0027] Step 2 aims to transform the standardized image obtained in Step 1. Each pixel in the image is classified as either diseased or non-disease-prone, and a binary feature map with the same size as the input image is output. To achieve this goal, this invention constructs an end-to-end trainable deep convolutional neural network (DCNN), the overall structure of which is as follows: Figure 4 As shown, the network mainly consists of two parts: a multi-scale expanded attention feature extraction module (encoder) and a feature fusion upsampling module (decoder).

[0028] (2.1) Encoding stage: Mathematical principles of multi-scale expanded attention feature extraction module.

[0029] The encoder's backbone network employs a ResNet pre-trained model with an expansion strategy. This expansion strategy introduces dilated convolutions, exponentially increasing the receptive field without resolution loss due to pooling operations, enabling the network to capture a wider range of contextual information. Specifically, this module contains multiple dilated convolutional layers with different dilation rates, each outputting a feature map. ,in These feature maps represent different expansion rates. They capture semantic information about the disease at different scales: small expansion rates correspond to local details (such as the texture of fine cracks), while large expansion rates correspond to global structures (such as the topological morphology of network cracks).

[0030] (2.2) Decoding stage: Mathematical principles and data connection of feature fusion upsampling module.

[0031] Feature maps output during the encoding stage While possessing rich semantic information, the low-resolution feature maps (typically 1 / 8 or 1 / 16 of the original image) have low spatial resolution. Directly using them for pixel-level classification leads to blurred segmentation boundaries and loss of small targets. Therefore, it is necessary to restore the low-resolution semantic features to the original input resolution and, in the process, fuse high-resolution detail information from shallow networks. The feature fusion upsampling module designed in this invention receives two inputs: one is a low-resolution feature map from a deep network. (Right now Secondly, high-resolution feature maps from shallow networks (such as the output of the first or second convolutional block of ResNet). First, concatenate the two feature maps along the channel dimension: Where [;] denotes channel concatenation. Subsequently, batch normalization is used to balance the scale differences between the two feature maps, preventing any one feature from dominating the fusion result. After this, an attention mechanism similar to that used in the encoding stage is applied again: the fused feature maps... Global max pooling, sigmoid activation, and normalization are performed to generate a fused attention vector, which is then processed... Channel-wise weighting is performed. This additional attention step allows for the reselection and combination of features from different levels, further refining the fused features. Finally, the expressive power of the features is enhanced through one or more convolutional layers (e.g., 3×3 convolutions), and upsampling is performed using bilinear interpolation or transposed convolutions to progressively restore the feature map size to the original size of the input image, resulting in the final pixel-level predicted probability map. Its size is The value at each position represents the probability that the pixel belongs to a disease (ranging from 0 to 1).

[0032] (2.3) Loss function and end-to-end training: Mathematical model of cross-entropy loss function and interpretation of its parameters.

[0033] In order for the network to learn the mapping from the input image to the probability map, a loss function needs to be defined to measure the difference between the predicted values ​​and the true labels, and the network weights need to be optimized through backpropagation. This invention employs the cross-entropy loss function, one of the most commonly used loss functions in binary image segmentation tasks. Its mathematical expression is:

[0034] The Chinese explanations of the symbols in the above formula are as follows: This represents all trainable weight matrices of the entire disease segmentation network; This represents the loss value calculated under the current weights. The goal of training is to minimize this loss value using gradient descent. This refers to the total number of pixels in a training batch; subscript It is the pixel index, from 1 to Iterate through each pixel; It is the first The grayscale value of each input pixel (after normalization). It is the first The true category label of each pixel, where 0 represents the background (non-disease) and 1 represents the disease (such as cracks, pits, etc.). It is the network targeting the first The probability value predicted for each pixel to belong to the disease category (i.e., the aforementioned probability map). (The value at the corresponding position in the text). When the actual label When (defective pixels) are considered, the loss term is: If the probability is predicted at this time The closer the prediction is to 1 (correct prediction), the smaller the loss; conversely, if the prediction is less accurate... When the error approaches zero (prediction error), the loss approaches infinity. When the true label... When (background pixels) are present, the loss term is: If the probability is predicted at this time The closer the value is to 0 (correct prediction), the smaller the loss; if it's close to 1 (misclassifying the background as a disease), the loss is large. The loss is minimized over the entire training set. The network gradually learns discriminative features to distinguish between diseases and background. After sufficient training, during the network inference phase, the output probability graph is... Binarization is performed using a threshold (usually 0.5): If If the value is 1 (disease), it is set to 0 (background), thus obtaining the final binary feature map of pavement defects. The binarized feature map is the output of step 2 and the input of step 3, completing the transformation from the original image to the spatial distribution map of the disease.

[0035] Step 3: Classification of various types of pavement defects and assessment of damage status – Mathematical model and data connection from pixel set to structured defect information.

[0036] Binarized feature map output from step 2 Although the locations of the diseased pixels were identified, the specific types of diseases (such as transverse cracks, alligator cracks, pits, etc.) were not distinguished, nor were the geometric parameters of the diseases (length, width, area) quantified. This step aims to address these issues by using morphological preprocessing, multi-dimensional feature extraction, feature vector construction, and probabilistic neural network classification to output structured disease classification results and parameters. The entire data stream gradually transforms from a raw set of binary pixels into physically meaningful information about disease entities.

[0037] (3.1) Morphological pretreatment: connecting the fractured areas.

[0038] In pixel-level segmentation results, due to uncertainties in illumination, noise, or model prediction, originally continuous disease areas may exhibit fine breaks or holes. To obtain complete disease-connected regions for subsequent feature extraction, this invention employs morphological image processing methods. First, dilation is used to expand the boundaries of the disease areas, followed by a closing operation (dilation followed by erosion), which effectively fills narrow breaks and small holes. For wider breaks, block filling and region growing operations are further employed: the binary image is divided into multiple sub-blocks, and adjacent disease areas are connected within each sub-block. Then, starting from a seed point, adjacent pixels with similar features are merged into the same connected region. After the above processing, several complete and connected disease-connected regions are obtained, denoted as . ,in This represents the number of diseased areas detected in a single image.

[0039] (3.2) Selection and extraction of pavement distress features: Construct a low-dimensional feature vector set.

[0040] Different types of pavement distress (unidirectional cracks, network cracks, potholes, and repairs) exhibit significant differences in geometric morphology, texture distribution, and statistical characteristics. This invention selects and extracts the following discriminative features, all derived from connected domains. These features are calculated and used as the low-dimensional vector set for subsequent classification models.

[0041] (3.2.1) Fractal Dimension – Used to distinguish the complexity categories of diseases. Fractal dimension quantifies the complexity and space-filling ability of an irregular geometric shape. Since the irregular fractals of various diseases exhibit statistical self-similarity, this invention employs the box-counting method to calculate the fractal dimension of the diseased region. Specifically, the diseased region... Use different sizes The grid coverage is used to count the number of grid cells that cover the diseased pixels. Box dimension Defined as when hour, The limit, in actual calculations, is determined by... The slope is obtained by fitting a straight line to the point pair. Information dimension. This further considers the probability distribution of diseased pixels in each grid, reflecting the uniformity of disease distribution. For unidirectional cracks (such as transverse or longitudinal cracks), their shape is close to a one-dimensional curve, and the box dimension is... The box dimension is close to 1; for network cracks (crazing, blocky cracks), their shape is close to a two-dimensional surface, with a box dimension close to 2; for pits and repairs, their shape is an approximately circular or elliptical mass, with a box dimension between 1.5 and 2. Therefore, the fractal dimension... These constitute the basic statistical characteristics for distinguishing major categories of diseases (unidirectional, network, pit / repair).

[0042] (3.2.2) Regional pixel distribution density - used to further distinguish between cracks and blocky fissures.

[0043] After identifying the disease as a broad category of network cracks, it is necessary to distinguish between crazing (fine, small-network cracks) and blocky cracks (large, sparse cracks). This distinction cannot be made solely by fractal dimension, as both are near-two-dimensional structures. This invention introduces the structural feature of regional pixel distribution density, and its calculation process is as follows: First, the connected components of the disease are extracted. The geometric centroid (center point) is determined, and then the area is expanded outward from this centroid to determine the smallest bounding rectangle that can completely enclose the diseased area. Within this rectangle, the total number of pixels in the diseased area is counted. And calculate the total number of pixels (sum) within the rectangle (i.e., the width multiplied by the height of the rectangle). Region pixel distribution density. Defined by the following formula

[0044] in, The Chinese explanation for '0' is: the number of pixels belonging to the disease area within the smallest bounding rectangle; the Chinese explanation for 'sum' is: the total number of pixels within the smallest bounding rectangle (including the disease area and the background). For cracked diseases, the cracks are fine and interwoven, and the proportion of disease pixels within the rectangle is relatively high, therefore... A larger value (e.g., greater than 0.4); for blocky cracks, the spacing between cracks is large, and most of the area within the rectangular frame is a complete road surface background, therefore The value is small (e.g., less than 0.2). By setting an empirical threshold (e.g., 0.3), cracks and blocky fissures can be reliably distinguished.

[0045] (3.2.3) Geometric centroid and azimuth angle – used to extract the directional features of unidirectional cracks.

[0046] For diseased areas where the fractal dimension indicates unidirectional cracks, it is necessary to further distinguish between transverse and longitudinal cracks. This invention first extracts the geometric centroid of the diseased area, and then determines the minimum bounding rectangle centered on the centroid. The angle between the longer side of this bounding rectangle and the horizontal direction is the azimuth angle of the crack. (Value range: 0° to 90°). Azimuth angles are typically calculated using principal component analysis or the rotating caliper algorithm. The judgment rule is as follows: If... The crack is then classified as a transverse crack (approximately perpendicular to the direction of travel); if Cracks that are longitudinal (approximately parallel to the direction of travel) are classified as longitudinal cracks. This characteristic allows the disease classification model to output more engineering-meaning directional information.

[0047] (3.3) Calculation of disease parameters: quantification of average crack width and area.

[0048] After the classification is completed, specific quantitative parameters for the Road Surface Damage Index (PCI) assessment need to be calculated.

[0049] (3.3.1) Mathematical model for average crack width. For linear cracks, their severity is mainly graded based on their average width. This invention uses the skeleton method to calculate the average width. First, the affected area... A thinning operation is performed, reducing it to a skeleton line of single-pixel width that preserves the topology of the crack. Let... This represents the total number of crack pixels in the original diseased area. This represents the total number of pixels in the crack skeleton line. Since the crack shape is approximately a narrow strip, its area (total number of pixels) is approximately equal to its length (number of skeleton pixels) multiplied by its average width. Therefore, the average width of the crack (in pixels) is given by the following formula:

[0050] in, The Chinese explanation is: the binary feature map output in step 2. In the context of the crack, the total number of pixels with a value of 1 within the connected region. The Chinese explanation is: the total number of pixels forming the skeleton lines after skeletonizing the connected components of the crack. Because... This approximates the length of the crack, so this ratio represents the average width at the pixel level. To convert the pixel width to a physically meaningful (millimeters) actual width, a spatial resolution parameter for the road surface image needs to be introduced. (mm / pixel). This parameter is obtained in advance through camera calibration and represents the actual road surface size corresponding to each pixel in the image. Final average width Calculate using the following formula:

[0051] The Chinese explanation is: the average physical width of a crack, measured in millimeters. According to... The numerical range (e.g., <3mm for mild, 3-5mm for moderate, >5mm for severe) can be used to automatically assess the severity level of the crack.

[0052] (3.3.2) Calculation of area parameters. For surface defects such as network cracks, pits, and repairs, the total number of pixels in the connected region of the binarized feature map is directly counted. Then multiply by the square of the spatial resolution to get the actual physical area: Area The unit is square meters.

[0053] (3.4) Radial basis probabilistic neural network classification: Outputs the final disease classification results and parameters. The fractal dimension extracted above... Regional distribution density Azimuth (Valid only for unidirectional cracks), after normalizing features such as average width or area, they are combined into a low-dimensional feature vector set. [Others]. This vector set is used as input and fed into a pre-trained Radial Basis Function Probabilistic Neural Network (RBFPNN) classification model. RBF-PNN is a feedforward neural network with advantages such as fast training speed, robustness to noise, and the ability to output posterior probabilities. The output layer nodes of the network correspond to disease categories: Horizontal cracks, longitudinal cracks, crazing, crater cracks, pits, repair At the same time, the network also outputs disease parameters associated with this category. (The length of the crack is calculated by multiplying the total number of skeleton pixels by) (obtained), average width (e.g., area, etc.). At this point, step 3 completes the transformation from pixel-level binary images to structured disease information, and the output result is denoted as... ,in Number of diseased areas in the image Step 4: Automatic calculation of road surface damage index – from damage parameters to technical condition evaluation.

[0054] After obtaining the classification results and parameters of all defects in a single image or a road segment, this step automatically calculates the Pavement Condition Index (PCI) according to national or industry standards (such as the "Highway Technical Condition Assessment Standard" JTG5210). The calculation process includes: First, finding the deductible value corresponding to each defect based on the defect category and severity level (determined by parameters such as width and area against thresholds in the standard); then, accumulating the deductible values ​​for all defects within a unit road segment and applying a reduction factor; finally, according to the formula... The total penalty score is calculated. When the total penalty score exceeds 100, 0 is taken as the lower limit. The higher the calculated PCI value, the better the road surface condition. Generally, PCI > 90 is excellent, 80-90 is good, 70-80 is average, 60-70 is poor, and < 60 is poor. The final output is the road surface technical condition evaluation result. The results are linked to road segment markers (such as station numbers) to provide a basis for subsequent visualization and decision-making.

[0055] Step 5: Visualization and Spatiotemporal Evolution Analysis – From Single Evaluation to Intelligent Decision-Making.

[0056] (5.1) GIS-based visualization.

[0057] This step, based on a Geographic Information System (GIS) platform, spatially correlates the road segment-level PCI evaluation results calculated in step 4 with traffic network data. First, a linear reference system for road inspection and evaluation is constructed, mapping pavement technical condition data to mileage markers on the road. Then, by publishing a vector tile dataset, segmented rendering of PCI values ​​is implemented on the map: different colors represent different PCI levels (e.g., green for excellent, red for poor). Users can interact with the map to perform queries, statistics, and historical data comparisons, intuitively grasping the pavement health status of the road network.

[0058] (5.2) Spatiotemporal evolution analysis.

[0059] To extract patterns from historical data and predict future pavement performance, this invention further introduces spatiotemporal big data analysis technology. It acquires pavement technical condition evaluation results from multiple periods (multiple years, multiple quarters) and integrates historical road inspection data, design and construction data (such as base course type and surface layer thickness), geographical environmental data (such as precipitation and traffic volume), and operation and management data (such as maintenance history). Based on fixed-length evaluation units (such as 100 meters or 1 kilometer), a spatiotemporal big data analysis model is constructed. A spatial autoregressive model (such as a spatial lag model or a spatial error model) is used to analyze the spatiotemporal distribution patterns of pavement technical conditions, identifying high-incidence sections of pavement defects and their intrinsic correlation with geographical and environmental factors. Furthermore, based on the concept of combined prediction, a multivariate regression model (used to fit macro trends) is combined with a grey theory model (used to handle small samples and uncertain information) to establish a spatiotemporal evolution combined prediction model for pavement technical conditions. This model can output predictions of future (next year) pavement technical conditions based on current and historical data. The accuracy of the predictions was verified through approximation fitting, and the weight coefficients of the multiple regression model and the grey model were dynamically adjusted. Finally, the predictions were also overlaid on a GIS map to provide direct data support for scientific decision-making in highway maintenance (such as the preparation of annual maintenance plans and the determination of the timing of preventive maintenance).

[0060] Interactive correction steps: To improve the reliability of automatic detection results, especially in boundary cases or complex scenarios, this invention provides a closed-loop human-computer interaction mechanism. After step 2 or 3, the binarized feature map of pavement distress, distress classification results, and distress parameters are loaded into the interactive delineation software interface. This software interface supports functions such as batch preprocessing (checking data integrity), loading project files, loading and displaying pavement images, manual delineation and modification of distress areas (adding, deleting, and modifying distress boundaries or classification labels), and exporting distress information. After the user completes the corrections, the software automatically updates the distress parameters based on the corrected results. This interactive inspection module re-triggers the road damage index calculation in step 4, ensuring the reliability and accuracy of the final PCI index calculation. It is particularly suitable for maintenance project audit scenarios with high precision requirements.

[0061] Example 2 like Figures 1-5 As shown, this invention also proposes an intelligent detection and evaluation system for pavement defects based on multi-scale data mining, comprising: The image acquisition and preprocessing module is used to acquire road surface image data and perform preprocessing to obtain standardized input image data; The multi-scale disease segmentation module includes a pavement disease segmentation network based on a deep convolutional neural network. It receives standardized input image data, performs pixel-level classification by fusing multi-scale expanded attention features and feature maps of different levels, and outputs a binarized feature map of pavement disease as the disease extraction result. The disease classification and parameter calculation module is used to receive the binary feature map of pavement disease, perform morphological operations to connect fracture areas, and construct a low-dimensional feature vector set using fractal dimension, regional pixel distribution density and geometric centroid features. It outputs disease classification results and disease parameters through a radial basis probability neural network classification model. Disease parameters include the length and average width of linear cracks, as well as the area of ​​network cracks, potholes and repairs. The pavement condition evaluation module is used to automatically calculate the pavement damage index based on the damage classification results and damage parameters, and obtain the pavement technical condition evaluation results. The visualization and interaction module is used to associate and visualize the road surface technical condition evaluation results with traffic network data based on the geographic information system, and provides an interactive interface to accept corrections to the disease classification results and parameters.

[0062] Preferably, the intelligent pavement distress detection and evaluation system based on multi-scale data mining also includes: The spatiotemporal evolution analysis module is used to obtain the evaluation results of pavement technical condition in multiple periods, integrate multi-source and multi-dimensional data, construct a spatiotemporal big data analysis model and a combined prediction model based on evaluation units, and output the spatiotemporal evolution law of pavement distress and the prediction results of future technical condition. The system deployment architecture consists of an image acquisition and preprocessing module deployed on a road image acquisition vehicle, a multi-scale disease segmentation module and a disease classification and parameter calculation module deployed on an edge AI computing device, and a road condition evaluation module, a visualization and interaction module, and a spatiotemporal evolution analysis module deployed on a cloud server. The edge AI computing device and the cloud server communicate via 4G or 5G networks.

[0063] The system of this invention includes: an image acquisition and preprocessing module, a multi-scale disease segmentation module, a disease classification and parameter calculation module, a pavement condition evaluation module, a visualization and interaction module, and an optional spatiotemporal evolution analysis module. The functions of each module completely correspond to the steps in Embodiment 1, enabling the execution of the entire process from image acquisition to final evaluation and prediction. Data flow between modules adopts a loosely coupled interface design, supporting flexible deployment at the edge and in the cloud. The system hardware includes pavement image acquisition equipment, a numerical control (PLC) device, an edge AI computing device (GPU computing unit), a tablet client, and an industrial control server; the software includes a PLC counting program, an industrial control server program, a tablet client app, a deep learning inference engine on the edge AI computing device, and a cloud-based maintenance condition visualization system. The subsystems communicate with each other via wireless networks (Wi-Fi, 4G / 5G) to collaboratively complete the intelligent detection and evaluation of pavement diseases.

[0064] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0065] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0066] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0067] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0068] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0069] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0070] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for intelligent detection and evaluation of pavement defects based on multi-scale data mining, characterized in that, Includes the following steps: Step 1: Collect road surface image data and preprocess the road surface image data to obtain standardized input image data; Step 2: Input the standardized input image data into a pre-constructed pavement disease segmentation network based on a deep convolutional neural network. The pavement disease segmentation network performs pixel-level classification on the input image data by fusing multi-scale expanded attention features and feature maps of different levels, and outputs a binarized feature map of pavement disease as the disease extraction result. Step 3: Based on the binarized feature map of the pavement distress, perform morphological operations on the extracted distress areas to connect the fracture areas, and construct a low-dimensional feature vector set using fractal dimension, regional pixel distribution density, and geometric centroid features. Input the low-dimensional feature vector set into the radial basis function probabilistic neural network classification model, and output the distress classification results and distress parameters. The distress parameters include the length and average width of linear cracks, as well as the area of ​​network cracks, potholes, and repairs. Step 4: Based on the disease classification results and disease parameters, automatically calculate the pavement damage index to obtain the pavement technical condition evaluation result; Step 5: Based on the geographic information system, the road surface technical condition evaluation results are correlated with traffic network data and visualized.

2. The intelligent detection and evaluation method for pavement defects based on multi-scale data mining according to claim 1, characterized in that, The road surface defect segmentation network in step 2 specifically includes a multi-scale expanded attention feature extraction module and a feature fusion upsampling module; The multi-scale expanded attention feature extraction module is used to extract multi-scale semantic features of the input image data under different receptive fields, and to weight the multi-scale semantic features through an attention mechanism to generate a weighted multi-scale feature map. The feature fusion upsampling module is used to connect and fuse high-resolution feature maps in shallow networks with low-resolution feature maps in deep networks, and then apply the attention mechanism again to refine the fused features, finally upsampling to the original resolution of the input image data, and outputting the binarized feature map of the road surface defects.

3. The intelligent detection and evaluation method for pavement defects based on multi-scale data mining according to claim 2, characterized in that, The multi-scale expanded attention feature extraction module uses a ResNet pre-trained model with an expanded network strategy as the backbone network, and captures the multi-scale semantic information through multiple dilated convolutional layers with different expansion rates; the attention mechanism constructs an attention feature vector for feature recalibration by performing max pooling, sigmoid activation and normalization on the feature map.

4. The intelligent detection and evaluation method for pavement defects based on multi-scale data mining according to claim 1, characterized in that, In step 3: The fractal dimension is calculated using the box counting method and is used to distinguish between unidirectional cracks, network cracks, pits, and repair-related defects. The pixel distribution density of the region is calculated by the ratio of the number of pixels in the diseased area within the smallest bounding rectangle to the total number of pixels, and is used to distinguish between cracks and blocky cracks. The geometric centroid is used to determine the minimum bounding rectangle of a unidirectional crack, and the structural features of transverse and longitudinal cracks are extracted in combination with the azimuth angle.

5. The intelligent detection and evaluation method for pavement defects based on multi-scale data mining according to claim 1, characterized in that, Step 4, calculating the pavement damage index, further includes: assessing the severity level of each defect based on the average width of the linear cracks and the area of ​​the network cracks, potholes, or repairs; and automatically calculating the pavement damage index based on the defect classification results and severity levels.

6. The intelligent detection and evaluation method for pavement defects based on multi-scale data mining according to claim 1, characterized in that, It also includes the spatiotemporal evolution analysis steps of road surface technical condition: Obtain the results of multiple phases of pavement technical condition evaluation, and integrate historical inspection and evaluation data, design and construction data, geographical environment data, and operation and management data of the road; Based on pavement evaluation units, a spatiotemporal big data analysis model is constructed, and a spatial autoregressive model is used to analyze the spatiotemporal distribution patterns and evolution trends of pavement technical conditions. A pavement distress evolution prediction model is established based on a combination of multiple regression model and grey theory, and the prediction results of future pavement technical conditions are output.

7. The intelligent detection and evaluation method for pavement defects based on multi-scale data mining according to claim 1, characterized in that, Following step 1, there is also an interactive correction step: The binarized feature map of the road surface defects, the defect classification results, and the defect parameters are loaded into the interactive drawing software interface. Receive manual operation instructions to add, delete, or modify the identification and classification results of diseases; The disease parameters are updated based on the corrected results, and the road damage index calculation in step 4 is retried.

8. The intelligent detection and evaluation method for pavement defects based on multi-scale data mining according to claim 1, characterized in that, The road surface image data is collected by a road surface image acquisition device installed on a mobile acquisition vehicle. The preprocessing, segmentation, classification, and evaluation steps are performed by an edge AI computing device or a cloud server. The visualization step is performed by a maintenance status visualization system deployed in the cloud.

9. A pavement distress intelligent detection and evaluation system based on multi-scale data mining, characterized in that, include: The image acquisition and preprocessing module is used to acquire road surface image data and perform preprocessing to obtain standardized input image data; The multi-scale disease segmentation module includes a pavement disease segmentation network based on a deep convolutional neural network, which receives the standardized input image data, performs pixel-level classification by fusing multi-scale expanded attention features and feature maps of different levels, and outputs a binarized feature map of pavement disease as the disease extraction result. The disease classification and parameter calculation module is used to receive the binary feature map of the pavement disease, perform morphological operations to connect the fracture areas, and construct a low-dimensional feature vector set using fractal dimension, regional pixel distribution density and geometric centroid features. It outputs the disease classification results and disease parameters through a radial basis probability neural network classification model. The disease parameters include the length and average width of linear cracks, as well as the area of ​​network cracks, potholes and repairs. The pavement condition evaluation module is used to automatically calculate the pavement damage index based on the disease classification results and disease parameters, and obtain the pavement technical condition evaluation results. The visualization and interaction module is used to associate and visualize the pavement technical condition evaluation results with traffic network data based on the geographic information system, and to provide an interactive interface to accept corrections to the disease classification results and parameters.

10. The intelligent detection and evaluation system for road surface defects based on multi-scale data mining according to claim 9, characterized in that, Also includes: The spatiotemporal evolution analysis module is used to obtain the evaluation results of pavement technical condition in multiple periods, integrate multi-source and multi-dimensional data, construct a spatiotemporal big data analysis model and a combined prediction model based on evaluation units, and output the spatiotemporal evolution law of pavement distress and the prediction results of future technical condition. The system deployment architecture is as follows: the image acquisition and preprocessing module is deployed on a road image acquisition vehicle; the multi-scale disease segmentation module and the disease classification and parameter calculation module are deployed on an edge AI computing device; and the road condition evaluation module, visualization and interaction module, and spatiotemporal evolution analysis module are deployed on a cloud server. The edge AI computing device and the cloud server communicate via 4G or 5G networks.

Images