AI-based road surface disease identification decision method and related device

Abstract

The application provides an AI-based road surface disease identification decision method and related device. The method comprises: acquiring a road surface aerial survey image of a road surface to be identified; extracting a plurality of disease feature maps of disease feature dimensions from the road surface aerial survey image; taking a first convolution matrix as a feature extraction control and a first moving step as a feature extraction frequency, extracting a feature sub-map for each disease feature map to obtain a first disease feature sub-map set corresponding to each disease feature map; performing feature interaction processing on each first disease feature sub-map set, and fusing a plurality of interaction feature sub-maps obtained through feature interaction to obtain a road surface disease feature map of the road surface to be identified; and determining a disease identification result of the road surface to be identified according to the road surface disease feature map. The application can prevent the problem of insufficient disease identification accuracy caused by local feature loss, and improve the reliability of disease identification.

Description

Technical Field

[0001] The present application relates to the field of data processing, and more specifically, to an AI-based pavement disease identification and decision-making method and related devices. Background Art

[0002] With the continuous acceleration of urbanization and the booming transportation industry, the quality and safety of road surfaces are becoming increasingly important. Pavement defects not only affect road service life and driving comfort but can also cause traffic accidents. Therefore, timely and accurate identification of pavement defects and effective repair measures are crucial.

[0003] Currently, traditional methods for identifying road surface defects rely primarily on manual inspections. This approach is not only inefficient and labor-intensive, but also subject to significant subjective influences, making it difficult to ensure the accuracy and consistency of identification results. Furthermore, manual inspections cannot quickly and comprehensively inspect large areas of road surface, making it difficult to detect potential defects in a timely manner.

[0004] While some pavement defect recognition technologies based on image recognition have improved recognition efficiency to a certain extent, they still suffer from issues such as incomplete feature extraction, local feature overload, and insufficient recognition accuracy. Therefore, there is an urgent need for an accurate and intelligent pavement defect recognition and decision-making method to improve the accuracy and reliability of pavement defect identification and provide scientific and reasonable decision-making support for pavement maintenance and management. Summary of the Invention

[0005] The purpose of this application is to provide an AI-based road surface disease identification and decision-making method and related devices. The embodiment of this application is implemented as follows:

[0006] In a first aspect, an embodiment of the present application provides an AI-based pavement defect identification and decision-making method, the method comprising: obtaining an aerial survey image of a pavement to be identified; extracting a defect feature map of multiple defect feature dimensions from the aerial survey image; performing feature sub-map extraction on each of the defect feature maps using a first convolution matrix as a feature extraction control and a first moving step as a feature extraction frequency to obtain a first defect feature sub-map set corresponding to each of the defect feature maps; performing feature interaction processing on each of the first defect feature sub-map sets, and fusing multiple interactive feature sub-maps obtained by feature interaction to obtain a pavement defect feature map of the pavement to be identified; and determining a defect identification result of the pavement to be identified based on the pavement defect feature map.

[0007] In a second aspect, the present application provides an AI-based road surface disease identification and decision-making device, comprising:

[0008] An image acquisition module, used to acquire an aerial survey image of the road surface to be identified;

[0009] a feature extraction module configured to extract a plurality of defect feature dimensions of a defect feature map from the pavement aerial survey image; perform feature sub-map extraction on each of the defect feature maps using a first convolution matrix as a feature extraction control and a first moving step as a feature extraction frequency, and obtain a first set of defect feature sub-maps corresponding to each of the defect feature maps;

[0010] a feature interaction module, configured to perform feature interaction processing on each of the first defect feature subgraph sets, and fuse multiple interactive feature subgraphs obtained through feature interaction to obtain a pavement defect feature graph of the road surface to be identified;

[0011] The disease identification module is used to determine the disease identification result of the road surface to be identified based on the road disease characteristic map.

[0012] The AI-based pavement defect identification and decision-making method provided in the embodiment of the present application obtains a pavement aerial survey image of the pavement to be identified, extracts defect feature maps of multiple defect feature dimensions from the pavement aerial survey image, and converts the extracted multiple defect feature maps into multiple disease feature maps. A first convolution matrix is ​​used as a feature extraction control, and a first moving step is used as a feature extraction frequency. Feature sub-graph extraction is performed on each defect feature map to obtain a first disease feature sub-graph set corresponding to each disease feature map. Feature interaction processing is performed on each first disease feature sub-graph set, and multiple interactive feature sub-graphs obtained by feature interaction are fused to obtain a pavement defect feature map of the pavement to be identified. The disease identification result of the pavement to be identified is determined based on the pavement disease feature map.

[0013] This application extracts multiple features from the disease feature graph in each disease feature dimension through feature subgraph extraction, thereby obtaining a feature set with higher information richness in each disease feature dimension. This prevents the problem of insufficient disease recognition accuracy caused by the loss of local features and improves the reliability of disease recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is a flowchart of an AI-based pavement damage identification and decision-making method provided in an embodiment of the present application.

[0015] Figure 2 This is a schematic diagram of the composition of an AI-based road hazard identification and decision-making device provided in an embodiment of the present application.

[0016] Figure 3 This is a schematic diagram of the composition of a computer system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0017] The AI-based pavement defect identification and decision-making method in the embodiments of this application is implemented by a computer system, including but not limited to a server, personal computer, laptop computer, tablet computer, smartphone, etc. This computer system can operate independently to implement this application, or it can be connected to a network and interact with other computer systems in the network to implement this application. The network in which the computer system resides includes but is not limited to the Internet, wide area network, metropolitan area network, local area network, VPN network, etc.

[0018] like Figure 1 As shown, the AI-based pavement damage identification and decision-making method provided in the embodiment of the present application includes:

[0019] Step 100: Acquire an aerial survey image of the road surface to be identified;

[0020] Step 200: extracting a damage feature map of multiple damage feature dimensions from the road surface aerial survey image.

[0021] Obtaining aerial images of the road surface to be identified can be accomplished with the help of aerial photography equipment, such as high-definition cameras mounted on drones. These drones capture the road surface from the air, producing images that contain comprehensive road surface information. For example, a drone can be controlled to fly at a specific altitude and route along a main urban road to capture the road, producing a high-resolution aerial image. These images contain various road surface information, such as color, texture, and shape, providing essential data for subsequent defect identification.

[0022] Defect feature dimensions describe different aspects of pavement defects, such as the length, width, and direction of cracks, and the depth and area of ​​potholes. These dimensions can be selected based on actual needs. Image processing and machine learning algorithms can be used to extract these features. For example, edge detection algorithms, such as the Canny edge detection algorithm, can be used to detect crack edges in an image, thereby determining their location and general shape. The width of a crack can be determined by calculating the pixel distance between edges. For example, if a crack is detected in an aerial road survey image using the Canny edge detection algorithm, the pixel distance between edges can be converted to its actual physical width based on the image's pixel resolution.

[0023] When extracting a defect feature map, different defect feature dimensions can be processed separately. For example, for pothole features, morphological operations such as dilation and erosion can be used to determine the pothole's boundaries and area. Specifically, the image is first binarized to separate the pothole area from the background. Then, the pothole area is expanded using dilation, and then erosion is used to remove noise and small connected areas. Finally, the area of ​​the remaining area is calculated, which is the area of ​​the pothole.

[0024] Deep learning models can be used to extract complex damage features. For example, a convolutional neural network (CNN) can be used to identify the type and severity of pavement damage. CNNs automatically learn features from images. By training on a large amount of pavement damage image data, the model can accurately identify different types of damage, such as cracks, potholes, and rutting, and assign corresponding severity scores.

[0025] Through steps 100 and 200, rich disease feature information can be extracted from the aerial road survey image, providing accurate data support for subsequent disease identification and decision-making. These feature maps contain multi-dimensional information about road diseases, helping to improve the accuracy and reliability of disease identification.

[0026] Step 300: Using the first convolution matrix as a feature extraction control and the first moving step as a feature extraction frequency, extract feature subgraphs from each disease feature graph to obtain a first disease feature subgraph set corresponding to each disease feature graph.

[0027] The first convolution matrix is ​​a two-dimensional matrix consisting of a set of weight values. It is used to perform convolution operations on images in image processing. Convolution is a linear operation that extracts local features of the image by sliding the convolution matrix over the image, multiplying it element-wise with the local area of ​​the image, and summing the results.

[0028] The first movement stride determines the interval at which the convolution matrix slides across the image. For example, if the first movement stride is 1, the convolution matrix shifts one pixel to the right or downward on the image at a time; if the first movement stride is 2, it shifts two pixels at a time. The choice of the first movement stride affects the number and size of feature subgraphs. A larger stride results in fewer feature subgraphs extracted, and each feature subgraph contains more macroscopic information. A smaller stride results in more feature subgraphs extracted, and each feature subgraph contains more detailed information.

[0029] When executing step 300, first, for each disease feature map, place the first convolution matrix in the upper left corner of the disease feature map. Then, according to the first movement stride, slide the first convolution matrix from left to right and from top to bottom in sequence to perform convolution operations on each local area of ​​the disease feature map. Assuming that the size of the disease feature map is M×N, the size of the first convolution matrix is ​​m×n, and the first movement stride is s, then in the horizontal direction, the number of times the convolution matrix can slide is ; In the vertical direction, the number of times the convolution matrix can slide is .in, Indicates a floor operation.

[0030] After each convolution operation, a convolution result is obtained, which can be considered a feature representation of the local area of ​​the damage feature map. By combining all convolution results, we can obtain the first set of damage feature subgraphs corresponding to the damage feature map. For example, for a damage feature map representing the width of a pavement crack, a 3×3 first convolution matrix and a first shift stride of 1 are used to extract the feature subgraph. When the convolution matrix slides over a local area of ​​the crack, the convolution operation can extract the changing characteristics of the crack width in that area, and these characteristics are stored in the first set of damage feature subgraphs.

[0031] In practical applications, the weight values ​​of the first convolution matrix are obtained through training, for example. A large amount of pavement disease image data can be used for training, and the weight values ​​of the first convolution matrix can be continuously adjusted through an optimization algorithm so that the extracted feature subgraphs can better represent the characteristics of the pavement disease. For example, the backpropagation algorithm can be used to update the weight values ​​of the first convolution matrix to minimize the loss function of the training data. In addition, in order to improve the efficiency and accuracy of feature extraction, some preprocessing operations such as normalization and filtering can be performed on the first disease feature subgraph set. The normalization operation can map the pixel values ​​of the feature subgraph to a specific range, so that different feature subgraphs are comparable; the filtering operation can remove noise in the feature subgraph and improve the quality of the features.

[0032] Step 400: performing feature interaction processing on each first defect feature subgraph set, and fusing multiple interactive feature subgraphs obtained by the feature interaction to obtain a pavement defect feature graph of the road surface to be identified.

[0033] The purpose of step 400 is to integrate the information of each disease feature dimension and explore the relationship between the features, so as to obtain a more representative and discriminative pavement disease feature representation.

[0034] During the feature interaction processing stage, it is necessary to consider the interrelationships between different defect feature dimensions. Through specific algorithms and operations, the features within each first defect feature subgraph set can communicate and influence each other. Taking the two defect feature dimensions of pavement cracks and potholes as an example, the presence of cracks may affect the development of potholes, and the size and location of potholes may also be related to the distribution of cracks. Feature interaction processing can be used to capture these potential correlations. In specific implementations, the attention mechanism within a neural network can be used to assign different weights to the features within the first defect feature subgraph set, highlighting important feature information and suppressing irrelevant or noisy information. For example, for a first defect feature subgraph representing crack length and a first defect feature subgraph representing pothole depth, the attention mechanism can assign different weights based on their importance to defect identification, allowing subsequent processing to focus more on features that contribute more to defect identification.

[0035] When fusing multiple interactive feature subgraphs obtained by feature interaction, one fusion method is weighted summation, that is, assigning a weight to each interactive feature subgraph according to its importance, and then linearly combining all interactive feature subgraphs according to the weight. Suppose there are n interactive feature subgraphs , and the corresponding weights are , then the fused pavement disease characteristic map F can be expressed as ,in These weights can be determined based on a variety of factors, such as the correlation between different defect feature dimensions and the pavement type of the road to be identified. For highways, crack characteristics may be more important for defect identification, so the weight of the interactive feature subgraph representing crack characteristics may be relatively large. Through step 400, the information of each defect feature dimension can be effectively integrated and interacted, eliminating redundant information, enhancing useful information, and obtaining a comprehensive pavement defect feature map.

[0036] Step 500: Determine a road disease identification result for the road to be identified based on the road disease characteristic map.

[0037] Specifically, the pavement defect feature map is first decoded and classified. In practice, the pavement defect feature map is input into a classification model trained on a large amount of data. This classification model can be a convolutional neural network (CNN), a support vector machine (SVM), or other models. Taking a convolutional neural network as an example, it consists of multiple convolutional layers, pooling layers, and fully connected layers. The convolutional layer slides the convolution kernel across the pavement defect feature map to extract local features of the image; the pooling layer performs dimensionality reduction on the extracted features to reduce computational complexity; and the fully connected layer integrates the extracted features and outputs the classification results. During the training process, the model learns the characteristic patterns corresponding to different defect types. By continuously adjusting the model parameters, the model can accurately classify pavement defects.

[0038] During the classification process, a confidence score is calculated for each possible defect type based on the model's output. The confidence score indicates the likelihood that the pavement has that defect type. For example, for three common pavement defect types—cracks, potholes, and rutting—the confidence score for cracks might be 0.8, for potholes 0.1, and for rutting 0.1. This means that cracks are the most likely defect type.

[0039] To determine the final defect identification result, a confidence threshold is set. When the confidence score for a particular defect type exceeds this threshold, that defect type is considered a road surface defect. For example, if the confidence threshold is set to 0.5, then in the above example, cracks would be considered a road surface defect type.

[0040] In addition to determining the disease type, the severity of the disease can also be evaluated according to the characteristic information of the road surface disease feature map. For example, for crack disease, the crack disease can be divided into three levels of mild, moderate and severe according to the length, width and depth of the crack and other characteristics. The specific evaluation method can be realized by establishing a mapping relationship between the characteristics and the severity of the disease. For example, a crack length less than 10 cm and a width less than 0.5 cm can be set as a mild crack; a crack length between 10-50 cm and a width between 0.5-1 cm can be set as a moderate crack; and a crack length greater than 50 cm and a width greater than 1 cm can be set as a severe crack.

[0041] In an implementation scheme, step 400, the feature interaction processing is performed on each first disease feature subgraph set, and the plurality of interaction feature subgraphs obtained by the feature interaction processing are fused to obtain a road surface disease feature map of the road surface to be recognized, including:

[0042] Step 410: determining a first image processing algorithm corresponding to each first disease feature subgraph set.

[0043] Different first image processing algorithms correspond to different disease feature dimensions, because different disease features have different characteristics and forms, and appropriate algorithms need to be used for feature interaction processing to fully exploit and utilize these feature information. For example, for the first disease feature subgraph set of the road surface crack, since the crack has the characteristics of linearity and continuity, a first image processing algorithm based on edge detection and morphological operation can be selected, which can highlight the edge features of the crack and better capture the shape and direction of the crack. For the first disease feature subgraph set of the road surface pit, considering that the pit has certain area and depth information, a first image processing algorithm based on region segmentation and three-dimensional reconstruction can be used to accurately determine the size, depth and position of the pit. Generally, a mapping relationship between the disease feature dimension and the first image processing algorithm is established according to historical data and experience, and in actual application, the corresponding first image processing algorithm is found from the mapping relationship according to the current disease feature dimension.

[0044] Step 420: inputting the first disease feature subgraph set corresponding to each disease feature map into the corresponding first image processing algorithm for feature interaction processing to obtain the interaction feature subgraph corresponding to each disease feature map.

[0045] During this process, the first image processing algorithm analyzes, compares, and combines the individual feature subgraphs in the first defect feature subgraph set, exploring the inherent connections and interactions between them, thereby generating interactive feature subgraphs with higher information content and greater representativeness. Taking the first image processing algorithm based on a convolutional neural network as an example, the algorithm performs a convolution operation on the first defect feature subgraph set, extracting local features from the feature subgraphs using different convolution kernels. These local features are then combined and transformed to form interactive feature subgraphs. Specifically, assuming that one feature subgraph in the first defect feature subgraph set represents the texture information of the road surface, and another feature subgraph represents the color information of the road surface, the convolutional neural network performs a convolution operation on these two feature subgraphs, extracting the correlation features between texture and color, and forming an interactive feature subgraph. Such an interactive feature subgraph can more comprehensively reflect the characteristics of road defects, providing a more accurate basis for subsequent defect identification.

[0046] Step 430: Perform feature fusion on multiple interactive feature sub-graphs to obtain a pavement disease feature graph of the road surface to be identified.

[0047] The goal of feature fusion is to integrate interactive feature subgraphs from different defect feature dimensions, eliminating redundant information and enhancing useful information, thereby obtaining a comprehensive and more representative pavement defect feature map. First, the importance of each interactive feature subgraph in the fusion process must be determined, namely its first impact factor (FIF). This factor reflects the contribution of the interactive feature subgraph to the final pavement defect feature map. Next, a weighted calculation is performed on each interactive feature subgraph based on the FIF to produce multiple weighted feature subgraphs, ensuring that important interactive feature subgraphs receive greater attention and representation in the fusion process. Finally, these weighted feature subgraphs are fused, for example using a weighted summation method. The pixel values ​​at corresponding locations in all weighted feature subgraphs are added according to their weights to produce the final pavement defect feature map. This feature fusion process effectively integrates information from different defect feature dimensions, providing a more accurate and comprehensive feature representation for subsequent defect identification, thereby improving the accuracy and reliability of pavement defect identification.

[0048] In one implementation, step 430 , performing feature fusion on multiple interactive feature subgraphs to obtain a pavement damage feature graph of the road to be identified, includes:

[0049] Step 431: Determine the first influencing factor of each interactive feature subgraph; wherein, the process of determining the first influencing factor can be: obtaining the pavement type of the road surface to be identified; and determining the first influencing factor of each interactive feature subgraph based on the relevant information of each disease characteristic dimension and the pavement type of the road surface to be identified.

[0050] In step 431, the first impact factor of each interactive feature subgraph is determined. This first impact factor reflects the importance of each interactive feature subgraph in the final fusion of the pavement defect feature graph. The determination process depends on the pavement type of the road to be identified and the correlation between each defect feature dimension and that pavement type. First, the pavement type of the road to be identified is obtained. There are many types of pavement, such as asphalt and cement pavements, and different types of pavements have different defect manifestations and characteristics. For asphalt pavements, crack characteristics may be a key factor in determining defects; for cement pavements, plate damage characteristics may be more important. Next, the first impact factor is determined based on the correlation between each defect feature dimension and the pavement type of the road to be identified. Assume that the road to be identified is an asphalt pavement and there are two interactive feature subgraphs: one for crack characteristics and the other for rutting characteristics. Since cracks have a greater impact on pavement defects in asphalt pavements, while rutting has a relatively smaller impact, the interactive feature subgraph for crack characteristics will be assigned a higher first impact factor, such as 0.7, and the interactive feature subgraph for rutting characteristics will be assigned a lower first impact factor, such as 0.3. A database can be established to store the importance information of each disease characteristic dimension under different road surface types. The corresponding importance coefficient can be found from the database according to the type of road surface to be identified and the disease characteristic dimension, and used as the first influencing factor.

[0051] Step 432: Perform weighted calculation on each interactive feature subgraph according to the first impact factor to obtain a plurality of weighted feature subgraphs.

[0052] In step 432, a weighted calculation is performed on each interaction feature subgraph based on the first influencing factor to obtain multiple weighted feature subgraphs. Weighted calculation involves multiplying the interaction feature subgraph by the corresponding first influencing factor. This emphasizes the information of important interaction feature subgraphs while weakening the influence of unimportant ones. This weighted calculation allows important defect characteristics to be more fully reflected in the subsequent fusion process, helping to improve the accuracy and reliability of the final pavement defect feature map.

[0053] Step 433: Perform feature fusion on multiple weighted feature sub-graphs to obtain a pavement disease feature graph of the road surface to be identified.

[0054] The goal of feature fusion is to integrate the information from each weighted feature sub-graph to form a comprehensive feature map that fully reflects the pavement damage condition. Feature fusion can be performed using a variety of methods, such as weighted summation. This approach effectively integrates information from different damage feature dimensions, eliminating redundant information and enhancing useful information. The resulting pavement damage feature map more accurately reflects the actual damage condition of the road to be identified, providing a more reliable basis for subsequent damage identification and decision-making. After feature fusion is complete, the pavement damage feature map will contain comprehensive information from multiple damage feature dimensions, helping to more accurately determine the type and severity of pavement damage, thereby providing more scientific decision-making support for pavement maintenance and management.

[0055] In one implementation, after extracting a plurality of damage feature dimensions of a damage feature map from the aerial road survey image in step 200, the method may further include:

[0056] Step 201: Using the second convolution matrix as a feature extraction control and the second moving step as a feature extraction frequency, extract feature subgraphs from each disease feature graph to obtain a second disease feature subgraph set corresponding to each disease feature graph.

[0057] In step 201, the second convolution matrix is ​​used as the feature extraction control, and the second moving step is used as the feature extraction frequency to extract feature subgraphs from each damage feature map, thereby obtaining a set of second damage feature subgraphs corresponding to each damage feature map. The second convolution matrix is ​​different from the first convolution matrix. It has different weight values ​​and sizes, and can extract information from the damage feature map from different angles and scales. The second moving step is also different from the first moving step. It determines the interval at which the convolution matrix slides on the damage feature map, thereby affecting the number and size of the extracted feature subgraphs. For example, for a damage feature map representing pavement cracks, the first convolution matrix may be a 3×3 matrix with a first moving step of 1 pixel, which is used to extract the detailed features of the cracks; while the second convolution matrix may be a 5×5 matrix with a second moving step of 2 pixels, which is used to extract the macro features of the cracks. When executing this step, the second convolution matrix is ​​placed in the upper left corner of the disease feature map, and the second convolution matrix is ​​slid from left to right and from top to bottom in sequence according to the second moving stride. A convolution operation is performed on each local area of ​​the disease feature map. Each convolution operation obtains a convolution result. All convolution results are combined to obtain the second disease feature sub-map set corresponding to the disease feature map.

[0058] At this time, step 400 performs feature interaction processing on each first defect feature subgraph set, and fuses multiple interactive feature subgraphs obtained by feature interaction to obtain a pavement defect feature graph of the road to be identified, including:

[0059] Step 401: Perform feature interaction processing on each first defect feature subgraph set, and fuse multiple first interaction feature subgraph sets obtained by feature interaction to obtain a first pavement sub-disease feature graph; and perform feature interaction processing on each second defect feature subgraph set, and fuse multiple second interaction feature subgraph sets obtained by feature interaction to obtain a second pavement sub-disease feature graph.

[0060] For feature interaction processing of the first defect feature subgraph set, a corresponding first image processing algorithm is used. This algorithm can exploit the associations and interactions between the individual feature subgraphs within the first defect feature subgraph set. For example, the first image processing algorithm based on the attention mechanism assigns different weights to each feature subgraph based on its importance to defect identification, highlighting important feature information. When fusing the multiple first interaction feature subgraph sets obtained after feature interaction processing, a weighted summation method can be used to assign weights to each first interaction feature subgraph based on its importance. These subgraphs are then linearly combined to produce the first pavement defect feature map. Similarly, for the second defect feature subgraph set, feature interaction processing is performed using the same or a different first image processing algorithm. The resulting multiple second interaction feature subgraph sets are then fused to produce the second pavement defect feature map. For example, the first pavement defect feature map may focus more on detailed features of the pavement defect, while the second pavement defect feature map may focus more on macroscopic features.

[0061] Step 402: The first pavement defect feature map and the second pavement defect feature map are merged to obtain a pavement defect feature map of the road to be identified.

[0062] This fusion approach comprehensively utilizes defect feature information extracted at different scales and angles, improving the accuracy and completeness of the pavement defect signature map. Fusion can be further performed using a weighted summation method, assigning weights to the first and second sub-pavement defect feature maps based on their importance to the final defect identification. This effectively integrates defect feature information extracted at different scales and angles, resulting in a more comprehensive and accurate pavement defect signature map. This provides a more reliable basis for subsequent defect identification, helps improve the accuracy and reliability of pavement defect identification, and better serves pavement maintenance and management decisions.

[0063] In one implementation, step 500, determining a road surface defect identification result based on a road surface defect characteristic map, includes:

[0064] Step 510: Input the pavement damage feature map into a second image processing algorithm to identify the damage type and obtain a first damage type confidence distribution.

[0065] The second image processing algorithm is a trained model that analyzes and judges the input pavement defect feature map, determining the possible types of road defects and their corresponding confidence levels. This algorithm is built on deep learning technologies, such as convolutional neural networks (CNNs). CNNs possess powerful feature extraction and classification capabilities and consist of multiple convolutional, pooling, and fully connected layers. The convolutional layers slide convolution kernels across the pavement defect feature map to extract local features. The pooling layers perform dimensionality reduction on the extracted features to reduce computational effort. The fully connected layers integrate the extracted features and output the classification results.

[0066] When training the second image processing algorithm, a large amount of pavement damage image data is used as training samples, each sample labeled with the corresponding damage type. By continuously adjusting the model parameters, the model is able to accurately classify different types of damage. When the pavement damage feature map is input into the trained second image processing algorithm, the algorithm performs a series of calculations and processing on the feature map, ultimately outputting a vector, which is the confidence distribution of the first damage type. Each element in the vector represents the confidence level that the corresponding damage type exists on the pavement. The confidence level typically ranges from 0 to 1, with values ​​closer to 1 indicating a greater likelihood of the presence of that type of damage.

[0067] For example, assuming that common road surface damage types include cracks, potholes, and rutting, the confidence distribution vector of the first damage type output by the second image processing algorithm is [0.8, 0.1, 0.1]. This means that the confidence level for the presence of cracks on the road surface is 0.8, the confidence level for the presence of potholes is 0.1, and the confidence level for the presence of rutting is 0.1.

[0068] Step 520: Determine the disease type of the road surface to be identified based on the first disease type confidence distribution.

[0069] For example, a confidence threshold is set. When the confidence of a certain disease type exceeds the threshold, the disease type is determined to be the disease type of the road surface. For example, if the confidence threshold is set to 0.5, in the above example, the confidence of the crack disease is 0.8, which exceeds the threshold, while the confidence of the pothole and rutting diseases are both below the threshold, then the crack will be determined as the disease type of the road surface to be identified. If there are multiple disease types in the confidence distribution of the first disease type whose confidence exceeds the threshold, different processing methods can be adopted according to the specific situation. One method is to determine the disease type with the highest confidence as the final disease type; another method is to regard all disease types with confidence exceeding the threshold as the disease type of the road surface. This situation may indicate that there are multiple diseases on the road surface at the same time.

[0070] Through steps 510 and 520, the road surface defect characteristic map can be accurately used to determine the type of road surface defect to be identified, providing an important decision-making basis for road maintenance and management. The second image processing algorithm, through learning and training on a large amount of data, can effectively identify road surface defects. The setting of the confidence distribution and confidence threshold for the first defect type ensures the accuracy and reliability of the defect identification results.

[0071] In one implementation, the plurality of first image processing algorithms and the second image processing algorithms are obtained by debugging according to a debugging example, and the plurality of first image processing algorithms and the second image processing algorithms are calibrated using the following steps:

[0072] Step 101: Acquire debugging instance data, where the debugging instance data includes instance road surface aerial survey images of multiple road surface instances and a priori disease type of each road surface instance;

[0073] Step 102: extracting an instance defect feature map of multiple defect feature dimensions from the instance road surface aerial survey image;

[0074] Step 103: Using the first convolution matrix as a feature extraction control and the first moving step as a feature extraction frequency, extracting feature subgraphs from each instance disease feature graph to obtain a first instance disease feature subgraph set corresponding to each instance disease feature graph;

[0075] Step 104: The first instance disease feature sub-graph set corresponding to each instance disease feature graph is input into the corresponding first image processing algorithm for feature interaction processing, and the instance interaction feature sub-graph corresponding to each instance disease feature graph obtained by feature interaction is fused to obtain the instance pavement disease feature graph;

[0076] Step 105: The example pavement defect feature image is input into a second image processing algorithm to identify the defect type, obtain a second defect type confidence distribution, and determine a training cost based on the second defect type confidence distribution and the corresponding prior defect type.

[0077] Step 106 : Iteratively adjust algorithm parameters of the plurality of first image processing algorithms and the second image processing algorithm according to the training cost.

[0078] In the AI-based pavement defect identification and decision-making method, multiple first image processing algorithms and second image processing algorithms need to be calibrated based on debugging examples to ensure that they can accurately identify pavement defects in actual applications. This calibration process includes steps 101 to 106.

[0079] In step 101, debugging instance data is obtained. The debugging instance data includes aerial survey images of instance pavement of multiple pavement instances and the prior damage type of each pavement instance. Instance pavement aerial survey images are obtained by photographing different pavement surfaces using aerial photography equipment, such as a high-definition camera mounted on a drone. These images contain rich pavement information, such as pavement texture, color, cracks, potholes, and other features. Prior damage types are pavement damage types that are predetermined through manual inspection or other reliable methods, such as cracks, potholes, and rutting. For example, 1,000 instance pavement aerial survey images of different pavement surfaces can be collected, and the prior damage type of the pavement instance corresponding to each image can be recorded. These data will serve as the basis for subsequent algorithm calibration.

[0080] In step 102, an example defect feature map is extracted from the aerial survey image of an example road surface, representing multiple defect feature dimensions. Different defect feature dimensions can describe the characteristics of road surface defects from different perspectives, such as the length, width, and direction of cracks, and the depth and area of ​​potholes. A variety of image processing and machine learning algorithms are used to extract these features. For example, to extract crack features, edge detection algorithms, such as the Canny edge detection algorithm, can be used to detect crack edges in the image, thereby determining the crack's location and general shape. The crack's width can be calculated by calculating the pixel distance between edges. Suppose a crack is detected in an aerial survey image of an example road surface using the Canny edge detection algorithm. Based on the image's pixel resolution, the pixel distance between edges can be converted to the actual physical width. For pothole features, morphological operations, such as dilation and erosion, can be used to determine the pothole's boundaries and area. First, the image is binarized to separate the pothole area from the background. Then, the pothole area is enlarged using dilation. Erosion removes noise and small connected regions. Finally, the area of ​​the remaining area is calculated, which is the pothole area.

[0081] In step 103, feature subgraphs are extracted from each instance disease feature map using the first convolution matrix as a feature extraction control and the first moving stride as the feature extraction frequency, obtaining a set of first instance disease feature subgraphs corresponding to each instance disease feature map. The first convolution matrix is ​​a two-dimensional matrix composed of a set of weight values ​​and is used to perform a convolution operation on the image. The convolution operation is a linear operation that extracts local features of the image by sliding the convolution matrix across the image, performing element-by-element multiplication with the local area of ​​the image, and summing the results. The first moving stride determines the interval at which the convolution matrix slides across the image.

[0082] In step 104, the first instance disease feature subgraph set corresponding to each instance disease feature graph is passed into the corresponding first image processing algorithm for feature interaction processing, and the instance interaction feature subgraph corresponding to each instance disease feature graph obtained by feature interaction is fused to obtain an instance pavement disease feature graph. The first image processing algorithm is used to explore the associations and interactions between the various feature subgraphs in the first instance disease feature subgraph set, and to enable the features to communicate and influence each other through specific algorithms and operations. For example, a first image processing algorithm based on an attention mechanism is used, which assigns different weights to each feature subgraph according to its importance to disease identification, highlighting important feature information and suppressing irrelevant or noise information. When fusing the instance interaction feature subgraphs corresponding to each instance disease feature graph obtained after feature interaction processing, a weighted summation method can be used to assign weights to each instance interaction feature subgraph according to its importance, and then linearly combine them to obtain an instance pavement disease feature graph.

[0083] In step 105, the instance pavement disease feature map is passed into the second image processing algorithm for disease type identification to obtain the second disease type confidence distribution, and the training cost is determined based on the second disease type confidence distribution and the corresponding prior disease type. The second image processing algorithm is a trained model that can analyze and judge the input instance pavement disease feature map to determine the possible disease types on the pavement and their corresponding confidence levels. The second disease type confidence distribution it outputs is a vector, and each element in the vector represents the confidence level of the corresponding disease type on the pavement. The training cost is used to measure the degree of difference between the output result of the second image processing algorithm and the prior disease type. A feasible training cost function is the cross entropy loss function. Assume that the prior disease type is represented by a one-hot encoding vector y, and the second disease type confidence distribution is represented by a vector Indicates that the calculation formula of the cross entropy loss function L is , where k is the number of damage types. For example, suppose there are three types of damage: cracks, potholes, and ruts. The prior damage type is cracks, and the corresponding one-hot encoding vector y = [1, 0, 0]. The confidence distribution of the second damage type is = [0.8, 0.1, 0.1], then the cross entropy loss L=-(1×\log(0.8)+0×log(0.1)+0×log(0.1)).

[0084] In step 106, the algorithm parameters of the first and second image processing algorithms are iteratively adjusted based on the training cost. An optimization algorithm, such as the stochastic gradient descent (SGD) algorithm, is used to update the algorithm parameters. The stochastic gradient descent algorithm calculates the gradient of the training cost function with respect to the algorithm parameters and then updates the parameters in the opposite direction of the gradient, so that the training cost gradually decreases. Assume that the algorithm parameters are represented by the vector Indicates that the learning rate is Indicates that the training cost function is , then the parameter update formula is ,in Represents the training cost function with respect to the parameter Steps 103 to 105 are repeated multiple times, and the algorithm parameters are continuously updated until the training cost converges to a smaller value. At this point, the parameters of the multiple first image processing algorithms and the second image processing algorithm reach an optimal state, which can more accurately identify road surface defects in practical applications.

[0085] Through steps 101 to 106, multiple first image processing algorithms and second image processing algorithms are effectively adjusted using debugging instance data, so that these algorithms can better adapt to different pavement disease characteristics, improve the accuracy and reliability of pavement disease identification, and provide more powerful support for pavement maintenance and management.

[0086] In one implementation, after extracting a plurality of instance defect feature maps of defect feature dimensions from the instance road surface aerial survey image in step 102, the method may further include:

[0087] Step 102a: Using the second convolution matrix as a feature extraction control and the second moving step as a feature extraction frequency, perform feature subgraph extraction on each instance disease feature graph to obtain a second instance disease feature subgraph set corresponding to each instance disease feature graph.

[0088] In step 102a, feature subgraphs are extracted from each instance defect feature map using the second convolution matrix as a feature extraction control and the second moving stride as the feature extraction frequency, obtaining a set of second instance defect feature subgraphs corresponding to each instance defect feature map. The second convolution matrix differs from the first convolution matrix in that it has different weight values ​​and sizes, enabling it to extract information from the instance defect feature map from different angles and scales. The second moving stride also differs from the first moving stride in that it determines the interval at which the convolution matrix slides across the instance defect feature map, thereby affecting the number and size of extracted feature subgraphs. For example, for an instance defect feature map representing pavement cracks, the first convolution matrix might be a 3×3 matrix with a first moving stride of 1 pixel, used to extract detailed features of the cracks; whereas the second convolution matrix might be a 5×5 matrix with a second moving stride of 2 pixels, used to extract macroscopic features of the cracks. When executing this step, the second convolution matrix is ​​placed in the upper left corner of the instance disease feature map, and the second convolution matrix is ​​slid from left to right and from top to bottom in sequence according to the second moving stride. A convolution operation is performed on each local area of ​​the instance disease feature map. Each convolution operation obtains a convolution result. All convolution results are combined to obtain the second instance disease feature subgraph set corresponding to the instance disease feature map.

[0089] Based on step 102a, step 104 is to pass the first instance disease feature subgraph set corresponding to each instance disease feature graph into the corresponding first image processing algorithm for feature interaction processing, and fuse the instance interaction feature subgraph corresponding to each instance disease feature graph obtained by feature interaction to obtain the instance pavement disease feature graph, including:

[0090] Step 1041: The first instance disease feature sub-graph set corresponding to each instance disease feature graph is passed into the corresponding first image processing algorithm for feature interaction processing, and the first instance interaction feature sub-graph set corresponding to each instance disease feature graph obtained by feature interaction is fused to obtain the first instance pavement sub-disease feature graph.

[0091] The first image processing algorithm is used to explore the connections and interactions between the individual feature subgraphs in the first instance defect feature subgraph set. Through specific algorithms and operations, features can communicate and influence each other. Taking the attention-based first image processing algorithm as an example, different weights are assigned to each feature subgraph based on its importance to defect identification, highlighting important feature information and suppressing irrelevant or noisy information. When fusing multiple first instance interaction feature subgraph sets obtained after feature interaction processing, a weighted summation method can be used to assign weights to each first instance interaction feature subgraph based on its importance. These subgraphs are then linearly combined to obtain the first instance pavement sub-disease feature map.

[0092] Step 1042: The second instance disease feature sub-graph set corresponding to each instance disease feature graph is passed into the corresponding first image processing algorithm for feature interaction processing, and the second instance interaction feature sub-graph set corresponding to each instance disease feature graph obtained by feature interaction is fused to obtain the second instance pavement sub-disease feature graph.

[0093] Similarly, the first image processing algorithm processes the second instance of the defect feature subgraph set, mining the associations between the feature subgraphs. A similar weighted summation approach is also employed when fusing the second instance of the interactive feature subgraph set. Alternatively, the first instance of the pavement defect subgraph feature map may focus more on the detailed features of the pavement defect, while the second instance of the pavement defect subgraph feature map may focus more on the macroscopic features.

[0094] Step 1043: Fusing the first instance pavement sub-disease characteristic map and the second instance pavement sub-disease characteristic map to obtain an instance pavement disease characteristic map of the pavement instance.

[0095] This fusion method can comprehensively utilize the disease feature information extracted at different scales and angles to improve the accuracy and completeness of the example pavement disease feature map. Fusion is again performed using a weighted summation method, assigning weights to the first and second example pavement disease sub-feature maps based on their importance to the final disease identification. For example, if in a specific pavement disease identification scenario, detailed features are more critical for determining the disease type, then the first example pavement disease sub-feature map can be given a higher weight. In this way, the disease feature information extracted at different scales and angles is effectively integrated to obtain a more comprehensive and accurate example pavement disease feature map. This provides a more reliable basis for subsequent disease identification and algorithm adjustment, helps to improve the accuracy and reliability of pavement disease identification, and better serve pavement maintenance and management decisions.

[0096] In one implementation, each first image processing algorithm includes a descriptor extraction operator and a feature interaction operator. Then, step 104, transferring the first instance disease feature subgraph set corresponding to each instance disease feature graph into the corresponding first image processing algorithm for feature interaction processing, and fusing the instance interaction feature subgraph corresponding to each instance disease feature graph obtained by feature interaction to obtain the instance pavement disease feature graph, may include:

[0097] Step 10401: Pre-extract image descriptors from a first instance disease feature subgraph set corresponding to each instance disease feature graph according to a descriptor extraction operator to obtain a first instance image descriptor.

[0098] A descriptor extraction operator is an operator used to extract image features. Any feasible feature extraction operator, such as a convolutional network layer, can convert the image information in the first-instance defect feature subgraph set into a more representative and comparable feature description, namely, an image descriptor. These image descriptors can capture key image features such as texture, shape, and color, thus providing a foundation for subsequent feature interaction processing. Taking crack features in a pavement defect image as an example, a descriptor extraction operator can extract feature information such as crack length, width, and direction, and encode this information into an image descriptor. Assuming that a subgraph in the first-instance defect feature subgraph set represents a localized area of ​​the road surface, the descriptor extraction operator may perform statistical analysis on the pixel values ​​in this subgraph, calculating statistics such as the pixel mean and variance. It may also detect features such as edges and corners in the subgraph, combining this information to form an image descriptor. Specifically, the descriptor extraction operator can also use the Local Binary Pattern (LBP) algorithm. This algorithm compares the size relationship between the central pixel and its neighboring pixels, encodes the neighboring pixel comparison results into binary numbers, and thus obtains a vector describing the image texture features. This vector is part of the image descriptor. By performing this processing on each sub-image in the first-instance disease feature sub-image set, the first-instance image descriptor can be obtained. This is a set of feature information containing multiple sub-images.

[0099] Step 10402: Perform feature interaction processing on the first instance image descriptor corresponding to each instance disease feature map according to the feature interaction operator to obtain an instance interaction feature sub-map corresponding to each instance disease feature map.

[0100] The feature interaction operator is a network layer used for feature interaction fusion. Its function is to explore the associations and interactions between features in the first-instance image descriptor. Through specific algorithms and operations, features can communicate and influence each other, thereby generating an instance-interaction feature subgraph with higher information content and representativeness. Taking the feature interaction operator based on the attention mechanism as an example, it can assign different weights to each feature based on its importance to defect identification, highlighting important features and suppressing irrelevant or noisy information. Assuming that the first-instance image descriptor contains features such as crack length, width, and orientation, the feature interaction operator analyzes the relationships between these features. For example, crack length and width may affect defect severity, while crack orientation may be related to the stress conditions of the pavement. By interactively processing these features, the feature interaction operator can discover the correlation between crack length and width and adjust feature weights based on this correlation, thereby focusing more on feature combinations that are most influential in defect judgment in subsequent defect identification. In this way, after feature interaction processing, the resulting instance-interaction feature subgraph can more comprehensively reflect the characteristics of pavement defects, providing a more accurate basis for subsequent defect identification.

[0101] Step 10403: Fuse the instance interaction feature subgraphs corresponding to each instance defect feature graph to obtain an instance pavement defect feature graph.

[0102] The goal of fusion is to integrate the information from different instance-based pavement defect feature maps, eliminating redundant information and enhancing useful information, thereby obtaining a comprehensive and more representative instance-based pavement defect feature map. Fusion can be performed using various methods, such as weighted summation. The weights can be determined based on various factors, such as the importance of different instance-based pavement defect feature maps to defect identification. For example, while both crack and pothole features may be important for pavement defect identification, in some cases, crack features may be more accurate in determining the type and severity of the defect. Therefore, the instance-based interaction feature submap representing crack features may be given a relatively higher weight. This fusion method effectively integrates the information from different instance-based pavement defect feature maps. The resulting instance-based pavement defect feature map can more accurately reflect the actual pavement defect condition, providing a more reliable basis for subsequent defect type identification and algorithm calibration. This helps improve the accuracy and reliability of pavement defect identification and better serve pavement maintenance and management decisions.

[0103] In one implementation, each first image processing algorithm further includes a spatiotemporal distribution coding operator. Then, in step 10401, after pre-extracting image descriptors from a first instance disease feature subgraph set corresponding to each instance disease feature graph using a descriptor extraction operator to obtain the first instance image descriptor, the method further includes:

[0104] Step 104011: Perform spatiotemporal distribution coding on the first instance disease feature subgraph set corresponding to each instance disease feature graph according to the spatiotemporal distribution coding operator to obtain the first instance disease spatiotemporal distribution feature subgraph set.

[0105] The spatiotemporal distribution encoding operator is an embedding layer that encodes the first instance of the defect feature subgraph set by combining spatial coordinates and temporal position. Each feature subgraph contains not only image feature information but also its spatial and temporal position information. This allows for better utilization of the spatiotemporal distribution of defect features in subsequent feature interaction processing, leading to more accurate pavement defect identification.

[0106] Specifically, the first step is to determine the contribution of the disease features corresponding to each instance of the disease feature subgraph set. The disease feature contribution reflects the importance of each feature subgraph to disease identification and can be determined based on factors such as the richness of the disease information contained in the feature subgraph and the uniqueness of the features. For example, for pavement disease identification, a feature subgraph containing obvious crack characteristics may have a higher disease feature contribution than a feature subgraph containing only a small amount of noise information. By analyzing a large amount of training data, the frequency and accuracy of each feature subgraph in identifying different disease types can be statistically analyzed to determine its disease feature contribution.

[0107] Then, the first instance disease feature subgraph set corresponding to each instance disease feature graph is subjected to spatiotemporal distribution encoding based on the disease feature contribution. Assuming that the spatial coordinates of a feature subgraph Si in the first instance disease feature subgraph set are (xi, yi), the temporal position is ti, and the disease feature contribution is ci, the spatiotemporal distribution encoding operator can combine this information into an encoding vector Ei. One encoding method can be Ei = [xi, yi, ti, ci], where xi and yi represent the spatial position of the feature subgraph in the image, ti represents its position in the temporal sequence, and ci represents the disease feature contribution. By performing such encoding on each feature subgraph in the first instance disease feature subgraph set, the first instance disease spatiotemporal distribution feature subgraph set is obtained. Each element in this set contains the spatiotemporal information of the feature subgraph and the disease feature contribution information.

[0108] At this time, step 10402 performs feature interaction processing on the first instance image descriptor corresponding to each instance disease feature map according to the feature interaction operator to obtain the instance interaction feature subgraph corresponding to each instance disease feature map, including:

[0109] Step 104021: Perform feature interaction processing on the first instance disease spatiotemporal distribution feature subgraph set corresponding to each instance disease feature graph according to the feature interaction operator to obtain the instance interaction feature subgraph corresponding to each instance disease feature graph.

[0110] When processing the first-instance disease spatiotemporal distribution feature subgraph set, the feature interaction operator considers the spatiotemporal information and disease feature contributions of the feature subgraphs, exploring the connections and interactions between them. For example, the feature interaction operator can use an attention mechanism to focus on feature subgraphs with different spatiotemporal locations and disease feature contributions. The feature interaction operator assigns higher attention weights to feature subgraphs that are spatially adjacent, temporally close, and have high disease feature contributions, allowing these feature subgraphs to more fully influence each other during the feature interaction process. Suppose there are two feature subgraphs E1 and E2 in the first-instance disease spatiotemporal distribution feature subgraph set. They are spatially adjacent, temporally close, and both have high disease feature contributions. The feature interaction operator assigns higher attention weights to them by calculating their similarity and correlation. It then performs feature interaction processing on them, such as fusing and transforming their feature information, to generate more representative features.

[0111] Through this feature interaction processing based on spatiotemporal distribution coding, the resulting instance-interaction feature subgraph can better reflect the spatiotemporal distribution patterns of pavement defects and the interrelationships between features, thereby improving the accuracy and reliability of defect identification. These instance-interaction feature subgraphs will be further integrated in subsequent steps to form an instance-based pavement defect feature map, providing stronger support for final pavement defect identification.

[0112] In one implementation, step 104011 performs spatiotemporal distribution coding on the first instance disease feature subgraph set corresponding to each instance disease feature graph according to a spatiotemporal distribution coding operator to obtain the first instance disease spatiotemporal distribution feature subgraph set, including:

[0113] Step 1040111: Determine the disease feature contribution corresponding to each instance disease feature subgraph set.

[0114] In step 1040111, the defect feature contribution corresponding to each instance of the defect feature subgraph set is determined. The defect feature contribution reflects the importance of each instance of the defect feature subgraph set to pavement defect identification. This helps highlight important feature information in the subsequent spatiotemporal distribution encoding process, thereby improving the accuracy of defect identification. The defect feature contribution is determined by comprehensively considering multiple factors. Firstly, the significance of the defect features contained in the feature subgraph set is considered. For example, for a pavement defect image, if a feature subgraph set contains significant features such as clear crack edges and clear pothole outlines, then the importance of this feature subgraph set to defect identification is relatively high, and its defect feature contribution will be correspondingly increased. Secondly, the effectiveness of the feature subgraph set in identifying different defect types is considered. Some feature subgraph sets may only be helpful for identifying a specific defect type, while others may contribute to the identification of multiple defect types, resulting in higher defect feature contributions for the latter. By analyzing large amounts of training data, we can quantify the contribution of each feature subgraph set in identifying different types of defects by measuring its accuracy and recall. For example, consider three instanced defect feature subgraph sets: S1, S2, and S3. Analysis reveals that, in crack defect recognition, S1 has an accuracy of 0.8, S2 has an accuracy of 0.6, and S3 has an accuracy of 0.4. Therefore, S1's defect feature contribution is relatively high and may be assigned a higher weight.

[0115] Step 1040112: Perform spatiotemporal distribution encoding on the first instance disease feature subgraph set corresponding to each instance disease feature graph according to the disease feature contribution, to obtain the first instance disease spatiotemporal distribution feature subgraph set.

[0116] In step 1040112, spatiotemporal distribution encoding is performed on the first instance disease feature subgraph set corresponding to each instance disease feature graph based on the disease feature contribution, resulting in a first instance disease spatiotemporal distribution feature subgraph set. The purpose of spatiotemporal distribution encoding is to integrate information such as the spatial coordinates, temporal position, and disease feature contribution of the instance disease feature subgraph set to form a more representative feature representation, enabling better utilization of this information in subsequent feature interaction processing. Each instance disease feature subgraph set is assigned a coding value related to the disease feature contribution. For example, feature subgraph sets with high disease feature contribution are given a more prominent identifier in the encoding, making them more noticeable in subsequent feature interaction. Assuming that an instance disease feature subgraph set S has spatial coordinates (x, y), temporal position t, and disease feature contribution c, an encoding method can be used to encode it as a vector E = [x, y, t, k × c], where k is a scaling factor used to adjust the influence of the disease feature contribution in the encoding. By encoding the first instance disease feature subgraph set corresponding to each instance disease feature graph in this way, a first instance disease spatiotemporal distribution feature subgraph set is obtained. Each element in this set contains the spatiotemporal information of the feature subgraph set and the disease feature contribution information, providing richer and more targeted information for subsequent feature interaction processing, which helps to improve the accuracy and reliability of pavement disease identification.

[0117] In one implementation, the feature interaction operator includes a multi-scale convolutional attention operator, a first dense residual skip normalization operator, a depthwise separable convolutional network operator, and a second dense residual skip normalization operator. Based on this, step 10402 performs feature interaction processing on the first instance image descriptor corresponding to each instance disease feature map according to the feature interaction operator, obtaining an instance interaction feature subgraph corresponding to each instance disease feature map, including:

[0118] Step 104021: Perform multi-scale convolution attention extraction on the first instance image descriptor corresponding to each instance disease feature map based on the multi-scale convolution attention operator to obtain an image extraction descriptor.

[0119] In step 104021, multi-scale convolutional attention extraction is performed on the first instance image descriptor corresponding to each instance disease feature map based on the multi-scale convolutional attention operator to obtain an image extraction descriptor. The multi-scale convolutional attention operator is a network layer that performs convolution operations on the first instance image descriptor at different scales and highlights important feature information through the attention mechanism. Multi-scale convolution allows the first instance image descriptor to be observed from different receptive fields, capturing feature patterns of different sizes. For example, small-scale convolution can capture detailed features in the image, such as the subtle texture of a crack; large-scale convolution can capture macro features, such as the overall shape of a pothole. The attention mechanism assigns different weights to each position based on the importance of the feature, allowing for greater focus on feature areas that have a significant impact on disease identification.

[0120] In specific implementation, the multi-scale convolution attention operator uses multiple convolution kernels of different sizes to perform convolution operations on the first instance image descriptor. Assuming that three different sizes of convolution kernels are used, 3×3, 5×5, and 7×7, each convolution kernel will slide on the first instance image descriptor to perform convolution operations. The attention mechanism calculates the attention weight of each position, maps the convolution result to a low-dimensional space through a fully connected layer, and then obtains the attention weight through an activation function (such as the Softmax function). Assuming that the convolution result is z, the attention weight a is obtained after passing through the fully connected layer and the Softmax function. Its calculation formula is:

[0121] ;

[0122] Where f is the mapping function of the fully connected layer.

[0123] Finally, the attention weight is multiplied by the convolution result to obtain the image extraction descriptor, that is:

[0124] .

[0125] Step 104022: Perform a normalization operation on the image extraction descriptor corresponding to each instance disease feature map and the first instance image descriptor according to the first dense residual skip normalization operator to obtain a normalized image descriptor.

[0126] In step 104022, the image extraction descriptor corresponding to each instance disease feature map and the first instance image descriptor are normalized using the first dense residual skip normalization operator to obtain a normalized image descriptor. The first dense residual skip normalization operator combines the advantages of dense connections, residual connections, and normalization. Dense connections allow the input of each layer to include the outputs of all previous layers, allowing feature information to be fully transferred between different layers. Residual connections alleviate the vanishing gradient problem by directly adding the input to the output, facilitating network training. Normalization normalizes the features, making their distribution more stable and improving network training efficiency and generalization.

[0127] Specifically, the image extraction descriptor and the first instance image descriptor are first concatenated to obtain a tensor containing more feature information. This tensor is then input into a normalization layer (such as the Batch Normalization layer) for normalization. The calculation formula of the Batch Normalization layer is: ; where x i are input features, is the mean of the feature, is the variance of the feature, is a small constant used to avoid the denominator being zero.

[0128] Next, the normalized features are input into a fully connected layer for linear transformation, yielding an intermediate result. Finally, the intermediate result is residually connected (directly added) to the input features to yield the normalized image descriptor. Assuming the input feature is x, the intermediate result is y, and the normalized image descriptor is z, then: z = x + y.

[0129] Step 104023: Perform a depth-wise separable convolution operation on the normalized image descriptor corresponding to each instance disease feature map according to the depth-wise separable convolutional network operator to obtain a convolutional image descriptor.

[0130] The depthwise separable convolutional network operator decomposes the traditional convolution operation into two steps: depthwise convolution and pointwise convolution. This significantly reduces the number of model parameters and computational complexity while maintaining performance. Depthwise convolution performs a convolution operation on each channel of the input feature, using a separate convolution kernel for each channel. Assuming the normalized image descriptor has C channels, a depthwise convolution is performed using a k×k convolution kernel for each channel. The computational formula for depthwise convolution is similar to that of conventional convolution, but each channel is calculated independently.

[0131] Point-by-point convolution uses a 1×1 convolution kernel to convolve the output of depthwise convolution, combining features from different channels. Through depthwise separable convolution, a convolutional image descriptor is obtained, which reduces the amount of computation while still capturing the important feature information in the normalized image descriptor.

[0132] Step 104024: Perform a normalization operation on the normalized image descriptor and the convolutional image descriptor corresponding to each instance disease feature map according to the second dense residual skip normalization operator to obtain an instance interaction feature submap corresponding to each instance disease feature map.

[0133] The working principle of the second dense residual skip normalization operator is similar to that of the first dense residual skip normalization operator, which also combines dense connections, residual connections and normalization operations.

[0134] First, the normalized image descriptor and the convolutional image descriptor are concatenated and then fed into a normalization layer (such as a Batch Normalization layer) for normalization. Next, the normalized features are fed into a fully connected layer for linear transformation, yielding an intermediate result. Finally, a residual connection is performed between the intermediate result and the input features to generate an instance interaction feature subgraph. This step further integrates the feature information of the normalized image descriptor and the convolutional image descriptor, enhancing the expressive power of the features. The resulting instance interaction feature subgraph can more comprehensively and accurately reflect the characteristics of pavement defects, providing stronger support for subsequent defect identification and algorithm tuning.

[0135] Through steps 104021 to 104024, a series of feature interaction processing is performed on the first instance image descriptor using the feature interaction operator, from multi-scale convolution attention extraction to normalization operation, and then to depthwise separable convolution and re-normalization, gradually mining and integrating feature information, and finally obtaining an instance interaction feature subgraph that can effectively reflect the characteristics of pavement defects, which helps to improve the accuracy and reliability of pavement defect identification.

[0136] In a derivative implementation, after determining the road surface defect identification result according to the road surface defect characteristic map in step 500, the method may further include:

[0137] Step 600: Obtain the disease type and the corresponding disease area confidence in the disease identification result.

[0138] The defect type refers to the type of road defect identified by the previous defect identification process, such as cracks, potholes, and ruts. The defect area confidence indicates the degree of confidence in the existence of the corresponding defect in the identified defect area. Its value range is between 0 and 1. The closer the value is to 1, the higher the confidence level is in the existence of the corresponding defect in the area. For example, in a road defect identification, it is determined that a certain section of the road surface has crack defects, and the defect area confidence given for the identified crack area is 0.8, which means that there is an 80% confidence that the area does have crack defects. This information will be extracted from the previous defect identification result records, and the defect type and the corresponding defect area confidence will be organized into a data set for subsequent processing and analysis.

[0139] Step 700: Perform multi-scale verification processing on the confidence of the diseased area to generate a multi-scale verification result; wherein the multi-scale verification processing includes: segmenting the diseased area with sliding windows of different scales to obtain multiple sub-areas; calculating the local confidence distribution of each sub-area, and weighted fusion of the local confidence distributions of all sub-areas to obtain a multi-scale verification result.

[0140] The purpose of multi-scale verification processing is to conduct a more detailed analysis of the diseased area from different scales to evaluate the reliability of the confidence of the diseased area. In specific implementation, the diseased area is segmented with sliding windows of different scales to obtain multiple sub-regions. The scale of the sliding window can be set according to the actual situation. For example, it can be set to a small scale (such as 10×10 pixels), a medium scale (such as 50×50 pixels) and a large scale (such as 100×100 pixels). For each scale of the sliding window, it slides on the diseased area according to a certain step size to divide the diseased area into multiple sub-regions. Then, the local confidence distribution of each sub-region is calculated. The local confidence distribution reflects the probability distribution of the existence of diseases in the sub-region. The local confidence can be calculated based on information such as the image features in the sub-region and the output of the previous disease recognition model. For example, the previously trained disease recognition model can be used to re-identify each sub-region to obtain the confidence that different types of diseases exist in the sub-region. Assume that the sub-region R i , the confidence level of the existence of crack disease is obtained through the disease identification model as p i1 , the confidence level of the existence of pothole disease is p i2 , the confidence level of the existence of rutting disease is p i3 , then the local confidence distribution of the sub-region can be expressed as a vector [p i1 , p i2 , p i3 ].

[0141] Finally, the local confidence distributions of all subregions are weighted and fused to obtain the multi-scale verification result. Weighted fusion is designed to comprehensively consider information from different scales and subregions to improve the accuracy of the verification results. Different weights can be assigned to the local confidence distribution of each subregion based on factors such as its size and location.

[0142] Step 800: Compare and analyze the multi-scale verification results with a preset confidence threshold. If there is a defect area below the confidence threshold in the multi-scale verification results, trigger a defect identification and correction instruction.

[0143] The confidence threshold is a pre-set standard value used to determine whether the confidence level of a defect area is sufficiently reliable. If a defect area in the multi-scale verification results falls below the confidence threshold, a defect identification correction instruction is triggered. For example, if the preset confidence threshold is 0.6 and the multi-scale verification results show that the confidence level of a crack defect in a defect area is 0.5, which is below the threshold, a corresponding correction instruction is triggered, indicating that the identification result of the defect area may be inaccurate and needs to be corrected.

[0144] Step 900: In response to the disease identification correction instruction, re-extract local features of the diseased area, and update the disease type and diseased area confidence in the disease identification result based on the re-extracted local features.

[0145] Local feature re-extraction involves re-extracting features from damaged areas below a confidence threshold to obtain more accurate information. This can be done using the same or different methods as the previous feature extraction. For example, a finer convolution kernel and a smaller stride can be used to extract features from the damaged area to capture more detailed features.

[0146] Based on the re-extracted local features, the damage area is re-identified using the damage recognition model, resulting in a new damage type and a new confidence level for the damage area. Suppose, after re-extracting local features, a certain damage area is re-identified, and the confidence level for the presence of a pothole is 0.8. However, the previous identification result indicated an inaccurate damage type for this area, with a low confidence level. In this case, the damage recognition result is updated, with the area's damage type updated to a pothole and the confidence level updated to 0.8. This method of re-extracting local features and updating results can correct previously potentially erroneous damage recognition results, improving the accuracy and reliability of damage recognition and providing more accurate decision-making for road maintenance and management.

[0147] Through steps 600 to 900, the pavement defect identification results are effectively verified and corrected. From obtaining the defect type and confidence level, to multi-scale verification, and then to comparative analysis and local feature re-extraction and update, the entire process forms a closed-loop correction mechanism, which helps to improve the performance and stability of the pavement defect identification system and better serve the detection and treatment of pavement defects.

[0148] In step 900, after updating the disease type and disease area confidence in the disease identification result based on the re-extracted local features, the method provided in the embodiment of the present application may further include the following derivative implementation schemes:

[0149] Step 1000: Generate a set of recommended parameters for disease repair based on the updated disease type and disease area confidence level.

[0150] In step 1000, a set of recommended repair parameters is generated based on the updated defect type and defect area confidence score. This set of recommended repair parameters contains a series of repair parameters required for different defect types and severity levels. These parameters form the basis for developing repair plans. First, the corresponding repair strategy is determined based on the updated defect type. For example, for crack defects, repair methods such as grouting and taping may be considered; for pothole defects, filling repair may be necessary. Next, the severity of the defect is assessed based on the defect area confidence score. A higher confidence score generally indicates a more severe defect. For example, for crack defects, a higher confidence score indicates a deeper and longer crack, requiring more grouting material and more sophisticated construction techniques. This information is then used to generate parameters such as material usage, construction process requirements, and construction time estimates. For a crack defect with a confidence score of 0.8, the generated recommended repair parameter set might include: 50 kg of high-elasticity grouting adhesive, high-pressure grouting, and an estimated construction time of two days.

[0151] Step 2000: Call the historical restoration case database and match the corresponding historical restoration solution features based on the disease restoration suggestion parameter set.

[0152] In step 2000, the historical repair case database is called to match the corresponding historical repair solution features based on the defect repair suggestion parameter set. The historical repair case database stores relevant information on previous pavement defect repairs, including defect type, repair solution, material usage, construction period, cost, etc. The defect repair suggestion parameter set will be compared with each historical repair case in the database to find cases with high similarity. The comparison process can be achieved by calculating the similarity between parameters, for example, using the Euclidean distance formula to measure the distance between the defect repair suggestion parameter set and the historical repair case parameters. The historical repair case with the smallest distance is selected, and its corresponding repair solution feature is the matched historical repair solution feature. For example, through comparison, a historical case is found whose parameters such as defect type, material usage, and construction process are very similar to the current defect repair suggestion parameter set. In this case, the repair solution features of the historical case are extracted, including information such as the brand of materials used, the experience of the construction team, and the construction process.

[0153] Step 3000: Input the historical restoration plan characteristics and the disease restoration recommended parameter set into the preset restoration optimization model to generate a target restoration optimization plan.

[0154] Specifically, the repair optimization model generates a target repair optimization plan through the following steps: extracting material parameters, construction cycle parameters and cost parameters from the characteristics of historical repair plans; calculating the adaptability of material parameters with the disease severity parameters in the disease repair recommendation parameter set to obtain material adaptation weights; performing spatiotemporal matching calculations based on the construction cycle parameters and the spatial distribution characteristics of the diseased area to generate a construction priority sequence; combining the material adaptation weights and the construction priority sequence to generate a target repair optimization plan with cost constraints.

[0155] The restoration optimization model is a trained model that comprehensively considers the characteristics of historical restoration solutions and the recommended parameter set for damage repair to generate the optimal restoration solution. Specifically, the restoration optimization model performs the following calculations.

[0156] First, extract material parameters, construction period parameters, and cost parameters from the historical repair solution characteristics. Material parameters include the type, quantity, and quality of the materials used; construction period parameters refer to the time required to complete the repair work; and cost parameters include material costs, labor costs, and equipment costs. For example, the historical repair solution characteristics indicate the use of 50 tons of a certain brand of asphalt, a construction period of 3 days, and a total cost of 100,000 yuan.

[0157] Next, the material parameters are compared with the damage severity parameters in the recommended repair parameter set to obtain the material's suitability weight. The damage severity parameter can be measured using information such as the confidence level of the damaged area. Assuming the damage severity parameter in the recommended repair parameter set is S and the material parameters in the historical repair solution are M, the suitability can be calculated using an adaptation function f(S, M). The higher the suitability, the more suitable the material is for the current repair.

[0158] Next, a spatiotemporal matching calculation is performed based on the construction cycle parameters and the spatial distribution characteristics of the damaged areas to generate a construction priority sequence. The spatial distribution characteristics of the damaged areas include their location, area, and distribution density. Construction convenience and efficiency are considered, with priority given to repairing areas with short construction periods and minimal traffic impact. For example, if the damaged areas are distributed across a busy main road and relatively quiet side roads, repairs on the side roads will be prioritized to minimize traffic impact. Through spatiotemporal matching calculations, the construction priority for each damaged area can be determined, forming a construction priority sequence.

[0159] Finally, combining material adaptation weights and construction priority sequences, a targeted repair optimization plan with cost constraints is generated. While meeting cost constraints, the most appropriate materials are selected based on the material adaptation weights, and the construction sequence is arranged according to the construction priority sequence to achieve the goal of optimal repair results and lowest cost. For example, based on the material adaptation weights, the most suitable material is selected. At the same time, according to the construction priority sequence, the damaged areas of the branch roads are repaired first, followed by the damaged areas of the main roads. This ensures repair quality while reducing costs and traffic impacts.

[0160] Through steps 1000 to 3000, from generating a set of recommended parameters for pavement repair, to matching the characteristics of historical repair plans, and then using the repair optimization model to generate a target repair optimization plan, a complete pavement damage repair decision-making process is formed. This provides a scientific and reasonable plan for the repair of pavement damage, helps to improve the efficiency and quality of pavement repair, and reduces repair costs.

[0161] After generating the target repair optimization plan, the method provided in the embodiment of the present application can also perform the following steps: real-time collection of construction monitoring data during the execution of the target repair optimization plan; then, dynamic deviation analysis is performed on the construction monitoring data and the expected parameters in the target repair optimization plan to generate a deviation indicator set; if there are indicators in the deviation indicator set that exceed the dynamic threshold, a plan adjustment instruction is triggered; in response to the plan adjustment instruction, the cost parameters and construction cycle parameters in the target repair optimization plan are iteratively optimized based on the real-time material consumption rate and construction progress parameters in the construction monitoring data; finally, the iteratively optimized parameters are fed back to the repair optimization model, and the associated case features in the historical repair case database are updated.

[0162] After generating the target repair optimization plan, the execution process of the plan needs to be monitored and adjusted dynamically in real time to ensure that the plan can be implemented smoothly and achieve the expected results. The specific process includes real-time collection of construction monitoring data during the execution of the target repair optimization plan, dynamic deviation analysis of the construction monitoring data and the expected parameters in the target repair optimization plan, generation of a deviation indicator set, and decision on whether to trigger the plan adjustment instruction based on the deviation indicator set. In response to the instruction, the target repair optimization plan is iteratively optimized, and the iteratively optimized parameters are fed back to the repair optimization model and the associated case features in the historical repair case database are updated.

[0163] Real-time construction monitoring data is collected during the execution of the targeted repair optimization plan. This data reflects the actual conditions of the pavement repair process and is acquired through various sensors and monitoring equipment. Material usage can be monitored in real time using weight sensors. For example, when using asphalt to fill potholes, sensors can accurately record the amount of asphalt applied at each application. Construction progress can be determined by using positioning systems and timing devices installed on construction equipment to track their location and time of operation, thereby calculating the amount of work completed and the progress of the project. For example, a pavement repair project is scheduled to complete the repair of 1,000 square meters of pavement within 10 days. On the fifth day of construction, the positioning system indicates that the equipment has completed 400 square meters of repair work. This constitutes construction progress monitoring data. For construction quality, a laser smoothness meter can be used to test the smoothness of the repaired pavement to determine whether it meets the expected standards.

[0164] Next, a dynamic deviation analysis is performed between the construction monitoring data and the expected parameters in the target repair optimization plan to generate a set of deviation indicators. The expected parameters in the target repair optimization plan include material usage, construction period, and construction quality standards. Dynamic deviation analysis compares the actual construction monitoring data with the expected parameters and calculates the difference between the two. Taking material usage as an example, if the target repair optimization plan estimates the use of 50 tons of asphalt, but the construction monitoring data shows that 55 tons of asphalt have actually been used, the deviation in material usage is 55 - 50 = 5 tons. Regarding construction progress, if the target repair optimization plan expects 500 square meters of repair work to be completed on the fifth day, but only 400 square meters are actually completed, the construction progress deviation is 500 - 400 = 100 square meters. These deviation values ​​are organized into a set of deviation indicators, each of which reflects the deviation from the expected level in a specific aspect of the construction process.

[0165] Then, determine whether there is an indicator in the deviation indicator set that exceeds the dynamic threshold. If so, a scheme adjustment instruction is triggered. The dynamic threshold is a pre-set allowable deviation range based on different construction parameters and actual conditions. Different parameters may have different thresholds. For example, the dynamic threshold of material usage may be set to ±10% of the expected usage, and the dynamic threshold of construction progress may be set to ±5% of the expected progress. When the material usage deviation exceeds 10% of the expected usage or the construction progress deviation exceeds 5% of the expected progress, a scheme adjustment instruction will be triggered. Assuming that the material usage deviation reaches 12%, exceeding the dynamic threshold of 10%, a scheme adjustment instruction will be issued immediately, indicating that the current construction situation is significantly different from expectations and the scheme needs to be adjusted.

[0166] In response to the scheme adjustment instruction, the cost parameters and construction period parameters in the target repair optimization scheme are iteratively optimized based on the real-time material consumption rate and construction progress parameters in the construction monitoring data. The real-time material consumption rate refers to the actual amount of material consumed per unit time, and the construction progress parameters reflect the actual progress of the construction. The cost and construction period are re-evaluated based on these parameters. If the real-time material consumption rate is higher than expected, the budget for the cost parameters will be increased; if the construction progress lags, the construction period will be extended accordingly. For example, if the construction monitoring data shows that the material consumption rate is 20% higher than expected, the cost parameters will be increased by 20%; if the construction progress lags by 2 days, the construction period will be extended by 2 days. By continuously iteratively optimizing according to the actual construction situation, the target repair optimization scheme can better adapt to the actual construction environment.

[0167] Finally, the iteratively optimized parameters are fed back to the repair optimization model, and the associated case features in the historical repair case database are updated. Feeding the iteratively optimized parameters back to the repair optimization model allows the model to learn from the experience and changes in actual construction, making it more accurate and reasonable when subsequently generating new target repair optimization plans. At the same time, updating the associated case features in the historical repair case database makes the case information in the database more realistic and comprehensive, providing a more valuable reference for future pavement disease repair decisions. For example, parameters such as material usage, construction cycle, and cost adjusted during this construction are updated to the corresponding historical cases. The next time a similar pavement disease is encountered, more accurate reference information can be obtained from the database to generate a more optimized repair plan.

[0168] Through the above series of operations, real-time monitoring and dynamic adjustment of the execution process of the target repair optimization plan are achieved, ensuring that the pavement disease repair work can be carried out according to the optimal plan, improving the repair efficiency and quality, reducing the repair cost, and also providing a more scientific decision-making basis for future pavement disease repair.

[0169] In addition, after obtaining the disease identification result in step 500, the method provided in the embodiment of the present application may also include a scheme for visual presentation of the result, for example, spatially superimposing the disease identification result with the aerial survey image of the road surface to generate a visual road surface image containing disease annotation information, and then color-coding the disease area in the visual road surface image to generate a disease distribution heat map, and then converting the disease distribution heat map into an image format of the target terminal and outputting it to the display interface of the target terminal through a preset interface; wherein the color coding rules of the disease distribution heat map include, for example: matching a first color identifier based on the disease type, matching a second color identifier based on the severity of the disease, and superimposing and rendering the first color identifier and the second color identifier.

[0170] Furthermore, the method provided in the embodiment of the present application may also include a multi-model verification process, and another AI model may be used to verify the results, and the results of the two may be compared to improve accuracy. For example, a pre-trained second disease recognition model is called to perform a second verification on the pavement disease characteristic map to obtain a second disease type confidence distribution. Then, the difference between the first disease type confidence distribution and the second disease type confidence distribution corresponding to the disease recognition result is calculated. If the difference exceeds a preset difference threshold, the pavement disease characteristic map is multi-scale feature resampled to generate a resampled feature map set. Finally, the resampled feature map set is input into the second disease recognition model to re-identify the disease type to obtain an updated disease recognition result.

[0171] When presenting the results of a disease, the disease can be prioritized. The results are categorized by type and severity, and the treatment priority is determined to generate maintenance recommendations. For example, based on the disease type and the area of ​​the diseased area in the disease identification results, a disease treatment priority sequence is determined. Then, pavement maintenance decision information is generated based on the disease treatment priority sequence. The pavement maintenance decision information includes the order in which the disease is repaired, the type of repair material, and the repair time window. The steps for determining the disease treatment priority sequence include matching the disease type with a preset weight factor, matching the area of ​​the diseased area with an area influence coefficient, and performing a weighted summation of the weight factor and the area influence coefficient to obtain a priority score.

[0172] Based on Figure 1 Based on the same principle as the method shown in , the present application embodiment also provides an AI-based road disease identification and decision-making device 10, such as Figure 2 As shown, the device 10 includes:

[0173] An image acquisition module 11 is used to acquire an aerial survey image of the road surface to be identified;

[0174] The feature extraction module 12 is configured to extract a plurality of defect feature dimensions of a defect feature map from the pavement aerial survey image; perform feature sub-map extraction on each of the defect feature maps using a first convolution matrix as a feature extraction control and a first moving step as a feature extraction frequency, to obtain a first set of defect feature sub-maps corresponding to each of the defect feature maps;

[0175] A feature interaction module 13 is configured to perform feature interaction processing on each of the first defect feature subgraph sets, and fuse multiple interactive feature subgraphs obtained through feature interaction to obtain a pavement defect feature graph of the road surface to be identified;

[0176] The disease identification module 14 is used to determine the disease identification result of the road surface to be identified based on the road surface disease characteristic map.

[0177] It can be understood that the principle of the AI-based pavement defect identification and decision-making device is consistent with the above-mentioned AI-based pavement defect identification and decision-making method, and will not be elaborated here.

[0178] The embodiment of the present application provides a computer system, such as Figure 3 As shown, computer system 100 includes: a processor 101 and a memory 103. Processor 101 and memory 103 are connected, for example, via bus 102. Optionally, computer system 100 may further include a transceiver 104. It should be noted that in actual applications, the number of transceivers 104 is not limited to one, and the structure of computer system 100 does not constitute a limitation on the embodiments of this application.

[0179] An embodiment of the present application provides a computer system. The computer system in the embodiment of the present application includes: one or more processors; a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory and configured to be executed by the one or more processors. When the one or more programs are executed by the processor, the method provided above is implemented.

Claims

1. An AI-based pavement disease identification and decision-making method, characterized by: The method comprises: Acquire an aerial survey image of the road surface to be identified; Extracting a plurality of damage feature maps of the damage feature dimensions from the road surface aerial survey image, wherein each damage feature map corresponds to one damage feature dimension; Using the first convolution matrix as a feature extraction control and the first moving step as a feature extraction frequency, performing feature subgraph extraction on each of the disease feature graphs to obtain a first disease feature subgraph set corresponding to each of the disease feature graphs; Performing feature interaction processing on each of the first defect feature subgraph sets to obtain an interactive feature subgraph corresponding to each defect feature dimension, and fusing multiple interactive feature subgraphs obtained by feature interaction to obtain a pavement defect feature map of the road surface to be identified; The disease identification result of the road surface to be identified is determined based on the road surface disease characteristic map.

2. The method according to claim 1, characterized in that The step of performing feature interaction processing on each of the first defect feature subgraph sets and fusing multiple interactive feature subgraphs obtained through feature interaction to obtain the pavement defect feature graph of the road to be identified includes: Determining a first image processing algorithm corresponding to each of the first disease feature sub-graph sets; Inputting the first disease feature subgraph set corresponding to each of the disease feature graphs into the corresponding first image processing algorithm for feature interaction processing to obtain an interactive feature subgraph corresponding to each of the disease feature graphs; Feature fusion is performed on a plurality of the interactive feature subgraphs to obtain a pavement disease feature graph of the road surface to be identified.

3. The method according to claim 2, characterized in that The step of fusing the plurality of interactive feature subgraphs to obtain a pavement damage feature graph of the road to be identified includes: Determining a first impact factor of each of the interactive feature subgraphs; Performing weighted calculation on each of the interactive feature subgraphs according to the first influencing factor to obtain a plurality of weighted feature subgraphs; Performing feature fusion on the multiple weighted feature subgraphs to obtain a pavement disease feature graph of the road surface to be identified; The first impact factor is determined by the following steps: Obtaining the road surface type of the road to be identified; A first influencing factor of each of the interactive feature subgraphs is determined according to relevant information of each of the disease feature dimensions and the road surface type of the road surface to be identified.

4. The method according to claim 1, wherein After extracting the damage feature maps of multiple damage feature dimensions from the road surface aerial survey image, the method further includes: Using the second convolution matrix as a feature extraction control and the second moving step as a feature extraction frequency, performing feature subgraph extraction on each of the disease feature graphs to obtain a second disease feature subgraph set corresponding to each of the disease feature graphs; The step of performing feature interaction processing on each of the first defect feature subgraph sets and fusing multiple interactive feature subgraphs obtained through feature interaction to obtain the pavement defect feature graph of the road to be identified includes: Performing feature interaction processing on each of the first defect feature subgraph sets, and fusing multiple first interaction feature subgraph sets obtained by the feature interaction to obtain a first pavement defect subgraph feature map; and performing feature interaction processing on each of the second defect feature subgraph sets, and fusing multiple second interaction feature subgraph sets obtained by the feature interaction to obtain a second pavement defect subgraph feature map; The first pavement sub-disease characteristic map and the second pavement sub-disease characteristic map are fused to obtain a pavement disease characteristic map of the road to be identified.

5. The method according to claim 2, characterized in that Determining the disease identification result of the road surface to be identified based on the road surface disease characteristic map includes: Inputting the pavement disease characteristic map into a second image processing algorithm to identify the disease type and obtain a confidence distribution of the first disease type; Determining the disease type of the road surface to be identified according to the first disease type confidence distribution; The plurality of first image processing algorithms and the second image processing algorithms are obtained by debugging according to a debugging example, and the plurality of first image processing algorithms and the second image processing algorithms are obtained by calibrating using the following steps: Acquiring debugging instance data, the debugging instance data including instance road surface aerial survey images of a plurality of road surface instances and a priori disease type of each road surface instance; Extracting an example disease feature map of multiple disease feature dimensions from the example road surface aerial survey image; Using the first convolution matrix as a feature extraction control and the first moving stride as a feature extraction frequency, performing feature subgraph extraction on each of the instance disease feature graphs to obtain a first instance disease feature subgraph set corresponding to each of the instance disease feature graphs; The first instance disease feature subgraph set corresponding to each of the instance disease feature graphs is passed into the corresponding first image processing algorithm for feature interaction processing, and the instance interaction feature subgraph corresponding to each of the instance disease feature graphs obtained by feature interaction is fused to obtain an instance pavement disease feature graph; Passing the example pavement defect feature image into a second image processing algorithm to identify the defect type, obtaining a second defect type confidence distribution, and determining a training cost based on the second defect type confidence distribution and the corresponding prior defect type; Iteratively adjust algorithm parameters of a plurality of the first image processing algorithms and the second image processing algorithm according to the training cost.

6. The method according to claim 5, characterized in that After extracting the example damage feature map of multiple damage feature dimensions from the example road surface aerial survey image, the method further includes: Using the second convolution matrix as a feature extraction control and the second moving step as a feature extraction frequency, extracting feature subgraphs from each of the instance disease feature graphs to obtain a set of second instance disease feature subgraphs corresponding to each of the instance disease feature graphs; The step of transferring the first instance disease feature subgraph set corresponding to each instance disease feature graph into the corresponding first image processing algorithm for feature interaction processing, and fusing the instance interaction feature subgraph corresponding to each instance disease feature graph obtained by feature interaction to obtain an instance road surface disease feature graph includes: The first instance disease feature sub-graph set corresponding to each of the instance disease feature graphs is passed into the corresponding first image processing algorithm for feature interaction processing, and the first instance interaction feature sub-graph set corresponding to each of the instance disease feature graphs obtained by feature interaction is fused to obtain a first instance pavement sub-disease feature graph; The second instance disease feature sub-graph set corresponding to each of the instance disease feature graphs is passed into the corresponding first image processing algorithm for feature interaction processing, and the second instance interaction feature sub-graph set corresponding to each of the instance disease feature graphs obtained by feature interaction is fused to obtain a second instance pavement sub-disease feature graph; The first instance pavement sub-disease characteristic map and the second instance pavement sub-disease characteristic map are fused to obtain an instance pavement disease characteristic map of the pavement instance.

7. The method according to claim 5, characterized in that Each of the first image processing algorithms includes a descriptor extraction operator and a feature interaction operator. The first instance disease feature subgraph set corresponding to each instance disease feature graph is input into the corresponding first image processing algorithm for feature interaction processing, and the instance interaction feature subgraph corresponding to each instance disease feature graph obtained by feature interaction is fused to obtain an instance pavement disease feature graph, including: Performing image descriptor pre-extraction on a first instance disease feature subgraph set corresponding to each instance disease feature graph according to the descriptor extraction operator to obtain a first instance image descriptor; Performing feature interaction processing on the first instance image descriptor corresponding to each instance disease feature map according to the feature interaction operator to obtain an instance interaction feature subgraph corresponding to each instance disease feature map; The instance interaction feature subgraphs corresponding to each instance defect feature graph are fused to obtain an instance pavement defect feature graph.

8. The method according to claim 7, characterized in that Each of the first image processing algorithms further includes a spatiotemporal distribution coding operator, and after performing image descriptor pre-extraction on a first instance disease feature subgraph set corresponding to each of the instance disease feature graphs according to the descriptor extraction operator to obtain a first instance image descriptor, the algorithm further includes: Performing spatiotemporal distribution coding on the first instance disease feature subgraph set corresponding to each instance disease feature graph according to the spatiotemporal distribution coding operator to obtain the first instance disease spatiotemporal distribution feature subgraph set; The step of performing feature interaction processing on the first instance image descriptor corresponding to each instance disease feature map according to the feature interaction operator to obtain an instance interaction feature subgraph corresponding to each instance disease feature map includes: Feature interaction processing is performed on the first instance disease spatiotemporal distribution feature subgraph set corresponding to each instance disease feature graph according to the feature interaction operator to obtain an instance interaction feature subgraph corresponding to each instance disease feature graph.

9. The method according to claim 8, characterized in that The step of performing spatiotemporal distribution coding on the first instance disease feature subgraph set corresponding to each instance disease feature graph according to the spatiotemporal distribution coding operator to obtain the first instance disease spatiotemporal distribution feature subgraph set includes: Determining the disease feature contribution corresponding to each of the instance disease feature subgraph sets; Performing spatiotemporal distribution coding on a first instance disease feature subgraph set corresponding to each instance disease feature graph according to the disease feature contribution, to obtain a first instance disease spatiotemporal distribution feature subgraph set; The feature interaction operator includes a multi-scale convolution attention operator, a first dense residual skip normalization operator, a depthwise separable convolutional network operator, and a second dense residual skip normalization operator. The feature interaction operator is used to perform feature interaction processing on the first instance image descriptor corresponding to each instance disease feature map to obtain an instance interaction feature subgraph corresponding to each instance disease feature map, including: Performing multi-scale convolution attention extraction on the first instance image descriptor corresponding to each instance disease feature map according to the multi-scale convolution attention operator to obtain an image extraction descriptor; performing a normalization operation on the image extraction descriptor and the first instance image descriptor corresponding to each instance disease feature map according to the first dense residual skip normalization operator to obtain a normalized image descriptor; Performing a depthwise separable convolution operation on the normalized image descriptor corresponding to each of the instance disease feature maps according to the depthwise separable convolutional network operator to obtain a convolutional image descriptor; The normalized image descriptor and the convolutional image descriptor corresponding to each of the instance disease feature maps are normalized according to the second dense residual skip normalization operator to obtain an instance interaction feature subgraph corresponding to each of the instance disease feature maps.

10. An AI-based road surface disease identification and decision-making device, characterized in that: include: An image acquisition module, used to acquire an aerial survey image of the road surface to be identified; a feature extraction module configured to extract a plurality of damage feature maps of the damage feature dimensions from the aerial survey image of the road surface; wherein each damage feature map corresponds to one damage feature dimension; perform feature sub-map extraction on each of the damage feature maps using a first convolution matrix as a feature extraction control and a first moving step as a feature extraction frequency, and obtain a first set of damage feature sub-maps corresponding to each of the damage feature maps; a feature interaction module, configured to perform feature interaction processing on each of the first defect feature subgraph sets to obtain an interaction feature subgraph corresponding to each defect feature dimension, and fuse multiple interaction feature subgraphs obtained by feature interaction to obtain a pavement defect feature graph of the road surface to be identified; The disease identification module is used to determine the disease identification result of the road surface to be identified based on the road disease characteristic map.

Images