A multi-feature matching semi-supervised pavement disease identification method and system

Abstract

The present application relates to the technical field of road intelligent operation and maintenance, in particular to a multi-feature matching semi-supervised pavement disease identification method and system. It aims to solve the problem that the existing pavement disease identification method highly depends on pixel-level labeled data, has high labeling cost, and the semi-supervised method has low utilization efficiency of unlabeled data and insufficient recognition accuracy. The present application performs weak enhancement and strong enhancement processing on unlabeled images, extracts the intermediate features of multi-view images, realizes cross-view feature matching through a multi-feature fusion matching mechanism, optimizes the model by fusing feature matching loss, supervised loss and unsupervised loss, and finally outputs the disease identification result through the trained model. The present application reduces the labeling cost, improves the accuracy and robustness of pavement disease identification in the few-label scenario, and is suitable for road intelligent operation and maintenance scenarios.

Description

Technical Field

[0001] This invention relates to the field of intelligent road operation and maintenance technology, specifically to intelligent pavement defect identification technology, and particularly to a semi-supervised pavement defect identification method and system using multi-feature matching. Background Technology

[0002] This invention belongs to the field of intelligent road operation and maintenance technology, specifically involving pavement defect identification technology. Accurate detection of pavement defects is a core basis for road maintenance decisions. Deep learning semantic segmentation technology, due to its ability to predict defect categories at the pixel level, has become the mainstream method for automated pavement defect detection. It can accurately quantify the geometric features of defects, providing data support for road maintenance.

[0003] However, existing deep learning semantic segmentation methods heavily rely on large-scale, finely labeled pavement defect datasets. Pavement images have irregular textures and complex structural details; pixel-level labeling of a single image is time-consuming and requires expert verification, resulting in extremely high labeling costs and severely limiting the engineering application of this technology. Furthermore, existing semi-supervised pavement defect identification methods often focus on consistency regularization of the output layer, neglecting the structural and relational semantic supervision signals inherent in the intermediate feature space. This leads to low utilization efficiency of unlabeled data, insufficient model recognition accuracy and robustness, and difficulty in adapting to diverse pavement scenarios in real-world engineering.

[0004] To address the aforementioned issues, there is an urgent need for a semi-supervised pavement defect identification scheme that can fully exploit the potential supervisory value of unlabeled data, reduce labeling costs, and effectively utilize intermediate features to improve recognition performance.

[0005] Chinese invention patent CN119625443B provides a method, system, and medium for pathogen classification of pneumonia CT images. The method includes: constructing a training set; performing feature extraction; performing semi-supervised learning training, including supervised learning of labeled images and assigning pseudo-labels to unlabeled images based on predicted probabilities and adaptive confidence thresholds; performing contrastive learning training, including self-supervised contrastive learning of remaining unlabeled images and supervised contrastive learning jointly with labeled and pseudo-labeled images; constructing a joint loss function and performing backpropagation optimization; repeating the steps of feature extraction, semi-supervised learning, contrastive learning, and loss function optimization until training is complete; establishing a classification system based on the trained feature extraction network and classification head, and using the classification system to classify pneumonia CT images. However, this patent for the classification of pneumonia CT images only relies on the prediction results of the classification head to carry out pseudo-label semi-supervised learning and comparative learning. It does not explore the structural and related semantic supervision signals in the intermediate feature space. It has low efficiency in utilizing the potential supervision information of unlabeled data, and lacks a category-aware memory and multi-dimensional feature matching constraints. The feature learning dimension is single, making it difficult for the model to learn more robust and discriminative deep features. The model performance improvement is limited in the case of few labels, and it cannot be directly applied to the application scenario of road surface defect identification. Summary of the Invention

[0006] The purpose of this invention is to overcome the technical problems of existing deep learning-based road surface defect semantic segmentation methods, which rely on large-scale fine pixel-level labeled data, resulting in high labeling costs, and existing semi-supervised road surface defect identification methods, which ignore the supervision signals of intermediate features and have low efficiency in utilizing unlabeled data, thus leading to insufficient identification accuracy and robustness and limited practical engineering application. This invention provides a semi-supervised road surface defect identification method and system with multi-feature matching.

[0007] In a first aspect, the present invention provides a semi-supervised pavement defect identification method based on multi-feature matching, wherein the method inputs an image of pavement defect to be identified into a semi-supervised pavement defect identification model and outputs the pavement defect identification result;

[0008] The semi-supervised pavement defect identification model is constructed through the following steps:

[0009] S1. Obtain road surface distress images, including labeled road surface distress images and unlabeled road surface distress images; perform pixel-level category labeling on the labeled road surface distress images to obtain labeled images;

[0010] Weak enhancement processing and strong enhancement processing are performed on the unlabeled pavement defect images respectively to obtain weakly enhanced unlabeled pavement defect images and strongly enhanced unlabeled pavement defect images;

[0011] S2. A convolutional feature extraction network is used to extract features from the labeled pavement defect image, the weakly enhanced unlabeled pavement defect image, and the strongly enhanced unlabeled pavement defect image, respectively, to obtain labeled intermediate features, unlabeled weakly enhanced intermediate features, and unlabeled strongly enhanced intermediate features.

[0012] S3. Employ a multi-feature fusion matching mechanism to perform cross-view feature matching on the labeled intermediate features and unlabeled weakly enhanced intermediate features, labeled intermediate features and unlabeled strongly enhanced intermediate features, and unlabeled weakly enhanced intermediate features and unlabeled strongly enhanced intermediate features, and output feature matching constraint information.

[0013] A deep convolutional decoding model is used to decode the labeled intermediate features, the unlabeled weakly enhanced intermediate features, and the unlabeled strongly enhanced intermediate features respectively, and outputs a pavement distress prediction map.

[0014] S4. Calculate the feature matching loss based on the feature matching constraint information, and calculate the supervised loss and unsupervised loss based on the pavement defect prediction map; fuse the feature matching loss, the supervised loss, and the unsupervised loss according to preset weights to obtain the total loss value; by minimizing the total loss value, backpropagate to update the parameters of the convolutional feature extraction network, the multi-feature fusion matching mechanism, and the deep convolutional decoding model, and iterate training until the preset convergence condition is met to obtain the semi-supervised pavement defect recognition model.

[0015] By performing strong and weak enhancement processing on unlabeled images, extracting intermediate features using a convolutional feature extraction network, achieving cross-view feature matching through a multi-feature fusion matching mechanism, outputting a predicted map using a deep convolutional decoding model, and iteratively training the model using three types of losses, the following were achieved: generation of multi-view samples of unlabeled data, extraction of effective semantic features of road surface defects, mining of potential supervision signals from unlabeled data, output of pixel-level defect prediction, precise optimization of model parameters, and semi-supervised road surface defect identification. This reduces labeling costs while improving recognition performance.

[0016] Preferably, the weak enhancement processing includes at least one of size adjustment, random cropping, horizontal flipping, and vertical flipping; the strong enhancement processing includes at least one of random color dithering, grayscale conversion, and Gaussian blur.

[0017] By clearly defining the methods of weak and strong augmentation, reliable multi-view learning samples are generated for unlabeled data. Weak augmentation provides the model with a stable learning objective, while strong augmentation forces the model to learn the essential characteristics of the defects, thereby improving the model's ability to resist disturbances in road scene.

[0018] Preferably, the convolutional feature extraction network is a deep convolutional neural network, which includes at least one of the ResNet series, DenseNet series, MobileNet series, and EfficientNet series.

[0019] Deep convolutional neural networks can effectively extract multi-level semantic features of road surface defects. By limiting the type of convolutional feature extraction network, diverse feature extraction schemes can be provided for road surface defect images. The method can be flexibly selected according to the accuracy and lightweight requirements of the detection scenario, thereby improving the engineering practicality of the method.

[0020] Preferably, the multi-feature fusion matching mechanism includes:

[0021] The labeled intermediate features of historical batches are subjected to orthogonal selection strategy and same semantic category aggregation processing to obtain pixel queue and prototype queue, and a memory bank is constructed using the pixel queue and prototype queue; the memory bank is updated progressively according to the first-in-first-out strategy.

[0022] Preprocess the unlabeled weakly enhanced intermediate features, the unlabeled strongly enhanced intermediate features, the pixel queue cached in the memory, and the prototype queue in the current training batch.

[0023] Calculate the first cross-image pairwise feature similarity matrix between unlabeled weakly enhanced intermediate features in the current training batch after preprocessing, and calculate the second cross-image pairwise feature similarity matrix between unlabeled strongly enhanced intermediate features in the current training batch after preprocessing. By minimizing the probability distribution difference metric between the first and second cross-image pairwise feature similarity matrices, the consistency of cross-image pixel-level feature association under different enhanced views is constrained.

[0024] Calculate the first pixel-to-pixel feature similarity matrix between the preprocessed unlabeled weakly enhanced intermediate features of the current training batch and the preprocessed pixel queue, and calculate the second pixel-to-pixel feature similarity matrix between the preprocessed unlabeled strongly enhanced intermediate features of the current training batch and the preprocessed pixel queue. By minimizing the probability distribution difference metric between the first pixel-to-pixel feature similarity matrix and the second pixel-to-pixel feature similarity matrix, constrain the consistency of cross-image pixel-level feature associations between the current batch and historical batches.

[0025] Calculate the similarity matrix between the preprocessed unlabeled weakly enhanced intermediate features of the current training batch and the first pixel-prototype feature similarity matrix of the preprocessed prototype queue, and the similarity matrix between the preprocessed unlabeled strongly enhanced intermediate features of the current training batch and the second pixel-prototype feature similarity matrix of the preprocessed prototype queue. By minimizing the probability distribution difference metric between the first pixel-prototype feature similarity matrix and the second pixel-prototype feature similarity matrix, constrain the pixel features of the current batch to move closer to the pixel-level category prototype feature center of the corresponding disease category.

[0026] The historical batches refer to all batches that have completed training and parameter updates before the current model training during the iterative training of the semi-supervised pavement defect identification model.

[0027] By unifying the preprocessing of features involved in the matching in the multi-feature fusion matching mechanism, calculating the similarity matrix in three categories and constraining consistency, and using the technical features of caching historical features in the memory bank and updating them in the first-in-first-out manner, pixel-level feature association constraints are realized for cross-images in the same batch and cross-images across batches, as well as convergence constraints of pixel features toward the center of the category prototype. This fully explores the supervision signals of intermediate features, improves the utilization efficiency of unlabeled data, and standardizes the distribution of feature space.

[0028] Preferably, the preprocessing includes sequentially performing L2 normalization and reshaping processes, wherein the reshaping process is used to convert two-dimensional intermediate features into one-dimensional feature vectors; the probability distribution difference metric is at least one of Kullback-Leibler divergence, Jensen-Shannon divergence, and cosine distance.

[0029] By limiting the preprocessing method and the type of probability distribution difference metric, the feature scale is unified and the two-dimensional features are converted into one-dimensional feature vectors, ensuring the stability of similarity calculation. At the same time, it provides a variety of difference metric methods that can be flexibly selected according to the matching scenario, thereby improving the adaptability of feature matching.

[0030] Preferably, the deep convolutional decoding model is a semantic segmentation decoder, which includes at least one of the DeepLab series, U-Net series, SegNet series, and PSPNet series.

[0031] By limiting the type of deep convolutional decoding model, diverse pixel-level decoding schemes are provided for intermediate features. Different decoders can adapt to different disease feature mapping requirements, improve the decoding accuracy of intermediate features to disease prediction maps, and ensure pixel-level recognition results.

[0032] Preferably, the supervised loss is the pixel-level cross-entropy loss between the pavement defect prediction map corresponding to the labeled pavement defect image and the labeled image; the unsupervised loss is the consistency loss between the pavement defect prediction map corresponding to the weakly enhanced unlabeled pavement defect image and the pavement defect prediction map corresponding to the strongly enhanced unlabeled pavement defect image; and the feature matching loss is the Kullback-Leibler divergence loss between cross-view feature similarity matrices.

[0033] By clearly defining supervised loss, unsupervised loss, and feature matching loss, we achieve precise supervision of labeled data, consistency constraints of multiple views for unlabeled data, and quantification of distribution differences in feature matching, providing precise loss guidance for model parameter updates and optimizing model learning performance.

[0034] Preferably, the preset convergence condition is that the number of iterations reaches a preset threshold, or the rate of change of the total loss value in consecutive preset rounds is less than or equal to the rate of change threshold.

[0035] By defining the convergence condition as the number of iterations reaching a threshold or the rate of change of the total loss value in consecutive rounds being lower than a threshold, we ensure that the model is trained sufficiently and avoids overfitting, guaranteeing the stability of the model's recognition performance after training and adapting to the needs of different training scenarios.

[0036] Preferably, the pavement defect identification results include pavement defect category information and pixel-level distribution information.

[0037] By defining the technical features of pavement defect identification results, including category information and pixel-level distribution information, the core defect information required for road maintenance is output, the defect types and spatial distribution are clarified, and data support is provided for road maintenance decisions.

[0038] In a second aspect, the present invention provides a semi-supervised pavement defect identification system based on multi-feature matching, wherein the system executes the aforementioned semi-supervised pavement defect identification method based on multi-feature matching during operation.

[0039] By implementing the semi-supervised pavement defect identification method into a system, the method can be engineered and automated, improving the efficiency of pavement defect detection and adapting to the batch detection needs of engineering sites.

[0040] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0041] This invention provides a semi-supervised pavement defect identification method and system based on multi-feature matching. By performing weak and strong enhancement processing on unlabeled pavement defect images during the construction of the semi-supervised pavement defect identification model, the method generates multi-view learning samples of unlabeled data, fully exploring the potential value of unlabeled data. Furthermore, by using a convolutional feature extraction network to extract features from various types of images, the method effectively acquires multi-level semantic features of pavement defects, providing a foundation for subsequent matching and identification.

[0042] This technology utilizes a multi-feature fusion matching mechanism to perform cross-view feature matching on various intermediate features, enabling in-depth mining of intermediate feature space supervision signals and strengthening feature correlation and consistency. It also employs a deep convolutional decoding model to output pavement defect prediction maps, achieving pixel-level accurate prediction of pavement defects. Furthermore, it integrates three types of loss values ​​to iteratively train the model, ensuring precise optimization of model parameters and guaranteeing model learning effectiveness. This approach reduces pixel-level annotation costs while improving the accuracy and robustness of pavement defect identification, adapting to multi-type pavement defect detection scenarios with limited labeled data. Attached Figure Description

[0043] Figure 1 This is a schematic diagram of road surface defects and labeled road surface defects in Example 1.

[0044] Figure 2 This is a schematic diagram of pixel-to-pixel feature matching in the current training batch in Example 1.

[0045] Figure 3 This is a partial schematic diagram of the framework of the semi-supervised pavement defect identification system with multi-feature matching in Example 2.

[0046] Figure 4 This is a comparison chart of representative pavement defects identification results in Example 3. Detailed Implementation

[0047] The present invention will now be described in further detail with reference to specific embodiments. However, this should not be construed as limiting the scope of the present invention to the following embodiments; all technologies implemented based on the content of the present invention fall within the scope of the present invention.

[0048] Unless otherwise specified, the terms "upper," "lower," "left," "right," "center," "inner," and "outer," etc., used in the description of specific embodiments of the present invention to indicate orientation or positional relationships, are based on the orientation or positional relationships shown in the accompanying drawings, or the orientation or positional relationship in which the product / equipment / device is usually placed during use. These terms are merely for the purpose of facilitating the description of the present invention or simplifying the description in specific embodiments, and for enabling those skilled in the art to quickly understand the solution, and do not indicate or imply that a particular device / component / element must have a specific orientation, or be constructed and operated in a specific positional relationship. Therefore, they should not be construed as limitations on the present invention.

[0049] Furthermore, the use of terms such as "horizontal," "vertical," "suspended," "parallel," and "coaxial" does not imply that the corresponding device / component / element must be absolutely horizontal, vertical, suspended, parallel, or coaxial. Slight tilt or deviation is permissible, as long as it does not affect the normal function of the relevant component. For example, "horizontal" simply means that its direction is more horizontal relative to "vertical," not that the structure must be perfectly horizontal; a slight tilt is acceptable. "Coaxial" means that two components are arranged as coaxially as possible, allowing them to move coaxially or approximately coaxially when their relative positions change. Alternatively, it can be simplified to mean that the corresponding device / component / element, when arranged in "horizontal," "vertical," "suspended," "parallel," or "coaxial" directions, can have an error / deviation of ±10% relative to the corresponding direction, more preferably within ±8%, more preferably within ±6%, more preferably within ±5%, and more preferably within ±4%. For example, the deviation in the "coaxial" direction is controlled within 0.2-1mm, preferably within 0.2-0.5mm. As long as the corresponding device / component / element is within the error / deviation range, it can still achieve its function in the solution of the present invention.

[0050] Furthermore, the use of terms such as "first," "second," and "third" in terminology is merely for distinguishing descriptions of identical or similar components and should not be interpreted as emphasizing or implying the relative importance of a particular component.

[0051] Furthermore, in the description of the embodiments of the present invention, "several", "more than", and "a number of" represent at least two. The number can be any number, such as two, three, four, five, six, seven, eight, or nine, and can even exceed nine.

[0052] Furthermore, in the description of the technical solution of this invention, unless otherwise explicitly specified / limited / restricted, the terms "set up," "install," "connect," "link," "provided with," "laid out," and "arranged" should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to connection methods commonly used in the art, such as welding, riveting, bolting, and threaded connections. Such connections can be mechanical, electrical, or communication connections; they can be direct connections or indirect connections through an intermediate medium; and they can refer to the internal communication between two components.

[0053] Example 1

[0054] This embodiment describes a semi-supervised pavement defect identification method based on multi-feature matching. The method inputs the pavement defect image to be identified into a semi-supervised pavement defect identification model and outputs the pavement defect identification result.

[0055] The semi-supervised pavement defect identification model is constructed through the following steps:

[0056] S1. Obtain pavement distress images, including labeled pavement distress images and unlabeled pavement distress images; perform pixel-level category labeling on the labeled pavement distress images to obtain labeled images; perform weak enhancement processing and strong enhancement processing on the unlabeled pavement distress images to obtain weakly enhanced unlabeled pavement distress images and strongly enhanced unlabeled pavement distress images.

[0057] All pavement distress image data used in this embodiment can be acquired using a 3D laser imaging system. This system can simultaneously acquire 2D images and precisely pixel-level aligned 3D pavement surface data. Each set of data includes a 2D pavement image, a corresponding 3D image, and a ground truth map, such as... Figure 1 As shown, different pavement defects are labeled with different colors at the pixel level in the image. This data was collected over the past four years from more than 20 asphalt pavement sections in China. The realistic annotations cover a variety of pavement defects, including cracks, potholes, sealants, repairs, and scratches, all manually generated and stored in PNG image format. All images have a uniform resolution of 512×256 pixels.

[0058] Dataset Construction: A training set was constructed by collecting pavement distress images. The training set included both labeled and unlabeled pavement distress images. The labeled pavement distress images underwent pixel-level category annotation to obtain labeled images. The labeled images provided pixel-level semantic annotations for common pavement distresses such as cracks, potholes, crack sealing, repairs, and scratches. The unlabeled pavement distress images only contained pavement distress images without any category annotation information.

[0059] The entire dataset was randomly divided into three subsets: 10,000 images for training, 3,000 images for validation, and 2,000 images for testing.

[0060] Image enhancement processing: Weak enhancement processing and strong enhancement processing are performed on all unlabeled pavement distress images to generate multi-view unlabeled samples, namely, weakly enhanced unlabeled pavement distress images and strongly enhanced unlabeled pavement distress images.

[0061] Weak enhancement processing employs one or more combinations of resizing, random cropping, horizontal flipping, and vertical flipping to provide a relatively reliable learning target for model training. Weak enhancement processing can also employ other lightweight image enhancement methods known in the field, as long as the core features of road surface defects are not altered.

[0062] The enhanced processing employs one or more combinations of random color dithering, grayscale conversion, and Gaussian blur to force the model to learn the essential characteristics of road surface defects and improve the model's resistance to interference from different shooting environments and textured backgrounds.

[0063] The labeled pavement defect image is defined as follows: The pixel-level category-labeled image is Unlabeled road surface defect images are The image of unlabeled pavement defects after weak enhancement is The enhanced image of unlabeled pavement defects is .in For index, indicating the first Zhang image.

[0064] S2. A convolutional feature extraction network is used to extract features from the labeled pavement defect image, the weakly enhanced unlabeled pavement defect image, and the strongly enhanced unlabeled pavement defect image, respectively, to obtain labeled intermediate features, unlabeled weakly enhanced intermediate features, and unlabeled strongly enhanced intermediate features.

[0065] Convolutional feature extraction networks were used to respectively... , , Perform feature extraction and output the corresponding intermediate features with labels. Unlabeled weakly enhanced intermediate features ( Unlabeled strongly enhanced intermediate features ( The convolutional feature extraction network is a deep convolutional neural network, which can be selected from the ResNet series, DenseNet series, MobileNet series, and EfficientNet series. In this embodiment, ResNet50, i.e., a residual network with 50 layers, is preferred. This network extracts low-dimensional texture features and high-dimensional semantic features of road surface defects through multi-layer convolution, pooling, and residual connections. The output intermediate features are two-dimensional feature matrices, which are suitable for subsequent preprocessing and feature matching operations.

[0066] S3. A multi-feature fusion matching mechanism is used to perform cross-view feature matching between the labeled intermediate features and the unlabeled weakly enhanced intermediate features, the labeled intermediate features and the unlabeled strongly enhanced intermediate features, and the unlabeled weakly enhanced intermediate features and the unlabeled strongly enhanced intermediate features, and output feature matching constraint information; a deep convolutional decoding model is used to decode the labeled intermediate features, the unlabeled weakly enhanced intermediate features and the unlabeled strongly enhanced intermediate features respectively, and output road surface defect prediction map.

[0067] The multi-feature fusion matching mechanism is invoked to perform cross-view feature matching on various intermediate features output by S2. The specific execution logic is as follows: first, preprocess all feature maps involved in the matching, then complete the matching of the three types of features in parallel, and output feature matching constraint information to provide a basis for feature matching loss calculation.

[0068] Preprocessing: The unlabeled weakly enhanced intermediate features, the unlabeled strongly enhanced intermediate features, the pixel queue cached in the memory bank, and the prototype queue are preprocessed respectively, namely, L2 normalization and reshaping are performed sequentially. L2 normalization unifies the feature scale to ensure the stability of subsequent similarity calculations, and reshaping converts the two-dimensional intermediate features into one-dimensional feature vectors to provide standardized input for feature matching across images and batches.

[0069] Memory configuration: The labeled intermediate features of historical batches are subjected to orthogonal selection strategy and same semantic category aggregation processing to obtain pixel queue and prototype queue. The memory is constructed with the pixel queue and prototype queue and is updated gradually according to the first-in-first-out strategy to avoid redundant accumulation of historical feature data and ensure the timeliness of feature matching.

[0070] Three-class feature matching execution:

[0071] Pixel-to-pixel feature matching within the same batch: Calculate the first cross-image pairwise feature similarity matrix between unlabeled weakly enhanced intermediate features within the current training batch after preprocessing, and calculate the second cross-image pairwise feature similarity matrix between unlabeled strongly enhanced intermediate features within the current training batch after preprocessing. By minimizing the probability distribution difference metric between the first and second cross-image pairwise feature similarity matrices, the consistency of cross-image pixel-level feature association under different enhanced views is constrained.

[0072] Pixel-to-pixel feature matching based on memory bank: Calculate the first pixel-to-pixel feature similarity matrix between the preprocessed unlabeled weakly enhanced intermediate features of the current training batch and the preprocessed pixel queue; calculate the second pixel-to-pixel feature similarity matrix between the preprocessed unlabeled strongly enhanced intermediate features of the current training batch and the preprocessed pixel queue; constrain the consistency of cross-image pixel-level feature associations between the current batch and historical batches by minimizing the probability distribution difference metric between the first pixel-to-pixel feature similarity matrix and the second pixel-to-pixel feature similarity matrix.

[0073] Pixel-to-prototype feature matching based on memory bank: The similarity matrix between the preprocessed unlabeled weakly enhanced intermediate features of the current training batch and the first pixel-to-prototype feature similarity matrix of the preprocessed prototype queue is calculated, as well as the similarity matrix between the preprocessed unlabeled strongly enhanced intermediate features of the current training batch and the second pixel-to-prototype feature similarity matrix of the preprocessed prototype queue. By minimizing the probability distribution difference metric between the first and second pixel-to-prototype feature similarity matrices, the pixel features of the current batch are constrained to converge towards the pixel-level category prototype feature center of the corresponding disease category. The aforementioned probability distribution difference metric can be selected from one or more combinations of Kullback-Leibler divergence, Jensen-Shannon divergence, and cosine distance. This embodiment preferably uses Kullback-Leibler divergence. Clustering can employ well-known clustering algorithms in the field, such as K-means and hierarchical clustering, all of which can achieve the aggregation of similar features and the generation of category prototype feature centers.

[0074] In detail, pixel-to-pixel feature matching within the same batch is as follows: Figure 2 As shown: Let the unlabeled intermediate features in the current training batch be... The unlabeled weakly enhanced intermediate features in the current training batch are: The unlabeled strongly enhanced intermediate features in the current training batch are ;in The batch size for unlabeled data; for any pair of unlabeled weakly augmented intermediate features in the current training batch. and (in , Indicates the first Zhang's unlabeled weakly enhanced intermediate features Indicates the first Zhang's unlabeled weakly enhanced intermediate features), whose corresponding intermediate features are represented as follows: , For the set of real numbers, The height of the feature map, The width of the feature map. This represents the depth of the feature map. After L2 normalization and reshaping, the feature embedding... and Convert to (in , (This refers to the total number of pixels in one dimension obtained by flattening the two-dimensional feature map). The pairwise feature similarity matrix across images is calculated as follows: This matrix encodes pixel-level feature correspondences across images.

[0075] Similarly, corresponding strongly enhanced images and Similarity matrix Calculated in the same manner. Intuitively, the cross-image pixel-level feature associations corresponding to different augmented versions of the same data should remain consistent. To reinforce this consistency, the similarity matrix obtained from strong augmentation is optimized by minimizing the Kullback-Leibler (KL) divergence. The matrix obtained by weak enhancement Alignment.

[0076] Pixel-to-pixel feature matching based on a memory bank: establishing and maintaining a category-aware pixel queue. ,in Represents the number of categories. This indicates the number of pixel features stored in each category. This represents the dimension of the feature embedding. In each iteration, only a small number of images from each image in the current training batch are selected. Add representative pixel features to the queue And the queue capacity satisfies In each training batch, all intermediate features extracted from the labeled images are embedded... All are considered pixel queues Candidate entries are selected. To avoid the loss of key representative information that may be caused by random sampling, an orthogonal selection strategy is adopted to greedily select the most representative feature embeddings. Specifically, for labeled images... Gradually construct a set of representative features Each representative feature is derived from the feature embedding. obtained from sampling, This involves selecting representative pixel features from a single labeled image. The selection process begins by randomly choosing a pixel feature, and then iteratively adds new features that have the strongest orthogonality to the selected feature (i.e., the lowest cosine similarity). This process is applied to all labeled images within the current training batch, ultimately forming a comprehensive set of representative features. , This is the batch size for labeled data. Then... Pixel features are organized by category and enqueued into a pixel queue. All embeddings are L2 normalized to ensure the stability of similarity calculations and the consistency of representation scale. The pixel queue is progressively updated in a first-in, first-out manner throughout the training process.

[0077] In pixel-level queue Based on the weak enhancement view and enhanced view Generated feature embeddings and It is considered an anchor point. For each category ( (Number of road surface defects categories), from the corresponding category-specific queues. (Right now Randomly sample a set of contrastive embeddings from a subset of (the subset of the set ... . For the first A set of contrasting pixel features for disease-like organisms. For the first The first type of disease Each contrasting pixel feature embedding, Dimensions and data types of features for a single contrasting pixel ( (dimensional real vector) For each type of disease, from the pixel queue The number of contrastive pixel features sampled in the middle. The sampled embeddings of all classes are then concatenated to form a complete set of contrastive features. , which contains Each feature is embedded. Based on these representations, the similarity matrix between the anchor embedding and the contrast embedding is calculated for both weakly and strongly augmented views. and Finally, by minimizing and The KL divergence between them is used to align them with each other, thereby forcing pixel-level feature association consistency based on the memory bank.

[0078] and The calculation formula is as follows:

[0079]

[0080]

[0081] Pixel queue size The sample size is set to 4096 samples per class, with a maximum of 100 samples per image. Each pixel feature. Originally, in each training iteration, random samples were taken from each class in the corresponding queue. Each pixel embedding is used as a contrastive embedding. All pixel embeddings are extracted from the output with a channel dimension of d=256.

[0082] Pixel-to-prototype feature matching based on memory bank: establishing a category-aware prototype queue ,in This represents the number of embeddings for each prototype. Each prototype embedding is obtained by aggregating pixel-level feature embeddings belonging to the same semantic category within the same image, serving as class-level feature centers to supplement the supervision signal provided by individual pixel features. Specifically, for each semantic category... Prototype embedding calculates the category in the labeled image. The average value of all pixel embeddings is obtained and then added to the prototype queue for dynamic updating during training. (This is related to the pixel queue.) Similarly, prototype queue Updates are performed following a first-in, first-out (FIFO) strategy. To constrain the feature correlation between pixel features and class prototypes under different enhancement perspectives, updates are performed from the prototype queue in each training iteration. Random sampling for each category These contrastive prototype embeddings are concatenated to form a comprehensive contrastive prototype feature set. . In order to target the first target Road surface defects are identified from the class-aware prototype queue in the memory bank. Random sampling A set of contrastive prototype features is composed of prototype feature embeddings. For unlabeled images... Calculate the feature maps obtained from the weakly enhanced view and the strongly enhanced view respectively. , With the prototype embedding of the sample The pixel-prototype similarity matrix between them is denoted as follows: and By minimizing and The KL divergence between pixels is used to force feature consistency from pixel to prototype.

[0083] and The calculation formula is as follows:

[0084]

[0085]

[0086] Prototype queue size The number of samples per class is fixed at 512. During each training iteration, samples are randomly selected from each class in the corresponding queue. Each prototype embedding is used as a contrastive embedding. All prototype embeddings are extracted from the output with a channel dimension of d=256.

[0087] A deep convolutional decoding model is used to decode labeled intermediate features, unlabeled weakly enhanced intermediate features, and unlabeled strongly enhanced intermediate features, respectively, and output the corresponding road surface defect prediction map. The deep convolutional decoding model is a semantic segmentation decoder, which can be selected from the DeepLab series, U-Net series, SegNet series, and PSPNet series. In this embodiment, DeepLabV3+ is preferred. This decoder maps the intermediate features output by the convolutional feature extraction network to a road surface defect prediction map that matches the size of the original image through upsampling, feature fusion, and pixel-level classification, thereby realizing pixel-level localization and category prediction of defect areas.

[0088] During decoding, the input is as follows and And output the corresponding prediction results respectively. and ; Input the above and output the prediction results. .

[0089] S4. Calculate the feature matching loss based on the feature matching constraint information, and calculate the supervised loss and unsupervised loss based on the pavement defect prediction map; fuse the feature matching loss, the supervised loss, and the unsupervised loss according to preset weights to obtain the total loss value; by minimizing the total loss value, backpropagate to update the parameters of the convolutional feature extraction network, the multi-feature fusion matching mechanism, and the deep convolutional decoding model, and iterate training until the preset convergence condition is met to obtain the semi-supervised pavement defect recognition model.

[0090] Multi-type loss calculation:

[0091] Based on the pavement defect prediction map and label image corresponding to the labeled pavement defect image, the supervision loss is calculated. The supervision loss is pixel-level cross-entropy loss, so as to achieve accurate supervision of labeled data.

[0092] Based on the pavement defect prediction maps corresponding to the unlabeled pavement defect images after weak and strong enhancement, the unsupervised loss is calculated. The unsupervised loss is the consistency loss, which realizes the prediction consistency constraint of multi-view unlabeled data.

[0093] Based on the feature matching constraint information output by S3, the feature matching loss is calculated. The feature matching loss is the loss of the probability distribution difference between the feature similarity matrices across views, thus realizing the consistency constraint at the feature level.

[0094] Joint loss fusion and model optimization: The supervised loss, unsupervised loss, and feature matching loss are fused according to preset weights to obtain the total loss value. The weight coefficients can be flexibly adjusted according to the number of labels and the distribution of disease categories in the road damage dataset. By minimizing the total loss value, well-known optimization algorithms in the field, such as stochastic gradient descent (SGD) and the adaptive moment estimation optimizer W version (AdamW), are used to backpropagate and update the parameters of the convolutional feature extraction network, the multi-feature fusion matching mechanism, and the deep convolutional decoding model, and iteratively train until the preset convergence condition is met.

[0095] Model optimization employed stochastic gradient descent (SGD) with an initial learning rate of 0.005, which was adjusted using a multinomial decay strategy during training. All methods were trained for 80 epochs with a batch size of 16, where each current training batch contained 8 labeled images and 8 unlabeled images.

[0096] Supervised loss for labeled images For the and Standard pixel-level cross-entropy loss between them;

[0097] Unsupervised loss for unlabeled images Forcing model and Maintain consistency;

[0098] Pixel-to-pixel feature matching loss for the current training batch of unlabeled images For the and KL divergence between them;

[0099] Pixel-to-pixel feature matching loss based on memory for labeled and unlabeled images For the and KL divergence between them;

[0100] Pixel-to-prototype feature matching loss based on memory for labeled and unlabeled images For the and The KL divergence between them.

[0101] The joint loss function is:

[0102] Supervised loss for labeled images For the and The standard pixel-level cross-entropy loss between them is calculated as follows:

[0103]

[0104] in, This is the batch size for labeled data; Representation model pairs The prediction This represents the cross-entropy loss function.

[0105] Unsupervised loss for unlabeled images Forcing model and To maintain consistency, the formula is as follows:

[0106]

[0107] in, The batch size for unlabeled data; and They are and The model prediction results; It is a preset confidence threshold used to reduce the impact of noise by excluding low-confidence false labels; while This represents the indicator function. By applying consistency regularization at the current training batch level, the model can learn reliable information from unlabeled data.

[0108] Pixel-to-pixel feature matching loss for the current training batch of unlabeled images For the and The KL divergence between them is given by the following formula:

[0109]

[0110] in, For unlabeled data batch size, KL loss function

[0111] Pixel-to-pixel feature matching loss based on memory for labeled and unlabeled images For the and The KL divergence between them is given by the following formula:

[0112]

[0113] Pixel-to-prototype feature matching loss based on memory for labeled and unlabeled images For the and The KL divergence between them is given by the following formula:

[0114]

[0115] Joint loss function The calculation formula is as follows:

[0116]

[0117] in and This is a weighting factor used to balance the contribution weights of each loss component.

[0118] S5, Application of Road Surface Defect Identification Model

[0119] A semi-supervised pavement defect identification model is constructed by integrating a trained convolutional feature extraction network, a multi-feature fusion matching mechanism, and a deep convolutional decoding model. The pavement defect image to be identified is input into the model, which sequentially performs feature extraction, multi-feature fusion matching, and decoding prediction, and finally outputs the pavement defect identification result. The identification result includes the category information of the pavement defect (cracks, potholes, crack sealing, etc.) and pixel-level distribution information (the spatial location, range, and area of ​​the defect in the image), providing accurate data support for road maintenance decisions.

[0120] Example 2

[0121] This embodiment is based on a semi-supervised pavement defect identification system using multi-feature matching. The overall system block diagram is as follows: Figure 3 As shown.

[0122] This system adopts a modular architecture, including a data acquisition module, an image enhancement module, a convolutional feature extraction module, a multi-feature fusion and matching module, a deep convolutional decoding module, a loss calculation and parameter optimization module, and a recognition result output module. These modules communicate bidirectionally via data interfaces to transmit feature maps, prediction maps, and loss values. The convolutional feature extraction module and the deep convolutional decoding module are each divided into two sub-modules, and these sub-modules employ parameter sharing mechanisms during training to ensure consistency between feature extraction and decoding prediction. The multi-feature fusion and matching module incorporates a category-aware memory (including a pixel queue and a prototype queue) to cache historical features, dynamically update them, and select representative features. Specific implementation details are as follows:

[0123] 1. Data Acquisition Module

[0124] The semi-supervised pavement distress image dataset was constructed, standardized, and input to provide basic image data for subsequent modules, and strictly distinguished between labeled and unlabeled pavement distress image types.

[0125] It supports three data input methods: local image file import, real-time data acquisition from road inspection equipment, and cloud dataset download, and is compatible with mainstream image formats. All input pavement distress images undergo standardization processing, including unified image resolution, image format conversion, and pixel value normalization. The processed images are divided into labeled and unlabeled pavement distress image sets. Each image in the labeled pavement distress image set is labeled with pixel-level category annotations, resulting in labeled images. These labeled images provide pixel-level semantic annotations for pavement distresses such as cracks, potholes, crack sealing, repairs, and scratches. The unlabeled pavement distress image set only retains the pavement distress images without any category annotation information. It supports batch import of multiple large-scale pavement distress datasets and utilizes multi-threading technology to improve data reading and standardization processing efficiency, adapting to pavement distress datasets of different sizes and acquisition scenarios.

[0126] 2. Image enhancement module

[0127] We apply weak and strong enhancement processing to unlabeled pavement defect images to generate multi-view unlabeled samples, providing a foundation for subsequent cross-view feature matching and unsupervised loss calculation.

[0128] The system includes a built-in weak enhancement processing unit and a strong enhancement processing unit. The two units operate independently and support custom configuration of enhancement strategies. The processing results are output to the convolutional feature extraction module. The weak enhancement processing unit performs one or more combined operations on the unlabeled pavement defect image, such as resizing, random cropping, horizontal flipping, and vertical flipping, to generate a weakly enhanced unlabeled pavement defect image, providing a relatively reliable learning target for model training. The strong enhancement processing unit performs one or more combined operations on the unlabeled pavement defect image, such as random color jittering, grayscale conversion, and Gaussian blurring, to generate a strongly enhanced unlabeled pavement defect image, forcing the model to learn the essential features of pavement defects under stronger perturbation.

[0129] Image enhancement algorithms accelerated by graphics processors can be used to improve the enhancement processing speed of large batches of unlabeled images; image enhancement toolkits known in the field can also be used to expand the enhancement methods without changing the core processing logic of weak and strong enhancement.

[0130] 3. Convolutional Feature Extraction Module

[0131] It adopts a deep convolutional neural network architecture, which is divided into two sub-modules: encoding module 1 and encoding module 2. The two sub-modules fully share model parameters during training, ensuring the consistency of feature extraction logic and the synergy of parameter optimization. It supports loading and switching of various network models such as residual network series, dense network series, lightweight mobile network series, and high-efficiency network series, which can be flexibly selected according to the accuracy and computing power requirements of the actual application scenario.

[0132] Encoding Module 1: Receives labeled pavement distress images and weakly enhanced unlabeled pavement distress images as input, extracts low-dimensional texture features and high-dimensional semantic features of pavement distress through multiple convolutional layers, pooling layers, and residual connection layers, and outputs labeled intermediate features and unlabeled weakly enhanced intermediate features respectively.

[0133] Encoding module 2: Receives the strongly enhanced unlabeled pavement defect image as input, and outputs the unlabeled strongly enhanced intermediate features using the same feature extraction logic as encoding module 1;

[0134] All intermediate features are in two-dimensional matrix format, suitable for preprocessing and feature matching operations in multi-feature fusion matching modules. It supports loading pre-trained weights to improve disease feature extraction capabilities and can be replaced with a lightweight deep convolutional neural network, adapting to low-computing-power hardware environments for edge detection devices.

[0135] 4. Multi-feature fusion matching module

[0136] The intermediate features output by the convolutional feature extraction module are uniformly preprocessed and cross-view feature matching is performed to mine potential supervision signals of unlabeled data, enhance the cross-image pixel-level feature correlation, standardize the category correlation of the feature space, output feature matching constraint information, and provide a basis for feature matching loss calculation.

[0137] The preprocessing unit preprocesses the unlabeled weakly enhanced intermediate features, the unlabeled strongly enhanced intermediate features, the pixel queue cached in the memory, and the prototype queue output by the convolutional feature extraction module in the current training batch, respectively. Specifically, it performs L2 normalization and reshaping. L2 normalization unifies the feature scale and ensures the stability of subsequent similarity matrix calculations. Reshaping converts the two-dimensional intermediate features into one-dimensional feature vectors, providing standardized input for feature matching across images and batches.

[0138] Category-aware feature storage queue (memory): The labeled intermediate features of historical batches are subjected to orthogonal selection strategy and same semantic category aggregation processing to obtain pixel queue and prototype queue, and the memory is constructed using the pixel queue and prototype queue; it is a feature storage queue that is dynamically updated during semi-supervised training, and is updated progressively according to the first-in-first-out strategy. The queue capacity can be flexibly adjusted according to the hardware memory configuration; it supports fast reading and writing of feature maps, provides historical feature benchmarks for cross-batch feature matching, and avoids statistical bias caused by a single training batch.

[0139] Three types of feature matching calculation units: The built-in similarity matrix calculation engine and probability distribution difference measurement engine support switching between multiple measurement methods such as relative entropy divergence, Jensen-Shannon divergence, and cosine distance. It completes three types of feature matching and outputs similarity matrices of each type as feature matching constraint information.

[0140] The first cross-image pairwise feature similarity matrix between unlabeled weakly enhanced intermediate features in the current training batch is calculated for cross-image pixel-level feature matching in the same batch. The second cross-image pairwise feature similarity matrix between unlabeled strongly enhanced intermediate features in the current training batch is calculated for cross-image pixel-level feature matching in the same batch. By minimizing the probability distribution difference between the first and second cross-image pairwise feature similarity matrices, the consistency of cross-image pixel-level feature association under different enhanced views is constrained.

[0141] Cross-batch, cross-image pixel-level feature matching: Calculate the first pixel-to-pixel feature similarity matrix between the preprocessed unlabeled weakly enhanced intermediate features of the current training batch and the preprocessed pixel queue; calculate the second pixel-to-pixel feature similarity matrix between the preprocessed unlabeled strongly enhanced intermediate features of the current training batch and the preprocessed pixel queue; constrain the consistency of cross-image pixel-level feature associations between the current batch and historical batches by minimizing the probability distribution difference metric between the first pixel-to-pixel feature similarity matrix and the second pixel-to-pixel feature similarity matrix.

[0142] Pixel-Category Prototype Feature Matching: Calculate the similarity matrix between the preprocessed unlabeled weakly enhanced intermediate features of the current training batch and the first pixel-prototype feature similarity matrix of the preprocessed prototype queue, and the similarity matrix between the preprocessed unlabeled strongly enhanced intermediate features of the current training batch and the second pixel-prototype feature similarity matrix of the preprocessed prototype queue. By minimizing the probability distribution difference metric between the first pixel-prototype feature similarity matrix and the second pixel-prototype feature similarity matrix, constrain the pixel features of the current batch to move closer to the pixel-level category prototype feature center of the corresponding disease category, thereby improving the class separability of the feature space.

[0143] 5. Depthwise Convolution Decoding Module

[0144] The system adopts a semantic segmentation decoder architecture, which is divided into two sub-modules: decoding module 1 and decoding module 2. The two sub-modules fully share model parameters during training, ensuring the consistency of decoding logic and the synergy of parameter optimization. The system supports loading and switching of various decoders, such as the Deep Lab series, U-shaped network series, segmented network series, and pyramid pooling network series. It upsamples, fuses, and classifies various intermediate features output by the convolutional feature extraction module, and outputs a road surface defect prediction map that matches the size of the original image, providing a basis for the calculation of supervised and unsupervised loss.

[0145] Decoding Module 1: Receives the labeled intermediate features and unlabeled weakly enhanced intermediate features output by Encoding Module 1 as input, restores the feature map size through an upsampling layer, fuses multi-scale disease features through a feature fusion layer, and predicts the disease category through a pixel-level classification layer, and outputs the road disease prediction map corresponding to the labeled road disease image and the road disease prediction map corresponding to the weakly enhanced unlabeled road disease image, respectively.

[0146] Decoding module 2: Receives the unlabeled strongly enhanced intermediate features output by encoding module 2 as input, and outputs a pavement defect prediction map corresponding to the strongly enhanced unlabeled pavement defect image using the same decoding logic as decoding module 1.

[0147] All predicted maps include the probability value of the road surface defect category for each pixel, enabling pixel-level localization and category prediction of road defects. Batch decoding of multiple intermediate features is supported, and parameter updates are coordinated with the convolutional feature extraction module to ensure consistency between feature extraction and decoding prediction.

[0148] 6. Loss Calculation and Parameter Optimization Module

[0149] The supervised loss, unsupervised loss, and feature matching loss are independently calculated and jointly fused. By minimizing the total loss value and backpropagating, the model parameters of the convolutional feature extraction module, the multi-feature fusion matching module, and the deep convolutional decoding module are updated to achieve iterative optimization training of the model.

[0150] It incorporates three types of loss calculation units, a joint loss fusion unit, and a parameter optimization unit, with each unit operating in a logically linked manner: "independent calculation - weighted fusion - parameter optimization".

[0151] Three types of loss calculation units: Dedicated calculation engines are constructed for supervised loss, unsupervised loss, and feature matching loss, respectively, to perform calculations: Supervised loss is the pixel-level cross-entropy loss between the pavement defect prediction map corresponding to the labeled pavement defect image and the labeled image with pixel-level category annotation; Unsupervised loss is the consistency loss between the pavement defect prediction map corresponding to the weakly enhanced unlabeled pavement defect image and the strongly enhanced unlabeled pavement defect image; Feature matching loss is the relative entropy divergence loss between the cross-view feature similarity matrices output by the multi-feature fusion matching module.

[0152] Joint loss fusion unit: The supervised loss, unsupervised loss, and feature matching loss are weighted and summed according to the user-preset weight coefficients to generate the total loss value; the weight coefficients can be flexibly adjusted according to the number of labels and the distribution of disease categories in the road damage dataset to achieve a reasonable balance between different loss components.

[0153] The parameter optimization unit loads well-known optimization algorithms in the field, such as stochastic gradient descent and adaptive moment estimation. By minimizing the total loss value through backpropagation, it synchronously updates all model parameters of the convolutional feature extraction module, the multi-feature fusion matching module, and the deep convolutional decoding module. It supports setting a threshold for the number of iterations and a threshold for the rate of change of the total loss value in consecutive preset rounds as preset convergence conditions. Training is automatically terminated when either condition is met to avoid model overfitting. After training, a converged semi-supervised pavement disease identification model is obtained.

[0154] It supports mixed-precision training to improve the speed of model iteration training; it can also be integrated with learning rate decay strategies to further optimize model training performance.

[0155] 7. Recognition Result Output Module

[0156] The trained semi-supervised pavement defect recognition model is applied to actual detection scenarios to complete end-to-end recognition of pavement defect images and output visualized and structured pavement defect recognition results, providing accurate data support for road maintenance decisions.

[0157] It supports single and batch input of road surface defects images to be identified. After the images to be identified are processed by the convolutional feature extraction module, the multi-feature fusion matching module, and the deep convolutional decoding module, the core recognition results are output, and visualization and structured output functions are provided.

[0158] It strictly outputs the category information and pixel-level distribution information of road surface defects. The category information includes common road surface defect types such as cracks, potholes, crack sealing, repair, and scratches. The pixel-level distribution information includes the spatial coordinates, coverage area, and pixel area of ​​the defects in the image, accurately quantifying the geometric features of road surface defects.

[0159] It provides a visual annotation function for diseased areas, which marks the identified diseased areas on the original road surface disease image with color pixels, intuitively showing the location and shape of the disease; at the same time, it generates a structured recognition report in tabular format, which includes statistical information such as disease type, quantity, and pixel ratio, and supports the export and saving of the report.

[0160] It supports two modes: offline recognition and cloud-based remote recognition. In offline recognition mode, the model is deployed on local road detection equipment to achieve real-time on-site recognition. In cloud-based remote recognition mode, the model is deployed on a cloud server and provides remote recognition services to multiple terminal devices through an application programming interface.

[0161] System workflow:

[0162] The data acquisition module constructs and inputs a semi-supervised pavement distress image dataset, completes image standardization processing, and distinguishes between labeled and unlabeled pavement distress images.

[0163] The image enhancement module performs weak enhancement processing and strong enhancement processing on the unlabeled pavement defect images respectively, generating weakly enhanced and strongly enhanced unlabeled pavement defect images;

[0164] The convolutional feature extraction module extracts features from labeled pavement distress images, weakly enhanced and strongly enhanced unlabeled pavement distress images, and outputs intermediate features of labeled, unlabeled weakly enhanced and unlabeled strongly enhanced pavement distress images.

[0165] The multi-feature fusion matching module preprocesses various intermediate features and performs three types of cross-view feature matching in parallel, outputting feature matching constraint information.

[0166] The deep convolutional decoding module decodes various intermediate features and outputs the corresponding pavement distress prediction map;

[0167] The loss calculation and parameter optimization module calculates supervised loss, unsupervised loss, and feature matching loss respectively, and merges them into a total loss value according to preset weights. The model parameters are updated by backpropagation by minimizing the total loss value, and the training is iterated until the preset convergence condition is met, thus completing the training of the semi-supervised pavement disease identification model.

[0168] The images of road surface defects to be identified are input into the semi-supervised road surface defect identification system that has been trained.

[0169] The image to be identified is sequentially processed by the convolutional feature extraction module to extract features, the multi-feature fusion and matching module to optimize features, and the deep convolutional decoding module to predict pixel-level diseases.

[0170] The recognition result output module analyzes the prediction results of the deep convolutional decoding module and outputs the category information and pixel-level distribution information of road surface defects. At the same time, it provides visual annotation of defect areas and structured recognition reports to complete the intelligent recognition of road surface defects.

[0171] Example 3

[0172] In this embodiment, the method proposed in this invention is compared and evaluated with five advanced semi-supervised semantic segmentation methods, including CPS using a collaborative training paradigm, and UniMatch, CorrMatch, RankMatch, and MCCL based on weak-to-strong consistency regularization. To comprehensively evaluate the effectiveness of the proposed method in semi-supervised pavement distress segmentation, extensive comparative experiments were conducted under various data partitioning protocols.

[0173] Following common practice, the average Intersection over Union (IOU) of all categories is used as the main indicator for evaluating segmentation performance. IOU measures segmentation quality by quantifying the proportion of overlap between the predicted region and the corresponding ground truth labeled region relative to their union. Table 1 shows the IOU results for each disease category under different data partitioning protocols (number of labeled images / number of unlabeled images), and the corresponding average IOU (mIOU). All methods use ResNet50 as the encoder and DeepLabV3+ as the decoder. "Labeled Only" refers to a fully supervised baseline trained using only the specified proportion of labeled data, without utilizing any unlabeled samples. As shown in Table 1, the proposed semi-supervised method consistently outperforms the fully supervised baseline under all partitioning protocols. In particular, under the 1 / 20 (500) partitioning protocol, compared to the fully supervised baseline, the IOU for each disease category—cracks, potholes, grouting, repair, and scratches—is improved by 8.77%, 12.18%, 5.95%, 26.10%, and 21.51%, respectively, with an overall mIOU gain of 14.91%. It is worth noting that the pavement distress dataset suffers from significant class imbalance, with potholes, repairs, and scratches having far fewer samples than cracks and grout. Despite this imbalance, the proposed method still achieves an IOU improvement of over 10% on these few classes, demonstrating strong robustness with limited labeled data and effectively mitigating the performance degradation caused by data imbalance. Furthermore, compared to recent state-of-the-art semi-supervised semantic segmentation methods, the method of this invention achieves superior IOU performance across all segmentation protocols and distress categories. These results indicate that intermediate features encode richer semantic information, and implementing feature-level consistency constraints between weak and strong enhancement perspectives enables more effective utilization of unlabeled data.

[0174] To further evaluate the performance of the method of the present invention, Figure 4 Representative segmentation results of different methods under the 1 / 5 partitioning protocol are presented. As shown in the first column, all methods can accurately segment cracks, which is consistent with the quantification results in Table 1—cracks achieved the highest IOU among all disease categories. This is mainly attributed to its high proportion in the dataset and relatively simple morphological features. In contrast, the "Labeled Only" baseline performs worse in processing... Figure 4 In the second and third images, false negatives (FNs) increased significantly, especially for segmentation of cracks, pits, and repairs. Although other advanced semi-supervised methods mitigate this problem by utilizing implicit supervision signals in unlabeled data, their segmentation results still suffer from fragmentation and incompleteness. In contrast, the method proposed in this invention produces more coherent and complete segmentation, particularly for complex crack morphologies and pit regions. Furthermore, as... Figure 4As shown in columns two and five, other methods tend to produce significant local noise when segmenting large repair areas. In contrast, the prediction results of this invention are clearer, more accurate, and visually closer to the actual annotations. Overall, these results further confirm the conclusions of the quantitative analysis, demonstrating that the proposed method significantly improves the accuracy and robustness of pavement distress segmentation.

[0175] In Table 1, CPS stands for Progressive Separation Co-training, RankMatch stands for Ranking Matching, UniMatch stands for Unified Semi-Supervised Semantic Segmentation, CorrMatch stands for Relevance Matching, and MCCL stands for Multi-Scale Contrast Consistency Learning.

[0176] In Table 1, "Crack" refers to a road surface crack, "Pothole" to a pothole or crater, "Sealed crack" to a crack filled or sealed, "Patch" to a road surface repair area or repair site, and "Scratch" to a road surface scratch. This represents the improvement in IOU (Intersection over Union) of the method of this invention compared to the fully supervised baseline.

[0177] Table 1 shows the IOU results of the seven methods on test datasets with different partitioning protocols.

[0178]

[0179] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A semi-supervised method for identifying pavement defects using multi-feature matching, characterized in that, The method inputs the image of the road surface defect to be identified into a semi-supervised road surface defect identification model and outputs the road surface defect identification result. The semi-supervised pavement defect identification model is constructed through the following steps: S1. Obtain road surface distress images, including labeled road surface distress images and unlabeled road surface distress images; perform pixel-level category labeling on the labeled road surface distress images to obtain labeled images; Weak enhancement processing and strong enhancement processing are performed on the unlabeled pavement defect images respectively to obtain weakly enhanced unlabeled pavement defect images and strongly enhanced unlabeled pavement defect images; S2. A convolutional feature extraction network is used to extract features from the labeled pavement defect image, the weakly enhanced unlabeled pavement defect image, and the strongly enhanced unlabeled pavement defect image, respectively, to obtain labeled intermediate features, unlabeled weakly enhanced intermediate features, and unlabeled strongly enhanced intermediate features. S3. A multi-feature fusion matching mechanism is used to perform cross-view feature matching on the labeled intermediate features and unlabeled weakly enhanced intermediate features, labeled intermediate features and unlabeled strongly enhanced intermediate features, and unlabeled weakly enhanced intermediate features and unlabeled strongly enhanced intermediate features. Output feature matching constraint information; The multi-feature fusion matching mechanism includes: The labeled intermediate features of historical batches are subjected to orthogonal selection strategy and same semantic category aggregation processing to obtain pixel queue and prototype queue, and a memory bank is constructed using the pixel queue and prototype queue; the memory bank is updated progressively according to the first-in-first-out strategy. Preprocess the unlabeled weakly enhanced intermediate features, the unlabeled strongly enhanced intermediate features, the pixel queue cached in the memory, and the prototype queue in the current training batch. Calculate the first cross-image pairwise feature similarity matrix between unlabeled weakly enhanced intermediate features in the current training batch after preprocessing, and calculate the second cross-image pairwise feature similarity matrix between unlabeled strongly enhanced intermediate features in the current training batch after preprocessing. By minimizing the probability distribution difference metric between the first and second cross-image pairwise feature similarity matrices, the consistency of cross-image pixel-level feature association under different enhanced views is constrained. Calculate the first pixel-to-pixel feature similarity matrix between the preprocessed unlabeled weakly enhanced intermediate features of the current training batch and the preprocessed pixel queue, and calculate the second pixel-to-pixel feature similarity matrix between the preprocessed unlabeled strongly enhanced intermediate features of the current training batch and the preprocessed pixel queue. By minimizing the probability distribution difference metric between the first pixel-to-pixel feature similarity matrix and the second pixel-to-pixel feature similarity matrix, constrain the consistency of cross-image pixel-level feature associations between the current batch and historical batches. Calculate the similarity matrix between the preprocessed unlabeled weakly enhanced intermediate features of the current training batch and the first pixel-prototype feature similarity matrix of the preprocessed prototype queue, and the similarity matrix between the preprocessed unlabeled strongly enhanced intermediate features of the current training batch and the second pixel-prototype feature similarity matrix of the preprocessed prototype queue. By minimizing the probability distribution difference metric between the first pixel-prototype feature similarity matrix and the second pixel-prototype feature similarity matrix, constrain the pixel features of the current batch to move closer to the pixel-level category prototype feature center of the corresponding disease category. A deep convolutional decoding model is used to decode the labeled intermediate features, the unlabeled weakly enhanced intermediate features, and the unlabeled strongly enhanced intermediate features respectively, and outputs a pavement distress prediction map. S4. Calculate the feature matching loss based on the feature matching constraint information, and calculate the supervised loss and unsupervised loss based on the pavement defect prediction map; fuse the feature matching loss, the supervised loss, and the unsupervised loss according to preset weights to obtain the total loss value; by minimizing the total loss value, backpropagate to update the parameters of the convolutional feature extraction network, the multi-feature fusion matching mechanism, and the deep convolutional decoding model, and iterate training until the preset convergence condition is met to obtain the semi-supervised pavement defect recognition model.

2. The semi-supervised pavement defect identification method based on multi-feature matching according to claim 1, characterized in that, The weak enhancement processing includes at least one of size adjustment, random cropping, horizontal flipping, and vertical flipping; the strong enhancement processing includes at least one of random color dithering, grayscale conversion, and Gaussian blur.

3. The semi-supervised pavement defect identification method based on multi-feature matching according to claim 1, characterized in that, The convolutional feature extraction network is a deep convolutional neural network, which includes at least one of the ResNet series, DenseNet series, MobileNet series, and EfficientNet series.

4. The semi-supervised pavement defect identification method based on multi-feature matching according to claim 1, characterized in that, The preprocessing includes L2 normalization and reshaping processes performed sequentially, wherein the reshaping process is used to convert two-dimensional intermediate features into one-dimensional feature vectors. The probability distribution difference metric is at least one of Kullback-Leibler divergence, Jensen-Shannon divergence, and cosine distance.

5. The semi-supervised pavement distress identification method based on multi-feature matching according to claim 1, characterized in that, The deep convolutional decoding model is a semantic segmentation decoder, which includes at least one of the DeepLab series, U-Net series, SegNet series, and PSPNet series.

6. The semi-supervised pavement defect identification method based on multi-feature matching according to claim 1, characterized in that, The supervised loss is the pixel-level cross-entropy loss between the pavement defect prediction map corresponding to the labeled pavement defect image and the labeled image; the unsupervised loss is the consistency loss between the pavement defect prediction map corresponding to the weakly enhanced unlabeled pavement defect image and the pavement defect prediction map corresponding to the strongly enhanced unlabeled pavement defect image; the feature matching loss is the Kullback-Leibler divergence loss between the cross-view feature similarity matrices.

7. The semi-supervised pavement defect identification method based on multi-feature matching according to claim 1, characterized in that, The preset convergence condition is that the number of iterations reaches a preset threshold, or the rate of change of the total loss value in consecutive preset rounds is less than or equal to the rate of change threshold.

8. The semi-supervised pavement distress identification method based on multi-feature matching according to claim 1, characterized in that, The pavement defect identification results include pavement defect category information and pixel-level distribution information.

9. A semi-supervised pavement defect identification system using multi-feature matching, characterized in that, When the system is running, it executes a semi-supervised pavement defect identification method with multi-feature matching as described in any one of claims 1 to 8.

Images