A cross-domain road defect detection method based on multi-kernel distribution alignment and category-adaptive topology perception

Abstract

This invention provides a cross-domain road defect detection method based on multi-kernel distribution alignment and category-adaptive topology awareness, belonging to the field of road defect detection technology. This method constructs a spatially perceptual mean teacher detection architecture based on uncertainty-weighted algorithms, achieving deep domain adaptation through a three-level strategy: at the feature level, multi-kernel maximum mean difference (MK-MMD) is used to narrow the statistical distance between the source and target domains at multiple feature levels; at the spatial level, an uncertainty-weighted spatial attention consistency mechanism is introduced, using prediction entropy weights to suppress noise interference and correct attention drift in the student network; at the output level, the physical geometric topological attributes of the defects are combined to perform category-adaptive filtering and collinear adjacency fusion on the prediction results, generating high-quality pseudo-labels to guide training. This invention effectively overcomes the problems of feature misalignment and target breakage missed detection in visually degraded scenarios, significantly improving the model's generalization ability and detection accuracy in cross-domain scenarios.

Description

Technical Field

[0001] This invention relates to the field of road defect detection technology, and in particular to a cross-domain road defect detection method based on multi-kernel distribution alignment and category adaptive topology perception. Background Technology

[0002] With the increasing volume of highway traffic, the frequent occurrence of road defects seriously affects road service levels and driving safety. Traditional manual inspection methods are inefficient, costly, and highly subjective. Deep learning-based object detection algorithms, such as YOLO and Faster R-CNN, have shown great potential in road defect detection.

[0003] However, the performance of these models heavily relies on large-scale, high-quality labeled data and typically assumes that the training and test data follow the same distribution. In practical applications, weather changes (such as fog, rain, and snow) can cause domain shifts in images, resulting in a sharp decline in the performance of models trained in clear weather under hazy conditions. Re-collecting and labeling data for hazy weather is extremely costly and impractical.

[0004] In existing technologies, some domain adaptation methods use single-kernel maximum mean difference for feature alignment, which makes it difficult to simultaneously capture multi-scale target features with huge shape differences in road defects. Although the mean teacher framework can improve the robustness of cross-domain detection, the student network is prone to attention drift to background noise under haze interference. In addition, existing pseudo-label generation mainly relies on confidence thresholds, which are prone to missed detections and false detections under haze conditions. Furthermore, long-distance defect targets are often broken into multiple segments due to occlusion, and there is a lack of effective geometric fusion mechanism, which limits the detection accuracy. Summary of the Invention

[0005] To address the aforementioned issues, this invention proposes a cross-domain road defect detection method based on multi-kernel distribution alignment and category-adaptive topology perception. This method constructs a spatial perception mean teacher detection architecture based on uncertainty weighting, aiming to systematically overcome the cross-domain distribution shift and visual feature degradation caused by severe weather such as smog through a three-level strategy of feature-level alignment, spatial distribution alignment, and output distribution alignment, thereby significantly improving the model's generalization ability and detection accuracy in cross-domain smog scenarios.

[0006] Specifically, this invention first measures and narrows the statistical differences between the source and target domains at multiple scale feature levels using multi-kernel maximum mean difference (MK-MMD), achieving feature-level alignment and thus solving the cross-domain adaptation problem of basic features. Second, it utilizes the spatial attention consistency mechanism of uncertain weights, using the attention distribution of the teacher network as a soft constraint to correct the attention bias of the student network in visual degradation scenarios, achieving spatial distribution alignment and ensuring the consistency of the model's attention area. Finally, it combines the physical geometric topological attributes of road defects to perform category adaptive filtering and collinear adjacency fusion on the prediction results generated by the teacher network, using the generated optimized pseudo-labels to guide the training of the student network, prompting the student network's prediction distribution in the target domain to move closer to the high-quality distribution in the source domain, thus achieving output distribution alignment.

[0007] The objective of this invention is achieved through the following technical solution: a cross-domain road defect detection method based on multi-kernel distribution alignment and category-adaptive topology perception, comprising the following steps:

[0008] Step S1, Data Acquisition and Preprocessing: Acquire road damage images under normal weather conditions as the source domain dataset. ,in, Image of road defects in the source region. Mark the road defects in the source area. The number of samples in the source domain includes road damage categories such as longitudinal cracks, transverse cracks, alligator cracks, and potholes. Additionally, road damage images taken during hazy weather are collected as the target domain dataset. ,in, For unlabeled target domain road damage images, This indicates the number of samples in the target domain and preprocesses the acquired image data.

[0009] Step S2: Construct a mean teacher framework based on uncertainty-weighted spatial awareness: The framework includes a student network and a teacher network, both of which use YOLOv11 as the base detector. The student network receives the source domain dataset and the target domain dataset, while the teacher network only receives the target domain dataset. Knowledge connections are established between the two networks through exponential moving average (EMA). Spatial attention extraction units are embedded at the end of the backbone network to generate attention heatmaps, and uncertainty-weighted units are embedded at the end of the classification branch of the detector head to calculate the prediction information entropy.

[0010] Step S3: Perform multi-level forward propagation and feature extraction to generate category-adaptive topology-aware pseudo-labels: The teacher network infers on the target domain dataset, uses the category-adaptive topology-aware strategy to filter the initial prediction boxes, and uses the collinear adjacency fusion strategy to reconstruct the broken targets and generate optimized pseudo-labels. The student network processes the source domain and target domain datasets simultaneously, extracts multi-scale feature maps, and generates prediction results.

[0011] Step S4: Calculate the multi-task joint loss function: Calculate the supervised detection loss of the source domain based on the source domain labeling; calculate the distance between the features of the source domain and the target domain in the Hilbert space of the regenerating kernel based on the multi-kernel maximum mean difference (MK-MMD) to obtain the feature-level multi-kernel distribution alignment loss; calculate the predicted information entropy and uncertainty weight matrix of the teacher network using the uncertainty weighted unit, and constrain the consistency of the teacher-student network attention heatmap to obtain the uncertainty-weighted spatial attention consistency loss; calculate the pseudo-label unsupervised loss based on the optimized pseudo-labels; and sum the above losses by weight to construct the total loss function.

[0012] Step S5: Update the student network parameters through backpropagation and the teacher network parameters through exponential moving average until the model converges. Then, use the trained model to detect road defects and obtain the detection results.

[0013] Furthermore, in the mean teacher framework based on uncertainty-weighted spatial awareness in step S2, the teacher network parameters... Based on student network parameters Real-time updates are performed; the EMA update method is as follows:

[0014]

[0015] in, The number of iterations. For smoothing coefficients, Indicates the first Teacher network parameters at the next iteration Indicates the first Teacher network parameters at the next iteration Indicates the first Student network parameters during the next iteration.

[0016] The spatial attention extraction unit in the mean-teacher framework based on uncertainty-weighted spatial awareness is used to compress high-dimensional feature maps into two-dimensional saliency heatmaps. For any input feature tensor, this unit first calculates the sum of absolute values ​​along the channel dimensions to obtain the original activation map. The calculation method is as follows:

[0017]

[0018] in, This represents the high-dimensional feature map output from the end of the feature extraction network in both the student and teacher networks. Indicates the number of channels. Indicates the height of the feature map, Indicates the width of the feature map. , , These represent the channel index, height index, and width index, respectively.

[0019] Furthermore, in step S3, the teacher network performs inference on the target domain dataset, including:

[0020] Teacher network for road damage images in the target domain Perform forward propagation to output multi-scale feature maps of the target domain and target domain prediction results By utilizing embedded spatial attention extraction units, the feature maps are... Aggregation along the channel dimension yields The spatial attention heatmap of teachers was obtained by normalization. :

[0021]

[0022] in, Indicates the coordinate index of the graph. To prevent constants with a denominator of zero.

[0023] Further, step S3, which uses a category-adaptive topology-aware strategy to filter initial prediction boxes, includes:

[0024] Set a high confidence threshold for smog scenarios Low confidence threshold and the aspect ratio threshold for cracks The aspect ratio threshold for pits and depressions ;

[0025] For longitudinal and transverse cracks, if the detection frame confidence is in the hesitation interval... And satisfy the geometric aspect ratio If so, it is determined to be a real crack obscured by smog and is preserved.

[0026] For a detection bounding box categorized as pits and cracks, if its aspect ratio satisfies If it is not, it is considered a misjudgment caused by background noise and is removed.

[0027] Furthermore, in step S3, the fracture target is reconstructed using a collinear adjacency fusion strategy to generate optimized pseudo-labels, including:

[0028] Calculate any two similar detection boxes The fusion score between The calculation method is as follows:

[0029]

[0030] in, The Euclidean distance between the center points of the two frames. To detect the length of the diagonal of the frame, The distance attenuation constant is The angle between the two frames along their main axes. and These are the weighting coefficients for the distance and angle terms, respectively.

[0031] Set fusion threshold If the calculated fusion score Greater than the fusion threshold Then, perform an extreme value union operation on the coordinates of the two detection boxes to generate a new detection box. Coordinates:

[0032]

[0033] in, They represent the first in the same category. The and the first The x-coordinate of the left boundary of each detection box. They represent the first in the same category. The and the first The ordinate of the upper boundary of each detection box. They represent the first in the same category. The and the first The x-coordinate of the right boundary of each detection box. They represent the first in the same category. The and the first The ordinate of the lower boundary of each detection box;

[0034] New detection frame The category of the new detection box inherits from the category of the original detection box, and the confidence score of the new detection box is taken as the maximum value of the confidence score of the original detection box, thus generating the optimized pseudo-label. Provided for students to learn online.

[0035] Furthermore, in step S3, the student network simultaneously processes the source domain and target domain datasets, including:

[0036] The student network extracts multi-scale feature maps of the source domain from the source domain dataset and the target domain dataset, respectively. and target domain multi-scale feature map The obtained target domain multi-scale feature map The original activation map of the target domain is obtained according to the original activation map calculation method in step S2. ,right The same normalization process as the teacher network was applied to obtain the student spatial attention heatmap. Meanwhile, the detection head in the student network utilizes multi-scale feature maps of the source domain. and target domain multi-scale feature map Further, the source domain prediction results were obtained separately. and target domain prediction results .

[0037] Furthermore, in step S4, the source domain has supervised detection loss. The calculation method is as follows:

[0038] Prediction results of source domain images based on student network With real labeling Calculate the classification loss Distribution focal loss and border loss The YOLOv11 native loss is used to obtain the source domain supervised detection loss. :

[0039] .

[0040] Furthermore, feature-level multi-kernel distribution alignment loss The calculation method is as follows:

[0041] Calculate the source domain multi-scale features extracted by the student network Multi-scale features of the target domain In the regenerating nucleus Hilbert space The squared distance in the equation is used as the loss function:

[0042]

[0043] in, This represents a nonlinear feature mapping function, where the inner product after mapping is defined as a multi-kernel function using the kernel trick. ,Right now , Let be any sample feature vector in the source domain feature distribution. For any one of the feature distributions in the target domain, ... Same-dimensional sample feature vectors ,in, Indicates the number of cores. Indicates bandwidth. No. The weighting coefficients are a linear combination of Gaussian kernel functions.

[0044] Furthermore, uncertainty-weighted spatial attention consistency loss The calculation method is as follows:

[0045] Calculating the teacher network using uncertainty-weighted units to calculate the teacher network in spatial location Predictive information entropy at the location And based on this, a pixel-level uncertainty weight matrix is ​​generated. The calculation method is as follows:

[0046]

[0047]

[0048] in, The number of categories of the detected target. Predicting the location of teachers on the network Belongs to the The probability of each category, Hyperparameters for controlling uncertainty sensitivity;

[0049] Finally, attentional consistency loss for:

[0050]

[0051] in, and These represent heatmaps of student spatial attention and teacher spatial attention, respectively. These represent the indices in the diagram. This represents the spatial dimensions of the feature map.

[0052] Furthermore, the unsupervised loss for pseudo-labels is calculated based on the optimized pseudo-labels, including:

[0053] Prediction results of road damage images in the target domain based on student networks Optimized pseudo-tags generated Using YOLOv11's native loss function, calculate the pseudo-label unsupervised loss. :

[0054] ;

[0055] Total loss function for:

[0056]

[0057] in, , , These represent the balance weight coefficients of the feature-level multi-kernel distribution alignment loss, attention consistency loss, and pseudo-label unsupervised loss, respectively.

[0058] Compared with the prior art, the present invention has the following beneficial technical effects:

[0059] To address the problem that existing single-kernel alignment methods struggle to account for the differences in crack and pothole distributions, this invention employs a multi-kernel maximum mean difference (MK-MMD) strategy. By combining Gaussian kernel functions with multi-scale bandwidth, the statistical distribution distance between the source and target domains is narrowed within the Hilbert space of the regenerating kernel, thereby achieving refined feature-level alignment at the global statistical level. This strategy effectively solves the problem of insufficient alignment at a single scale and significantly improves the model's ability to capture road disease features of different sizes and shapes, as well as its feature robustness.

[0060] To address the issues of reduced image contrast caused by smog and the tendency for student network attention to drift towards background noise, this invention introduces an uncertainty-weighted spatial attention consistency mechanism. By forcibly aligning the attention heatmaps of teacher and student networks, it constrains the student network to focus on areas with significant defects, achieving spatial alignment of key areas. At the same time, it uses prediction entropy weights to quantify the prediction uncertainty of the teacher network, avoiding guidance from low-quality supervision signals and improving the accuracy of feature extraction in visual degradation scenarios.

[0061] To address the issues of low-confidence targets being easily missed, background noise causing false detections, and long-distance linear lesions being prone to breakage under haze interference, this invention optimizes pseudo-labels based on the physical and geometric attributes of lesions. By employing strategies of linear target topological recall and compact target geometric suppression, it alleviates the limitations of traditional methods that rely solely on confidence thresholds. Furthermore, in conjunction with a collinear adjacency fusion strategy, it reconstructs broken long-distance lesion targets, achieving high-quality output-level alignment at the semantic prediction level and enabling the model to achieve better generalization performance in the target domain. Attached Figure Description

[0062] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0063] Figure 1 This is a flowchart illustrating a cross-domain road defect detection method based on multi-kernel distribution alignment and category adaptive topology perception provided in an embodiment of the present invention.

[0064] Figure 2 This is a schematic diagram of the confusion matrix of the identification results of various road defects by the model of the present invention provided in the embodiments of the present invention;

[0065] Figure 3 This is the PR curve of the model of the present invention provided in the embodiments of the present invention;

[0066] Figure 4This is a comparison chart of the detection results of the model of the present invention and the real labels provided in the embodiments of the present invention. Detailed Implementation

[0067] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0068] This invention provides a cross-domain road defect detection method based on multi-kernel distribution alignment and category-adaptive topology awareness, such as... Figure 1 As shown, firstly, the collected road damage data under clear weather (source domain) and hazy weather (target domain) are preprocessed; secondly, semi-supervised joint training is completed using supervised data from the source domain and unsupervised data from the target domain. In the feature extraction stage, MK-MMD loss is used to narrow down the multi-scale feature distributions of the source and target domains; in the feature interaction stage, an uncertainty weighting mechanism is used to calculate the spatial attention difference between the teacher network and the student network to correct the attention drift of the student network; in the label generation stage, the teacher network dynamically filters low-confidence cracks and merges fracture targets based on the aspect ratio and topology of the detection boxes to generate high-quality pseudo-labels to supervise the student network; finally, the parameters of the student and teacher networks are updated through backpropagation and the EMA strategy until the model converges.

[0069] The method of the present invention specifically includes the following steps:

[0070] Step 1, Data Acquisition and Multi-Domain Dataset Construction: Road damage images collected by cameras under normal weather conditions are used as the source domain dataset. ,in, Image of road defects in the source region. Mark the road defects in the source area. This represents the number of samples in the source domain. The types of road defects include longitudinal cracks, transverse cracks, alligator cracks, and potholes. Additionally, road defect images taken during hazy weather are used as the target domain dataset. ,in, For unlabeled target domain road damage images, This represents the number of samples in the target domain.

[0071] Step 2, Data Preprocessing and Network Input Preparation: This involves processing the source domain image obtained in Step S1. With target domain image Preprocessing is performed to scale all image sizes to... The pixels are processed using a maximum-minimum normalization method, i.e. ,in Represents the original pixel values ​​in the image. This represents the minimum pixel value. This represents the maximum pixel value. This represents the normalized pixel value. The processed image data is organized into batches and used as input for subsequent networks.

[0072] Step 3: Construct a mean-based teacher framework based on uncertainty-weighted spatial awareness. This framework comprises a student network and a teacher network, both using the YOLOv11 architecture as the basic detector. Knowledge connections are established between the two networks through exponential moving average (EMA). The teacher network parameters... It does not participate in gradient backward updates, but is based on student network parameters. Real-time updates are performed using the following formula:

[0073]

[0074] in, The number of iterations. For smoothing coefficients, Indicates the first Teacher network parameters at the next iteration Indicates the first Teacher network parameters at the next iteration Indicates the first In the next iteration, the student network parameters are initially obtained using random initialization.

[0075] Building upon this, spatial attention extraction units are embedded at the ends of the backbone networks of the student and teacher networks. These units are configured to perform channel statistical aggregation operations to compress high-dimensional feature maps into two-dimensional saliency heatmaps. For any input feature tensor... This unit first calculates the sum of absolute values ​​along the channel dimension to obtain the original activation map. The calculation formula is:

[0076]

[0077] in, This represents the high-dimensional feature map output from the end of the feature extraction network in the student and teacher networks, with the shape of... , Indicates the number of channels. Indicates the height of the feature map, Indicates the width of the feature map. , , These represent the channel index, height index, and width index, respectively.

[0078] Step 4, Perform multi-level forward propagation and feature extraction: Input the processed image data into the mean teacher framework based on uncertainty-weighted spatial awareness in batches, and perform the following step-by-step operations:

[0079] Step 41, Teacher Network Forward Propagation and Heatmap Generation: The teacher network generates heatmaps of the target domain road defect images. Perform forward propagation to output multi-scale feature maps of the target domain and target domain prediction results By utilizing embedded spatial attention extraction units, the feature maps are... Aggregation along the channel dimension yields The spatial attention heatmap of teachers was obtained by normalization. :

[0080]

[0081] in, Indicates the coordinate index of the graph. To prevent tiny constants with a denominator of zero.

[0082] Step 42, generating category-adaptive topology-aware pseudo-labels: Based on the initial detection box set (the initial detection box set refers to the set of preliminary detection results output by the teacher network after processing the input target domain road defect image, containing location, category, and confidence information), the initial prediction results of the teacher network are filtered and reconstructed using the physical geometric features and spatial topological relationships of road defects. This is specifically divided into the following three sub-stages:

[0083] (1) Adaptive category selection based on topology awareness. A high confidence threshold is set for haze scenarios. Low confidence threshold and the aspect ratio threshold for cracks The aspect ratio threshold for pits and depressions Based on the physical morphological differences of disease types, an asymmetric screening strategy is implemented: for longitudinal and transverse cracks, considering that haze causes texture feature degradation and thus reduces model confidence, their significant elongated topological features are utilized for recall. That is, the detection box confidence is within the hesitation interval. And satisfy the geometric aspect ratio If a crack is found to be a genuine crack obscured by smog, it will be forcibly preserved. For potholes and cracks, their physical form should be compact and blocky. If the detection frame category is this type, but the aspect ratio meets the requirements... If the pattern appears as an abnormally long strip, it is considered a misjudgment caused by background noise such as lane lines or roadside shadows, and is therefore discarded.

[0084] (2) Collinear Adjacency-Based Fracture Target Fusion. To address the issue of long-distance linear defects fracturing into multiple fragments due to occlusion, a collinear adjacency fusion strategy is introduced for the retained similar detection boxes after screening. This involves calculating the fusion of any two similar fracture detection boxes. The fusion score between :

[0085]

[0086] in, The Euclidean distance between the center points of the two frames; To detect the length of the diagonal of the frame; The distance attenuation constant; The angle between the two frames along their main axes; and These are the weighting coefficients for the distance and angle terms, respectively.

[0087] (3) Pseudo-tag attribute reconstruction and generation. Set the fusion threshold. If the calculated fusion score Greater than the fusion threshold Then it is determined to be and Fusion is required, and a new detection box is generated using an extreme value union strategy. Coordinates:

[0088]

[0089] in, They represent the first in the same category. The and the first The x-coordinate of the left boundary of each detection box. They represent the first in the same category. The and the first The ordinate of the upper boundary of each detection box. They represent the first in the same category. The and the first The x-coordinate of the right boundary of each detection box. They represent the first in the same category. The and the first The ordinate of the lower boundary of each detection box. This indicates the coordinate information of the processed detection box.

[0090] New detection frame The category inherits the category of the original detection box, that is... , , These represent the detection boxes. , The category. To preserve the confidence information of local salient features and prevent the confidence from decreasing due to averaging after fusion, the confidence of the new detection box is... Take the maximum confidence score of the original detection box, that is... , , These represent the detection boxes. , The confidence level is used to generate optimized pseudo-labels. Provided for students to learn online.

[0091] Step 43, Student Network Forward Propagation and Heatmap Generation: The student network simultaneously processes source domain road defect images. and target domain road disease images Extracting multi-scale feature maps from the source domain and target domain multi-scale feature map Used for subsequent MK-MMD alignment; the target domain multi-scale feature map obtained from the student network. The original activation map of the target domain is obtained according to the original activation map calculation formula in step 3. .right Perform the same normalization process as in step 41, i.e. This allows us to obtain a heatmap of students' spatial attention. Meanwhile, the detection head in the student network utilizes multi-scale feature maps of the source domain. and target domain multi-scale feature map Further, the source domain prediction results were obtained separately. and target domain prediction results .

[0092] Step 5: Calculate the joint loss function for multiple tasks. Calculate the total loss function of the method of this invention. Specifically, it includes the following four steps:

[0093] Step 51, calculate the supervised detection loss in the source domain. Student network-based source domain images Prediction results With real labeling Calculate the classification loss Distribution focal loss and border loss YOLOv11 native supervised loss:

[0094]

[0095] Step 52, calculate the feature-level multi-kernel distribution alignment loss. : Calculate the source domain multi-scale features extracted by the student network Multi-scale features of the target domain The squared distance in the reproducing kernel Hilbert space (RKHS) is used as the loss function:

[0096]

[0097] in, This represents a nonlinear feature mapping function, which maps the original feature data to a high-dimensional or even infinite-dimensional reproducing kernel Hilbert space. .

[0098] Through this mapping, the complex distributional differences in the original space, which are difficult to measure, are transformed into the Euclidean distance between the means of two sample sets in RKHS. Because The dimension is too high to be directly calculated. This invention utilizes kernel tricks to calculate it using multiple Gaussian functions with different bandwidths. The distribution distance is represented by directly calculating the inner product, i.e. ,in, Let be any sample feature vector in the source domain feature distribution. For any one of the feature distributions in the target domain, ... Sample feature vectors of the same dimension.

[0099]

[0100] in, Indicates the number of cores; Indicates bandwidth. No. The linear combination of Gaussian kernel functions and weighting coefficients; to cover feature distributions at multiple scales and ensure balanced alignment of feature distributions at multiple scales, this embodiment preferably uses... , .

[0101] Simultaneously, to cover distribution differences at different scales, bandwidth This is based on the median heuristic. First, the median of the Euclidean distance between the feature vectors of all samples in the current batch is calculated. Then, construct a multi-scale bandwidth set:

[0102]

[0103] Step 53: Calculate the uncertainty-weighted spatial attention consistency loss. Using the teacher spatial attention heatmap generated in step 4 Student attention heatmap and the target domain prediction results of the teacher network Perform the following calculations: Calculate the prediction uncertainty of the teacher network on the target domain image using the uncertainty-weighted unit, for any spatial location on the feature map. Assuming the target to be detected contains The teacher network predicts that the location belongs to the [number] category. The probability of each category is Then the information entropy at that position The calculation formula is as follows:

[0104]

[0105] Subsequently, the pixel-level uncertainty weight matrix is ​​generated using the information entropy values ​​obtained from the above calculations. To suppress gradient backpropagation in regions of high uncertainty, the calculation formula is as follows:

[0106]

[0107] in, The hyperparameters used to control sensitivity to uncertainty; the higher the teacher network prediction entropy, i.e., the greater the uncertainty, the higher the weight of the corresponding position. The closer it gets to 0.

[0108] Finally, combining the weight matrix Using the attention heatmap, the weighted mean squared error is calculated as the consistency loss. The calculation formula is as follows:

[0109]

[0110] in, and These represent heatmaps of student spatial attention and teacher spatial attention, respectively. These represent the indices in the diagram.

[0111] Step 54: Calculate the unsupervised loss of pseudo-labels based on the optimized pseudo-labels. Prediction results of road damage images in the target domain based on student networks The optimized pseudo-tags generated in step 42 Using the native loss function formula of YOLOv11, calculate the pseudo-label unsupervised loss:

[0112] .

[0113] Step 6: Update network parameters via backpropagation: Construct the total loss function based on weighted summation. :

[0114]

[0115] in, , , These represent the balanced weight coefficients for the feature-level multi-kernel distribution alignment loss, attention consistency loss, and pseudo-label unsupervised loss, respectively. The network parameters are updated using the total loss function, including:

[0116] Step 61, Update student network parameters: Calculate the total loss for the student network parameters using backpropagation. The gradient is used to update the student network parameters. ;

[0117] Step 62, Update teacher network parameters: Update using the EMA formula described in step 3. .

[0118] Step 7, Model Inference and Road Defect Detection: Repeat steps 4 to 6 until the model converges. After training, fix the student network parameters as the final road defect detection model for hazy weather. Input the hazy image to be detected into the student network, and after non-maximum suppression (NMS), the model can output the category, bounding box, and confidence score of all defects in the image.

[0119] Figure 2 The confusion matrix results of the model of this invention on the road defect test set are shown. The vertical axis of the matrix represents the predicted category of the model, and the horizontal axis represents the true labeled category of the sample. The values ​​in the matrix represent the prediction percentage of the corresponding category. The results show that the model's classification accuracy for five types of road defects, namely longitudinal cracks, transverse cracks, alligator cracks, other defects, and potholes, reached 82%, 72%, 64%, 88%, and 69%, respectively. At the same time, the matrix also clearly shows that the false detection and false negative of various defects are at a low level, which verifies that the model has a stable basic classification and recognition ability for road defect targets.

[0120] Figure 3 The PR curves for various road defects represented by the model in this invention are used to quantify the model's comprehensive performance in target classification and localization under different confidence thresholds. The horizontal axis represents recall, and the vertical axis represents precision. The area under a single curve represents the average precision (AP) for the corresponding category, and the mean AP for all categories is the model's mAP@0.5. The results show that, with an IoU (Intersection over Union) threshold of 0.5, the model achieves an average precision (mAP@0.5) of 72.7% across all categories. Among these, the detection precision for other defects and longitudinal cracks is the best, with APs reaching 87.3% and 75.6%, respectively. The overall curve trend verifies that the model can maintain a high detection precision while improving target recall, demonstrating good comprehensive detection performance.

[0121] Figure 4The image shows the intuitive detection results of the trained model on road damage images under hazy conditions. The left side of the image shows the real annotation information of road damage under hazy weather, and the right side shows the model's output detection results and corresponding confidence scores. The results intuitively show that even in complex scenarios with image degradation and reduced contrast caused by haze, the model can still accurately identify and locate various types of road damage, such as longitudinal cracks, transverse cracks, and other defects. The detection results highly match the target locations and categories of the real annotations, and the output target confidence scores are all within a reasonable range. This fully verifies the model's generalization ability and practical engineering application value in adverse weather scenarios.

[0122] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A cross-domain road defect detection method based on multi-kernel distribution alignment and category-adaptive topology perception, characterized in that, The method includes: Step S1, Data Acquisition and Preprocessing: Acquire road damage images under normal weather conditions as the source domain dataset. ,in, Image of road defects in the source region. Mark the road defects in the source area. The number of samples in the source domain includes road damage categories such as longitudinal cracks, transverse cracks, alligator cracks, and potholes. Additionally, road damage images taken during hazy weather are collected as the target domain dataset. ,in, For unlabeled target domain road damage images, This indicates the number of samples in the target domain and preprocesses the acquired image data. Step S2: Construct a mean teacher framework based on uncertainty-weighted spatial awareness: The framework includes a student network and a teacher network, both of which use YOLOv11 as the base detector. The student network receives source domain datasets and target domain datasets, while the teacher network only receives the target domain dataset. Knowledge connections are established between the two networks through exponential moving average (EMA). Spatial attention extraction units are embedded at the end of the backbone network to generate attention heatmaps, and uncertainty-weighted units are embedded at the end of the classification branch of the detector head to calculate the prediction information entropy. Step S3: Perform multi-level forward propagation and feature extraction to generate category-adaptive topology-aware pseudo-labels: The teacher network infers on the target domain dataset, uses the category-adaptive topology-aware strategy to filter the initial prediction boxes, and uses the collinear adjacency fusion strategy to reconstruct the broken targets and generate optimized pseudo-labels. The student network processes the source domain and target domain datasets simultaneously, extracts multi-scale feature maps, and generates prediction results. Step S4: Calculate the multi-task joint loss function: Calculate the supervised detection loss of the source domain based on the source domain labeling; calculate the distance between the features of the source domain and the target domain in the Hilbert space of the regenerating kernel based on the multi-kernel maximum mean difference (MK-MMD) to obtain the feature-level multi-kernel distribution alignment loss; calculate the predicted information entropy and uncertainty weight matrix of the teacher network using the uncertainty weighted unit, and constrain the consistency of the teacher-student network attention heatmap to obtain the uncertainty-weighted spatial attention consistency loss; calculate the pseudo-label unsupervised loss based on the optimized pseudo-labels; and sum the above losses by weight to construct the total loss function. Step S5: Update the student network parameters through backpropagation and the teacher network parameters through exponential moving average until the model converges. Then, use the trained model to detect road defects and obtain the detection results. In step S3, the teacher network performs inference on the target domain dataset, including: Teacher network for road damage images in the target domain Perform forward propagation to output multi-scale feature maps of the target domain and target domain prediction results By utilizing embedded spatial attention extraction units, the feature maps are... Aggregation along the channel dimension yields The spatial attention heatmap of teachers was obtained by normalization. : in, Indicates the coordinate index of the graph. To prevent constants with a denominator of zero; The initial prediction box selection using the category-adaptive topology-aware strategy includes: Set a high confidence threshold for smog scenarios Low confidence threshold and the aspect ratio threshold for cracks The aspect ratio threshold for pits and depressions ; For longitudinal and transverse cracks, if the detection frame confidence is in the hesitation interval... And satisfy the geometric aspect ratio If so, it is determined to be a real crack obscured by smog and is preserved. For a detection bounding box categorized as pits and cracks, if its aspect ratio satisfies If it is not, it will be considered a misjudgment caused by background noise and will be removed. The process of reconstructing the fractured target using a collinear adjacency fusion strategy and generating optimized pseudo-labels includes: Calculate any two similar detection boxes The fusion score between The calculation method is as follows: in, The Euclidean distance between the center points of the two frames. To detect the length of the diagonal of the frame, The distance attenuation constant is The angle between the two frames along their main axes. and These are the weighting coefficients for the distance and angle terms, respectively. Set fusion threshold If the calculated fusion score Greater than the fusion threshold Then, perform an extreme value union operation on the coordinates of the two detection boxes to generate a new detection box. Coordinates: in, They represent the first in the same category. The and the first The x-coordinate of the left boundary of each detection box. They represent the first in the same category. The and the first The ordinate of the upper boundary of each detection box. They represent the first in the same category. The and the first The x-coordinate of the right boundary of each detection box. They represent the first in the same category. The and the first The ordinate of the lower boundary of each detection box; New detection frame The category of the new detection box inherits from the category of the original detection box, and the confidence score of the new detection box is taken as the maximum value of the confidence score of the original detection box, thus generating the optimized pseudo-label. Provided for students to learn online.

2. The method according to claim 1, characterized in that, In the mean teacher framework based on uncertainty-weighted spatial awareness in step S2, the teacher network parameters Based on student network parameters Real-time updates are performed; the EMA update method is as follows: in, The number of iterations. For smoothing coefficients, Indicates the first Teacher network parameters at the next iteration Indicates the first Teacher network parameters at the next iteration Indicates the first Student network parameters at the next iteration; The spatial attention extraction unit in the mean-teacher framework based on uncertainty-weighted spatial awareness is used to compress high-dimensional feature maps into two-dimensional saliency heatmaps. For any input feature tensor, this unit first calculates the sum of absolute values ​​along the channel dimensions to obtain the original activation map. The calculation method is as follows: in, This represents the high-dimensional feature map output from the end of the feature extraction network in both the student and teacher networks. Indicates the number of channels. Indicates the height of the feature map, Indicates the width of the feature map. , , These represent the channel index, height index, and width index, respectively.

3. The method according to claim 2, characterized in that, In step S3, the student network simultaneously processes the source and target domain datasets, including: The student network extracts multi-scale feature maps of the source domain from the source domain dataset and the target domain dataset, respectively. and target domain multi-scale feature map The obtained target domain multi-scale feature map The original activation map of the target domain is obtained according to the original activation map calculation method in step S2. ,right The same normalization process as the teacher network was applied to obtain the student spatial attention heatmap. Meanwhile, the detection head in the student network utilizes multi-scale feature maps of the source domain. and target domain multi-scale feature map Further, the source domain prediction results were obtained separately. and target domain prediction results .

4. The method according to claim 3, characterized in that, In step S4, the source domain has supervised detection loss. The calculation method is as follows: Prediction results of source domain images based on student network With real labeling Calculate the classification loss Distribution focal loss and border loss The YOLOv11 native loss is used to obtain the source domain supervised detection loss. : 。 5. The method according to claim 4, characterized in that, Feature-level multi-kernel distribution alignment loss The calculation method is as follows: Calculate the source domain multi-scale features extracted by the student network Multi-scale features of the target domain In the regenerating nucleus Hilbert space The squared distance in the equation is used as the loss function: in, This represents a nonlinear feature mapping function, where the inner product after mapping is defined as a multi-kernel function using the kernel trick. ,Right now , Let be any sample feature vector in the source domain feature distribution. For any one of the feature distributions in the target domain, ... Same-dimensional sample feature vectors ,in, Indicates the number of cores. Indicates bandwidth. No. The weighting coefficients are a linear combination of Gaussian kernel functions.

6. The method according to claim 5, characterized in that, Uncertainty-weighted spatial attention consistency loss The calculation method is as follows: Calculating the teacher network using uncertainty-weighted units to calculate the teacher network in spatial location Predictive information entropy at the location And based on this, a pixel-level uncertainty weight matrix is ​​generated. The calculation method is as follows: in, The number of categories of the detected target. Predicting the location of teachers on the network Belongs to the The probability of each category, Hyperparameters for controlling uncertainty sensitivity; Finally, attentional consistency loss for: in, and These represent heatmaps of student spatial attention and teacher spatial attention, respectively. These represent the indices in the spatial attention heatmap. This represents the spatial dimensions of the feature map.

7. The method according to claim 6, characterized in that, The unsupervised loss for pseudo-labels is calculated based on the optimized pseudo-labels, including: Prediction results of road damage images in the target domain based on student networks Optimized pseudo-tags generated Using YOLOv11's native loss function, calculate the pseudo-label unsupervised loss. : ； Finally, the total loss function for: in, , , These represent the balance weight coefficients of the feature-level multi-kernel distribution alignment loss, attention consistency loss, and pseudo-label unsupervised loss, respectively.

Images