Road crack remote sensing image semantic segmentation active learning method based on self-supervised dual representation mechanism

Abstract

This invention discloses an active learning method for semantic segmentation of remote sensing images of road cracks based on a self-supervised dual representation mechanism, belonging to the field of computer vision and intelligent interpretation technology of remote sensing images. The method includes: extracting global semantic distribution features of unlabeled sample pool images using a first self-supervised model with frozen parameters; calculating pixel-level reconstruction error maps using a second self-supervised model with frozen parameters; generating a comprehensive sampling value score by weighted fusion of the two types of self-supervised features, with a cold-start hybrid sampling strategy adopted in the first iteration and dynamic adjustment of fusion weights in subsequent iterations; selecting a high-value sample set from the unlabeled sample pool and performing pixel-level annotation; and independently training the semantic segmentation main network using the labeled sample set, with the self-supervised model not participating in gradient updates throughout the process. This invention achieves complete decoupling between the sampling module and the segmentation model, significantly improving crack edge segmentation accuracy and model convergence efficiency with low annotation costs.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and intelligent interpretation technology of remote sensing images, specifically involving an active learning method for semantic segmentation of road crack remote sensing images based on a self-supervised dual representation mechanism. Background Technology

[0002] In the field of transportation infrastructure operation and maintenance, roads are the core carriers of road network operation. Road cracks, as the most common pavement defects, can continue to expand, causing problems such as pavement damage, collapse, and driving safety hazards, directly affecting the service life of roads and traffic safety. In recent years, drones, vehicle-mounted LiDAR, and satellite remote sensing inspections have become mainstream technologies for routine road defect detection due to their advantages of wide coverage, high efficiency, and non-contact operation. Semantic segmentation technology based on deep learning is the core support for achieving automated and refined identification of road cracks in remote sensing / pavement images.

[0003] With the rapid increase in urban and rural road mileage, the amount of road inspection image data is growing exponentially. Road cracks in these images are characterized by their elongated shape, varying width, blurred boundaries, and complex backgrounds (asphalt / cement textures, shadows, oil stains, gravel, etc.), demanding extremely high pixel-level segmentation accuracy. However, pixel-level annotation of road cracks requires professionals to meticulously outline each crack pixel by pixel, a process that is tedious, time-consuming, and labor-intensive. Simply relying on full-scale annotation training of the model is impractical for large-scale road inspection projects. Therefore, combining active learning with efficient sample sampling strategies has become a core research direction for improving the data utilization efficiency of road crack segmentation models and significantly reducing manual annotation costs.

[0004] In existing technologies, active learning methods based on uncertainty, entropy, and marginal sampling improve segmentation efficiency by prioritizing the labeling of high-information samples, which is the mainstream annotation optimization scheme for road crack detection. However, most methods are still limited to a fixed paradigm of model inference-uncertainty calculation-sampling, with sampling and segmentation models highly coupled, making it impossible to achieve independent and unbiased sample selection using self-supervised features. This leads to the model struggling to overcome its own prediction bias during iteration, especially with insufficient coverage of difficult examples such as fine cracks, edge cracks, intersecting cracks, and weakly textured cracks, resulting in a cycle of biased sampling and biased training, making it difficult to improve the segmentation accuracy of edge and fine cracks. Even if sampling quality is improved through more complex uncertainty calculations, it still relies on the model's own output and lacks a systematic and efficient sampling mechanism for decoupling from dual representations.

[0005] Furthermore, most existing research on active learning for road cracks relies on initial labeled sets. For cold-start, unlabeled scenarios, there is still a lack of effective solutions for quickly selecting high-value samples containing typical and difficult-to-identify cracks using only self-supervised features under completely unsupervised conditions. At the same time, existing models have inherent biases in identifying slender cracks, low-contrast cracks, and cracks in complex backgrounds. These biases are difficult to correct at their root cause using conventional sampling strategies alone, thus failing to meet the practical requirements of road maintenance projects for high detection rates and high positioning accuracy. Summary of the Invention

[0006] The purpose of this invention is to provide an active learning method for semantic segmentation of road crack remote sensing images based on a self-supervised dual representation mechanism, which solves the vicious cycle of biased sampling and biased training caused by the strong coupling between sampling strategy and segmentation model in traditional active learning methods.

[0007] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: This invention addresses the core pain points of existing active learning methods, such as strong coupling between sampling and classification models, low sampling efficiency in unlabeled cold-start scenarios, poor optimization effects of boundary region annotation, and excessive annotation costs due to increased sample size. It proposes a semantic segmentation active learning method based on self-supervised dual representations. This method independently uses two types of self-supervised features—DINOv2 global clustering features and MAE boundary reconstruction errors—as the core criteria for selecting high-value samples, thus guiding the entire sampling process. It also designs two core mechanisms: dual representation decoupling and cold-start hybrid sampling, simulating the learning process of global clustering, boundary focusing, and iterative optimization.

[0008] The technical solution to achieve the purpose of this invention is: An active learning method for semantic segmentation of road crack remote sensing images based on a self-supervised dual representation mechanism includes the following steps: Step S1, Data Preparation and System Initialization Phase: Import the remote sensing image dataset of road cracks to be processed, and divide it into unlabeled sample set and labeled sample set according to the labeling status; initialize two sets of self-supervised models for global clustering feature extraction and local boundary error calculation respectively, and initialize the semantic segmentation main network simultaneously; construct the initial unlabeled sample pool and labeled sample set to lay the data foundation for subsequent iterative sampling and model training.

[0009] That is, for a semantic segmentation dataset ,in To input remote sensing images, Provide pixel-level labels for the images. This represents the total number of images in the dataset.

[0010] Let the set of samples to be labeled selected by the model in round t be denoted as . The segmentation model after training is denoted as ; Set the iteration round counter Set the maximum number of iterations. At the same time, initialize the unlabeled sample pool. and labeled sample set The labeled sample set The initial state can be set to an empty set.

[0011] Step S2: Self-supervised dual representation parallel feature extraction: The DINOv2 and MAE models, pre-trained on a large-scale general image dataset with fully locked parameters, are used to perform parallel feature extraction on remote sensing images of road cracks in the unlabeled sample pool. The DINOv2 model encodes the global spatial semantic information of the images, generating a high-dimensional semantic distribution feature vector to characterize the global distribution patterns of different road conditions and crack morphologies. The MAE model performs masked reconstruction of the images to obtain the reconstructed images, and calculates the pixel-level reconstruction error map between the reconstructed images and the original images to accurately locate structural anomalies such as crack edges and minute cracks. The global semantic distribution features and the pixel-level reconstruction error map together constitute the core basis for sample value evaluation, used to screen high-information-content candidate samples.

[0012] Step S3, Hierarchical Iterative Sampling and Model Training Loop: The active learning iterative process involves the following steps in each iteration: Step S3-1: Construct a sampling strategy: Explicitly weight and fuse the global semantic distribution features output by the DINOv2 model with the boundary reconstruction error features output by the MAE model. In the first iteration, a hybrid sampling strategy combining cluster centers and high-error boundary regions is used. In the second to fifth iterations, the fusion weight of the MAE boundary features is gradually increased. After the fifth iteration, an adaptive weighted fusion strategy is used. Step S3-2, High-value sample screening: According to the preset sampling criteria, the unlabeled sample pool is selected. A high-value sample set with high information density and strong discriminative power was selected from the samples. and the set since Removed from the Chinese text; Step S3-3, Labeling and Data Update: For the selected sample set Perform fine-grained pixel-level annotations, and then integrate them into the annotated sample set. ; Step S3-4, Independent Training of Segmentation Model: After each round of sample labeling is completed, independent full training of the segmentation model is triggered, using the updated complete set of labeled samples. For training data, the DeepLabV3+ segmentation main network was trained end-to-end; throughout the training process, the parameters of the DINOv2 and MAE models were kept completely locked and did not participate in any gradient updates or backpropagation to ensure the independence and stability of the sampled features. Step S4, Iteration Termination Condition Determination: Repeat the closed-loop process of self-supervised dual representation pre-extraction, high-value sample screening, pixel-level manual annotation, and independent training of the segmentation model until any of the following termination conditions are met: ① All annotation budget is consumed; ② The average intersection-union ratio (mIoU) of the segmentation model on the validation set improves by less than 0.5% for three consecutive rounds, and the performance tends to converge; ③ The number of iterations reaches the preset maximum limit. After the loop terminates, the segmentation model trained in the last round is output as the final result. .

[0013] Furthermore, in step S1, after each round of active learning iteration, the selected high-value samples are formally included in the labeled sample set after pixel-level annotation, while the remaining samples are still kept in the unlabeled sample pool. The labeled sample set and the unlabeled sample pool are physically isolated during data storage, reading and updating to ensure that the data source for sampling feature extraction and segmentation model training is completely independent, thereby decoupling the sampling and training processes from the bottom layer.

[0014] Furthermore, in step S3-1, when constructing the sampling strategy, the DINOv2 global clustering features and MAE boundary reconstruction error features are incorporated into the sampling process in the form of explicit weighted fusion, serving as the sole core judgment criterion for high-value sample screening, thereby achieving self-supervised dual representation to independently guide the entire sampling process.

[0015] Furthermore, in step S3-2, the two-dimensional sampling criterion followed by sample screening is as follows: after feature calculation, if a sample is located in the distribution center area of ​​different road conditions and different crack types in the DINOv2 semantic feature space, or if there are crack edges, fine cracks, or cross crack areas with pixel-level high errors in the MAE reconstruction error map, the sample is determined to be a high-value sample to be labeled.

[0016] Furthermore, in steps S3-4, the core design logic of the model independent training mechanism is to achieve complete decoupling between the sampling module and the segmentation model, avoid the prediction bias of the segmentation model from affecting the sampling strategy in reverse, break the vicious cycle of biased sampling-biased training from the root, ensure that the model state and labeled knowledge iterate synchronously, and continuously improve the pixel-level semantic classification accuracy.

[0017] Furthermore, the method also includes iterative sampling process control logic, specifically: before reaching the maximum iteration round, continuously execute the self-supervised dual representation extraction, high-value sample screening, manual sample annotation, and independent training process of the segmentation model; if the termination condition is met before reaching the maximum iteration round due to exhaustion of the annotation budget or model performance reaching the preset index, then terminate the process early and output the currently trained model; when the iteration round reaches the preset upper limit, use the segmentation model obtained in the last round of training as the final output result.

[0018] Local structural anomaly score The calculation further includes a crack direction-sensing weighted step: Multi-scale Gabor filtering was applied to the raw remote sensing images in the unlabeled sample pool to extract the local orientation field of the crack region. A directional consistency weight map w(x,y) is constructed based on the local directional field, such that the weight value of pixel (x,y) is related to the difference between the local directional angle φ and the neighborhood mean φ_bar. When the difference approaches 0 or π, the weight approaches 1. When the difference approaches π / 2, the weight approaches the preset basic weight coefficient α. Where α∈(0,1). The directional consistency weight map and the pixel-level reconstruction error map are compared. Pixel-wise weighting is performed to obtain the orientation enhancement reconstruction error map. ; Based on the aforementioned direction-enhanced reconstruction error map Calculate the local structural anomaly score ,in This indicates pixel-by-pixel multiplication.

[0019] Furthermore, the directional consistency weight graph The construction method is as follows:

[0020] in, To achieve the desired pixel-level resolution after multi-scale Gabor filtering of the original remote sensing image. The extracted local principal direction angle; In pixels local neighborhood centered on The arithmetic mean of the local principal orientation angles of each pixel within the area; These are the basic weighting coefficients, used to control the modulation amplitude of weights based on directional consistency. It is a cosine square function; this weight map assigns higher weights to pixels that are consistent with the direction of the local crack to enhance the detection sensitivity of continuous crack edges.

[0021] In another aspect, the present invention also improves an active learning system for semantic segmentation of road crack remote sensing images based on a self-supervised dual representation mechanism, applied to the aforementioned active learning method for semantic segmentation based on self-supervised dual representation, including: The self-supervised feature extraction module is used to drive the pre-trained DINOv2 and MAE models to process the input image during the initialization phase and each iteration, generating semantic distribution representations and pixel-level reconstruction error maps, respectively. These features will serve as the basis for subsequent high-value sample selection. Hierarchical Iterative Sampling Module: As the core of the framework, it is used to fuse DINOv2 features and MAE features in each iteration, and perform clustering sampling and boundary sampling; this module has an embedded sample priority judgment mechanism to determine whether a sample is high-value information, and uses this as the basis for sample selection. Cold start hybrid sampling module: It works in conjunction with the hierarchical iterative sampling module. In the case of a lack of labeled data in the first round, it automatically starts a hybrid sampling strategy consisting of cluster center samples and high-error boundary region samples in a preset ratio to build an initial set of samples to be labeled, and updates the labeled sample library accordingly. Iteration process control module: Used to manage the start, execution and end of the entire active learning loop, including iteration round counting, determining whether the cold start strategy is triggered, and monitoring whether the annotation budget is exhausted or the preset maximum iteration round is reached; Sample set management unit: used to dynamically maintain and update the labeled sample set and the unlabeled sample pool during the iteration process. This information is the basis for the framework to perform self-supervised guided sampling and can output a complete sample allocation record after the learning is completed.

[0022] Compared with the prior art, the present invention has the following beneficial effects: This invention eliminates the bias accumulation problem caused by the reliance on the segmentation model's output for sampling strategies in traditional active learning methods by completely decoupling the sampling module from the segmentation model. The parameters of the first and second self-supervised models are frozen throughout the process and used only for extracting global semantic distribution features and pixel-level reconstruction error maps. Sampling decisions are independent of the segmentation model training process, completely breaking the vicious cycle of biased sampling and biased training. This ensures that the samples selected in each iteration objectively reflect the true information density of the data distribution, significantly improving the accuracy and stability of high-value sample selection.

[0023] This invention utilizes a cold-start hybrid sampling mechanism to efficiently construct a high-quality initial annotation set in the initial stage with completely unlabeled data. The first iteration combines cluster center samples with high-error boundary region samples in a preset ratio, balancing global representativeness with coverage of local structural anomalies. This simulates the cognitive logic of manual annotation—first grasping the overall distribution and then focusing on key points—effectively addressing the technical bottleneck of low sampling efficiency in the cold-start phase of traditional active learning methods, laying a reliable data foundation for subsequent iterative training. By fusing a crack direction-aware weighting strategy, the sampling sensitivity for continuous crack edges and fine cracks is further enhanced. Multi-scale Gabor filtering is used to extract the local orientation field of the image, constructing an orientation consistency weight map to enhance the orientation of the reconstruction error map. This gives higher sampling priority to continuous pixels along the crack extension direction, suppressing invalid high-error regions caused by background texture and noise interference, and improving the recognition ability and sampling coverage integrity of thin, low-contrast crack edges.

[0024] This invention achieves an adaptive balance between global representativeness scores and local structural anomaly scores during the iteration process through a dynamic adjustment mechanism of fused weight coefficients in hierarchical iterative sampling. The cold start phase emphasizes global diversity coverage, while in the early iteration stage, the weight of local anomaly scores is gradually increased to focus on difficult samples. In the mid-to-late iteration stages, the weight allocation is automatically adjusted based on the edge segmentation performance of the segmentation model on the validation set, ensuring that the sampling strategy always matches the model's current capabilities, accelerating model convergence and maximizing the efficiency of annotation resource utilization.

[0025] This invention achieves synergistic optimization of sampling efficiency and segmentation accuracy in the semantic segmentation of remote sensing images of road cracks. It has excellent scene adaptability, engineering feasibility and technical interpretability, and can be widely applied in pixel-level semantic segmentation fields such as intelligent inspection of traffic infrastructure, detection of lesions in medical images, and recognition of visual defects in industrial applications. Attached Figure Description

[0026] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained from these drawings without creative effort.

[0027] Figure 1 This is a schematic diagram of the overall process of the semantic segmentation active learning method based on self-supervised dual representation proposed in this invention.

[0028] Figure 2 This is a flowchart of the cold start hybrid sampling branch of the present invention.

[0029] Figure 3This is a flowchart of the local structural anomaly scoring calculation sub-process based on direction-aware weighting according to the present invention. Detailed Implementation

[0030] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the spirit or scope of the embodiments of the invention. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0031] The following is in conjunction with the appendix Figures 1-3 The embodiments of the present invention will be described in detail below.

[0032] Example 1: This example discloses an active learning method for semantic segmentation of road crack remote sensing images based on a self-supervised dual representation mechanism, with the goal of improving the annotation efficiency and segmentation accuracy of pixel-level segmentation tasks for road crack remote sensing images.

[0033] This embodiment addresses the core shortcomings of existing active learning methods through two core mechanisms: First, a self-supervised sampling mechanism with dual representation decoupling is proposed. Abandoning the strong coupling between traditional model inference, uncertainty calculation, and sampling, this mechanism explicitly inputs DINOv2 clustering features and MAE reconstruction error features as independent sampling criteria into the sampling module in each iteration, forcing the sampling process to overcome the prediction bias of the segmentation model. This mechanism enables sampling to comprehensively cover the data distribution and key boundary regions, simulating the human learning process of first observing the global structure and then focusing on detailed and difficult points, significantly improving the efficiency of high-value sample selection.

[0034] Secondly, a cold-start hybrid sampling mechanism is proposed. To address the problem of ineffective sampling in the initial unlabeled stage, a hybrid selection of cluster center samples and boundary high-error samples is automatically performed in the first iteration. This mechanism simulates the human exploration behavior in unfamiliar scenarios, balancing diversity and difficulty, and quickly obtains labeled samples with the highest information gain under unsupervised conditions, laying a high-quality foundation for subsequent model training.

[0035] The proposed method achieves automatic termination through a labeling budget and a performance threshold. During iteration, once the labeling budget is exhausted or the model performance reaches a preset metric, the loop is immediately terminated and the currently trained segmentation model is output. This design allows the framework to end training with minimal labeling cost while achieving reliable segmentation accuracy.

[0036] To facilitate a better understanding of the present invention by those skilled in the art, the present invention will be further described below in conjunction with specific implementation processes, the overall process of which is as follows: Figure 1 As shown: Step S101, Input and Initialization: Receive a semantic segmentation dataset of remote sensing images of road cracks, and distinguish between unlabeled and labeled images according to their annotation status; for example... Figure 2 As shown, the pre-trained DINOv2 and MAE models are loaded, and feature extraction is performed on the input image to construct an initial global clustering feature set. and pixel-level reconstruction error set Initialize an unlabeled sample pool. With labeled sample set The labeled sample set The initial state can be set to an empty set, and the current iteration number can be set. and maximum number of iterations , mark budget .

[0037] Step S102: Enter the hierarchical iterative sampling and training loop: Entering a loop, its continuation condition is: At the start of each loop, the iteration counter is incremented by 1.

[0038] Step S103: Construct a sampling strategy and filter samples: The sampling strategy for the current round is constructed, with its core being the fusion of DINO global clustering features and MAE boundary reconstruction error features. Specifically, if... (During the cold start phase), a hybrid sampling rule combining cluster centers and boundary high-error sampling is adopted; if Gradually increase the fusion weight of MAE boundary features; when An adaptive weighted fusion strategy is adopted, which dynamically adjusts the fusion weights of the global semantic distribution features and the pixel-level reconstruction error map based on the current crack segmentation error distribution of the model. The constructed sampling strategy is then applied to the unlabeled sample pool. Select the high-value sample set to be labeled in the current round. .

[0039] Step S104: Perform sample screening and updating. Determine the sample of the current round Does it meet the high-value criterion, i.e., the sample is located in the cluster center region or the boundary high-error region in the self-supervised feature space? If the following conditions are met, then... since Removed from the original text, and then added to the annotation set after pixel-level annotation. Update the annotation budget. If the conditions are not met, readjust the sampling weights and filter samples again until the conditions are met.

[0040] Step S105: Execute the image re-observation mechanism: Determine the current iteration round Whether sampling and labeling have been completed (i.e., training is triggered after each round of sampling).

[0041] If so: Trigger independent model training. Use the updated annotation set. The DeepLabV3+ segmentation model was trained while freezing the parameters of DINOv2 and MAE throughout the process, excluding them from gradient updates, resulting in the model for this round. .

[0042] If not, proceed directly to the next round of condition judgment.

[0043] Step S106, Loop Termination and Output: Repeat steps S102 to S105.

[0044] If the termination condition is met prematurely during the loop due to exhaustion of the annotation budget or model performance reaching the expected metric, the loop terminates and the currently trained model is output. .

[0045] If the loop reaches its maximum number of iterations... Termination means the segmentation model that has completed the last round of training will be terminated. As the final output.

[0046] Example 2: This embodiment discloses an active learning method for semantic segmentation of road crack remote sensing images based on a self-supervised dual representation mechanism. The specific steps are as follows: Step S101, Input and Initialization: Receive road crack remote sensing image semantic segmentation dataset ,in Indicates the first Zhang inputs remote sensing images, Indicates and The corresponding pixel-level annotation labels, This represents the total number of images in the dataset. The dataset is then organized according to its annotation status. It is divided into an unlabeled image subset and a labeled image subset.

[0047] like Figure 2 As shown, the DINOv2 and MAE models were pre-trained on the ImageNet-22k large-scale general image dataset. The DINOv2 model uses the ViT-B / 14 architecture, and the MAE model uses the ViT-Large architecture, with the parameters of both models kept frozen throughout the training. (Unlabeled sample pooling is also shown.) Each image in Perform the following feature extraction operations: Will Input the DINOv2 model, and take the feature vector corresponding to the [CLS] token output by the last Transformer layer as the global semantic feature representation of the image, denoted as . .

[0048] Will Input the MAE model, randomly mask the image according to a 75% mask ratio, and then reconstruct the image to obtain the reconstructed image. And calculate pixel-level reconstruction error map The calculation formula is:

[0049] in This indicates the operation of taking the absolute value pixel by pixel.

[0050] Constructing the initial global clustering feature set and pixel-level reconstruction error set Initialize an unlabeled sample pool. With the labeled sample set The labeled sample set The initial state is set to an empty set, that is Set the current iteration round. Maximum number of iterations (In this embodiment, we take) ) and total budget (in units of image count, for example) ).

[0051] Step S102: Enter the hierarchical iterative sampling and training loop: Entering a loop, its continuation condition is: At the start of each loop, the iteration counter increments. That is, execution .

[0052] Step S103: Construct a sampling strategy and filter samples: The sampling strategy for the current round is constructed, with the core strategy being the fusion of DINOv2 global clustering features and MAE boundary reconstruction error features. Samples are defined. The sampling value scoring function is:

[0053] in: :sample The comprehensive sampling value score indicates that the higher the score, the greater the value of the sample being selected.

[0054] :sample Global representativeness score based on DINOv2 global semantic features. The specific calculation method is as follows: First, the K-Means++ algorithm is used to evaluate the current labeled sample set. DINOv2 feature vectors of all images Clustering is performed to obtain Cluster centers (In this embodiment, we take) );when At that time, during the initial cold start phase, the unlabeled sample pool was directly used. Cluster the DINOv2 feature vectors of all images in the image to generate initial cluster centers; Then calculate the sample eigenvectors The minimum cosine distance to all cluster centers, i.e.:

[0055] in, This indicates that the first self-supervised model is used to analyze the samples. Extracted global semantic feature vector; Indicates the first The feature vectors of the cluster centers; The total number of cluster centers; This represents the vector dot product operation; Representing vectors Norm; the range of this rating is The larger the value, the better the sample. In semantic space, the farther away from the distribution center of already labeled samples, the stronger their global representativeness. A larger value indicates a more representative sample. The farther away from the distribution center of existing labeled samples in the semantic space, the stronger the global representativeness.

[0056] :sample Local structural anomaly scoring based on MAE reconstruction error. The specific calculation method is as follows: for the sample... Corresponding reconstruction error map All pixel values ​​are sorted in descending order, and the average of the error values ​​of the top 10% of pixels is used as the local anomaly score for that sample.

[0057] in, This indicates that the second self-supervised model is effective for the samples. After mask reconstruction, the pixel-level reconstruction error map is calculated. To reconstruct the image, This indicates the operation of taking the absolute value of each pixel; $\text{top}_{10%}(\cdot)$ means extracting the set of the top 10 values ​​after sorting all pixel values ​​in the error map in descending order. This represents the arithmetic mean of the values ​​within the set; the larger the value, the more significant the presence of high-error structural regions such as crack edges and fine cracks in the image.

[0058] The fusion weighting coefficient is used to balance the contribution ratio of global representative scores and local abnormal scores in the overall score.

[0059] Fusion weights The adjustment rules are as follows: when (Cold start phase) A fixed hybrid strategy is adopted, and the settings are... This means that the global representativeness and local anomalies are combined with equal weights to ensure that the initial sample set takes into account both diversity and coverage of difficult points.

[0060] when (In the initial iteration), gradually increase the fusion weight of MAE boundary features. Specifically, this is done in each round. This has led to a shift in sampling strategies towards identifying challenging samples, such as crack boundaries, as labeled data accumulates.

[0061] when (In the later stages of iteration), an adaptive weighted fusion strategy is adopted: the segmentation model obtained from the previous training round is calculated. The average cross-union ratio mIoU in the crack edge region on the validation set is denoted as . The crack edge region is defined as a ring-shaped area that expands outward by 5 pixels from the crack boundary in the label. If This indicates that the model's ability to segment crack edges is still insufficient, so further reduction is needed. to Strengthen the weights of MAE boundary features; otherwise, maintain constant.

[0062] Based on the above scoring function For the currently unlabeled sample pool Each sample Calculate the comprehensive sampling value score and sort all samples in descending order of score. Select the top-ranked samples. A set of 100 samples constitutes a high-value collection of samples to be labeled. The number of samples in a single round Determined by the following formula:

[0063] in This indicates the current size of the unlabeled sample pool. This represents the marked budget allocated in each round (set in this embodiment). ), This indicates rounding down to the nearest integer.

[0064] Step S104: Perform sample screening and updating. Selected high-value sample set Unlabeled sample pool Remove it from the set, i.e., perform a set difference operation:

[0065] For this set Each image in the dataset underwent meticulous pixel-level crack contour annotation by professional annotators using the LabelMe tool, generating corresponding annotation labels. After completing the annotation, the set will be... All images and their new annotations are added to the already labeled sample set. That is, to execute:

[0066] At the same time, from the total marked budget The number of samples already labeled in this round is deducted, i.e., the following is executed:

[0067] Step S105: Perform independent model training: Use the updated labeled sample set The DeepLabV3+ segmentation main network was trained end-to-end using the training data. The DeepLabV3+ model used ViT-B / 16 as the backbone network, and the decoder part used the standard DeepLabV3+ architecture. The hyperparameters used during training were: batch size of 8, initial learning rate of 0.001 with a multinomial decay strategy, AdamW optimizer with a weight decay coefficient of 0.0001, and 50 training epochs. Throughout training, the parameters of the DINOv2 and MAE models were kept completely frozen, meaning these two models were not included in the optimizer's parameter list to ensure they did not participate in any gradient calculations or backpropagation updates. After training, the segmentation model for this epoch was obtained. .

[0068] Step S106, Loop Termination and Output: Repeat steps S102 to S105 until any of the following termination conditions are triggered: Annotation budget exhausted: Total annotation budget This means there is no remaining budget for labeling new samples.

[0069] Performance convergence: The average intersection-union ratio (mIoU) of the segmentation model on the independent validation set improves by less than 0.5% for three consecutive rounds, i.e., it satisfies:

[0070] The validation set consists of 10% of the images randomly selected from the original dataset and is not used in training.

[0071] Reaching the maximum number of iterations: Current iteration number Reaching the preset limit .

[0072] If the termination condition is met prematurely during the loop due to either condition 1 or condition 2, the loop terminates immediately and the currently trained model is output. As the final result. If the loop reaches its maximum number of iterations. Termination means the segmentation model that has completed the last round of training will be terminated. This is the final output. The final output model. It can be directly used to perform pixel-level crack segmentation and prediction on newly added road crack remote sensing images.

[0073] Example 3: Building upon Example 2, this example further provides an enhanced self-supervised dual-representation active learning method that integrates a crack direction perception mechanism. This example addresses the unique elongated and continuous morphological characteristics of road cracks by introducing a multi-scale direction perception weighting strategy into the MAE reconstruction error scoring. This solves the technical problems of insufficient sensitivity to continuous crack edge sampling and easy omission of weak crack segments in existing active learning methods.

[0074] Specifically as follows: In Example 1, MAE reconstruction error scoring By averaging the pixels in the top 10% quantile of the reconstructed error map, it is possible to effectively locate structurally abnormal areas such as crack edges. However, the following phenomena are commonly found in road crack images: Crack direction continuity: Cracks usually appear as continuous curves with a certain extension direction, while the MAE reconstruction error map only reflects pixel-level reconstruction deviation and does not consider the structural direction information of the crack.

[0075] Missing weak cracks: For tiny cracks with low contrast and extremely narrow width, the absolute value of their reconstruction error is small and they are easily masked by background noise, and may be missed in global quantile screening.

[0076] Directional false detection: Reconstruction errors caused by interference factors such as asphalt pavement texture and shadows do not have a consistent directionality, but may reach a high error threshold in numerical terms, resulting in wasted sampling.

[0077] To address this, this embodiment proposes a crack orientation perception error weighting method based on Gabor multi-scale orientation filtering, which integrates prior geometric knowledge of cracks into the self-supervised feature evaluation process, further enhancing the sampling sensitivity of continuous crack edges.

[0078] In this embodiment, in step S103 of embodiment 1, ... The calculation method is enhanced and replaced as follows, and the remaining steps are exactly the same as in Example 2.

[0079] Step S103': Construct a directional enhanced sampling strategy: In calculating samples Local structural anomaly score At this time, a crack direction sensing weighted mechanism is introduced, and the specific process is as follows: (1) Multi-scale Gabor orientation field extraction For the raw remote sensing images in the unlabeled sample pool Perform grayscale conversion to obtain a grayscale image. Construct a set of two-dimensional Gabor filters with different scales and orientations:

[0080] in, , These are the rotated pixel coordinates; The standard deviation of the Gaussian envelope controls the filter scale; The direction angle of the filter; The aspect ratio of the space; The wavelength of a sine wave; For phase shift; It is an exponential function; It is a cosine function.

[0081] The symbols are defined as follows: , ; The standard deviation of the Gaussian envelope controls the filter scale; Let be the filter direction angle, take Six discrete directions; For the aspect ratio of the space, in this embodiment we take... ; The wavelength is sine wave, and in this embodiment, it is taken as... ; For phase shift, in this embodiment, we take... .

[0082] A multi-scale strategy is adopted, using three scales. For each scale and direction Calculate the amplitude of the Gabor filter response. ,in This indicates a convolution operation.

[0083] For each pixel The direction with the largest response across all scales and orientations is selected as the local principal orientation angle for that pixel. :

[0084] (2) Construction of directional consistency weight graph Defined in pixels local neighborhood centered on In this embodiment, we take Window. Calculate the mean of the principal orientation angles within this neighborhood. :

[0085] in This represents the total number of pixels in the neighborhood.

[0086] Constructing a directional consistency weight graph :

[0087] The symbols are defined as follows: : pixel The local principal orientation angle at a given location is determined by performing multi-scale Gabor filtering on the original remote sensing image, and the direction with the largest response amplitude is selected as the principal orientation of that pixel. In pixels local neighborhood centered on The arithmetic mean of the local principal orientation angles of all pixels within the area; : Basic weighting coefficient, taken in this embodiment This is used to ensure that even pixels with weak directional consistency retain a certain weight, avoiding excessive suppression; The cosine square function has the angle difference as its independent variable. When the angle difference is 0 or... The maximum value is 1 when the angle difference is 1. The minimum value of 0 is taken at this time, which is consistent with the physical meaning of the crack direction consistency; For pixels The directional consistency weight value at the location.

[0088] This weighted graph has the following characteristics: For consecutive pixels along the direction of crack extension, their principal direction is consistent with the neighborhood mean. Values ​​close to 1 indicate weights Close to 1; For background texture or noise pixels, the main direction differs greatly from the neighborhood. Values ​​close to 0, weight near ; For pixels at the crack edge, although their reconstruction error is relatively large, their weight will be appropriately reduced if the orientations are inconsistent (such as intersections), and a global representative score will be used. Supplementary sampling.

[0089] (3) Calculation of directional enhancement reconstruction error map Original reconstruction error map Consistency weighting diagram with direction Pixel-wise weighting is performed to obtain the orientation enhancement reconstruction error map. :

[0090] in, Represents the original reconstruction error map In pixel coordinates Reconstruction error value at the location; The direction consistency weight map in coordinates The weight value at the location; This represents the reconstruction error value after directional enhancement. This indicates numerical multiplication operations.

[0091] (4) Calculation of directional enhancement local anomaly score Error map reconstruction based on orientation enhancement Calculation direction enhances local structural anomaly scoring :

[0092] The mean of the first 10% quantile pixels in the reconstructed error map after directional enhancement is used as the final local anomaly score.

[0093] (5) Update of comprehensive sampling score Substitute the enhanced local anomaly score into the comprehensive sampling score function in Example 2:

[0094] in, For the sample The overall sampling value score; For fusion weighting coefficients; The global representative score is based on the global semantic features output by the first self-supervised model. To introduce a crack direction-sensing weighted enhanced local structural anomaly score; fusion weights The adjustment rules are consistent with those in Example 2.

[0095] To verify the effectiveness of this embodiment, a comparative experiment was conducted on a self-developed road crack remote sensing image dataset. The dataset contains 2000 drone-captured road surface images with a resolution of 1024*1024, covering four types of cracks: transverse cracks, longitudinal cracks, alligator cracks, and block cracks. The comparison scheme is as follows: Baseline approach: The method described in Example 2 (no directional enhancement); This embodiment uses the method described in Embodiment 3 (including directional enhancement).

[0096] The experiments used the same initial conditions: cold start phase. Select 50 samples, and then select 50 samples in each subsequent round. Total annotation budget The segmentation models used were all DeepLabV3+ (ViT-B / 16), and the evaluation metrics were the mean intersection-union ratio (mIoU) and the F1-score of the crack edge.

[0097] Experimental results show that: Under the same annotation budget, the mIoU value achieved by the proposed solution is 3.2% higher than that of the baseline solution; The F1-score for crack edges was improved by 5.8%, with a particularly significant 12.4% improvement in the detection rate of minute cracks less than 3 pixels wide. Sampling trajectory analysis shows that the scheme in this embodiment locks the continuous crack segment earlier in the iteration, reducing invalid sampling in the background high error area.

[0098] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0099] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. It should be noted that any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An active learning method for semantic segmentation of road crack remote sensing images based on a self-supervised dual representation mechanism, characterized in that, Includes the following steps: Step S1, Data Preparation and Initialization: Obtain the remote sensing image dataset of road cracks and divide it into an unlabeled sample pool and a labeled sample set; initialize the first self-supervised model for global semantic feature extraction, the second self-supervised model for local reconstruction error calculation, and the semantic segmentation main network; Step S2, Self-supervised dual representation feature extraction: The first self-supervised model with parameter freezing is used to extract global semantic distribution features from the images in the unlabeled sample pool. The second self-supervised model with parameter freezing is used to perform mask reconstruction on the images in the unlabeled sample pool and calculate the pixel-level reconstruction error map. The global semantic distribution features and the pixel-level reconstruction error map together constitute the basis for sample value evaluation. Step S3, Hierarchical Iterative Sampling and Model Training Loop: Step S3-1: Construct a sampling strategy: The global semantic distribution features and the pixel-level reconstruction error map are weighted and fused to generate a comprehensive sampling value score. In the first iteration, a hybrid sampling strategy combining cluster center samples and high-error boundary region samples is adopted. Subsequent iterations gradually adjust the fusion weight of the global semantic distribution features and the pixel-level reconstruction error map. Step S3-2, High-value sample screening: Based on the comprehensive sampling value score of the samples, a set of high-value samples is screened out from the unlabeled sample pool and removed from the unlabeled sample pool; Step S3-3, Labeling and Data Update: Perform pixel-level labeling on the high-value sample set and add the labeled samples to the labeled sample set; Step S3-4: Independent training of segmentation models: The semantic segmentation main network is trained using the updated labeled sample set as training data; during the training process, the parameters of the first self-supervised model and the second self-supervised model are kept frozen and do not participate in gradient updates; Step S4, Iteration Termination and Model Output: Repeat steps S2 to S3-4 until the preset termination condition is met, and output the final trained segmentation model.

2. The active learning method for semantic segmentation of road crack remote sensing images based on a self-supervised dual representation mechanism as described in claim 1, characterized in that, The first self-supervised model is a DINOv2 model pre-trained on a large-scale general image dataset; the second self-supervised model is a MAE model pre-trained on a large-scale general image dataset.

3. The active learning method for semantic segmentation of road crack remote sensing images based on a self-supervised dual representation mechanism as described in claim 1, characterized in that, In step S3-1, the comprehensive sampling value score of the sample is determined. The calculation formula is: in, Indicates sample The overall sampling value score; For the sample Global representativeness score based on global semantic features output by the first self-supervised model; For the sample Local structural anomaly scoring based on the reconstruction error map output by the second self-supervised model; For fusion weighting coefficients.

4. The active learning method for semantic segmentation of road crack remote sensing images based on a self-supervised dual representation mechanism as described in claim 3, characterized in that, Global representative score By calculating samples The minimum cosine distance between the global semantic feature vector and the cluster centers of the currently labeled sample set is obtained; the local structural anomaly score is obtained. By calculating samples The mean error of the pixels with preset high quantiles is obtained from the reconstruction error map.

5. The active learning method for semantic segmentation of road crack remote sensing images based on a self-supervised dual representation mechanism according to claim 3, characterized in that, The fusion weight coefficient The adjustment rules are as follows: The initial value is set to a preset value during the first iteration; it is gradually reduced during the first few iterations. The weight of the score for local structural anomalies is increased; in subsequent iterations, the weight is adaptively adjusted based on the edge region segmentation accuracy of the current segmentation model on the validation set. value.

6. The active learning method for semantic segmentation of road crack remote sensing images based on a self-supervised dual representation mechanism according to claim 1, characterized in that, In step S3-2, the specific method for screening high-value samples is as follows: calculate the comprehensive sampling value score for each sample in the unlabeled sample pool, sort them in descending order of score, and select the top... The high-value sample set consists of [number] samples, where [number] samples constitute the high-value sample set. Determined by the single-round annotation budget.

7. The active learning method for semantic segmentation of road crack remote sensing images based on a self-supervised dual representation mechanism according to claim 1, characterized in that, The semantic segmentation main network in steps S3-4 adopts the DeepLabV3+ architecture and uses VisionTransformer as the backbone network.

8. The active learning method for semantic segmentation of road crack remote sensing images based on a self-supervised dual representation mechanism according to claim 1, characterized in that, The preset termination conditions include any one or more of the following: the entire budget for labeling is consumed, the average intersection-union ratio of the segmentation model on the validation set is lower than a preset threshold for several consecutive rounds, and the number of iterations reaches a preset maximum limit.

9. The active learning method for semantic segmentation of road crack remote sensing images based on a self-supervised dual representation mechanism according to any one of claims 1 to 8, characterized in that, The unlabeled sample pool and the labeled sample set are physically isolated during data storage, retrieval and updating to ensure that the data sources for sampling feature extraction and segmentation model training are independent of each other.

10. The active learning method for semantic segmentation of road crack remote sensing images based on a self-supervised dual representation mechanism according to claim 3, characterized in that, Local structural anomaly score The calculation further includes a crack direction-sensing weighted step: Multi-scale Gabor filtering was applied to the raw remote sensing images in the unlabeled sample pool to extract the local orientation field of the crack region. ; A directional consistency weight map w(x,y) is constructed based on the local orientation field, such that the weight value of pixel (x,y) is related to the difference between the local orientation angle φ and the neighborhood mean φ_bar. When the difference approaches 0 or π, the weight approaches 1; when the difference approaches π / 2, the weight approaches a preset basic weight coefficient α; where α∈(0,1); the directional consistency weight map is then compared with the pixel-level reconstruction error map. Pixel-wise weighting is performed to obtain the orientation enhancement reconstruction error map. ; Based on the aforementioned direction-enhanced reconstruction error map Calculate the local structural anomaly score ,in This indicates pixel-by-pixel multiplication.

Images