Weak supervision segmentation method combining multi-dimensional uncertainty evaluation and anomaly correction

Abstract

The invention discloses a weak supervision segmentation method in combination with multi-dimensional uncertainty evaluation and anomaly correction. The method comprises the following steps: constructing an initial pseudo-label graph with physical boundary constraints; outputting a pixel-level category probability distribution diagram through multi-scale feature extraction and cavity convolution decoding; pixel semantic similarity, space consistency and time stability are comprehensively calculated through multi-dimensional uncertainty evaluation, and a full-graph self-adaptive uncertainty graph is generated in real time; comparing the depth feature with a class prototype by using an exception interception mechanism, and associating an MADUE weight to realize dynamic refining of a pseudo tag; and a dynamic temperature coefficient is introduced, a weighted loss function is constructed in combination with refined pseudo labels, prototype constraints and graffiti supervision, parameters are iteratively updated, and a high-precision binary segmentation map is output. According to the method, the problem of noise interference caused by complex illumination on the surface of Mars and changeable terrain can be solved, and the segmentation precision and boundary quality of the rock target are remarkably improved while semantic confusion of pseudo tags is effectively suppressed.

Description

Technical Field

[0001] This application belongs to the field of deep learning and remote sensing image segmentation technology, specifically involving a weakly supervised segmentation method that combines multidimensional uncertainty assessment and anomaly correction. Background Technology

[0002] In Mars exploration missions, accurate rock segmentation is a core prerequisite for analyzing Martian landforms, geological structures, and environmental evolution, providing crucial data support for probe path planning, obstacle avoidance, and scientific sampling and analysis. Traditional segmentation methods largely rely on dense, pixel-level manual annotation. However, the Martian environment is complex, and obtaining large-scale, high-quality annotations is not only extremely costly but also time-consuming and labor-intensive, severely limiting the development of related algorithms.

[0003] While existing research has incorporated semi-supervised learning (SSL) to reduce reliance on labeled data, it still requires a small amount of fully labeled data as a training foundation. Weakly supervised learning (WSL), as an alternative paradigm, can achieve efficient model training using image-level labels, bounding boxes, point labels, or scribble labels. Among these, scribble labels achieve a good balance between annotation efficiency and spatial guidance. However, for the specific task of identifying Martian rocks, existing weakly supervised segmentation frameworks still have significant limitations in adapting to the complex Martian landscape.

[0004] In the pseudo-label generation stage, traditional region growing or probabilistic graphical models rely heavily on local color and spatial continuity cues, making it difficult to capture accurate semantic boundaries. This results in coarse supervision signals containing significant noise. While basic models like SAM enhance boundary awareness, mainstream methods such as the SAM-induced framework often rely on point-guided local decoupling, neglecting the global scene context and easily producing partial or biased masks. During label propagation and refinement, existing methods like DBFNet or URSS often ignore feature similarity between objects, and single deep features lack stability when handling the heterogeneity of rocks and backgrounds. Furthermore, most uncertainty regularization methods, such as CC4S, focus only on reducing noise within pseudo-labels rather than enhancing semantic alignment. Existing mechanisms lack multidimensional evaluation based on high-dimensional semantic prototypes, failing to comprehensively quantify the reliability of prediction results from dimensions such as pixel spatial consistency and training time stability. The lack of this evaluation dimension further leads to the lack of anomaly correction logic, making it difficult for the model to effectively identify and correct isolated noise and confused regions in pseudo-labels through feature prototype comparison under extremely sparse annotation. Ultimately, it cannot fundamentally resolve the inherent semantic alignment risk of pseudo-labels, limiting the further narrowing of the performance gap between weak supervision and full supervision.

[0005] In summary, existing weakly supervised segmentation methods still have significant shortcomings in generating high-quality pseudo-labels, measuring multidimensional uncertainty, and correcting anomalies in audits. Summary of the Invention

[0006] This application provides a weakly supervised segmentation method that combines multidimensional uncertainty assessment and anomaly correction to solve the above-mentioned technical problems.

[0007] To address the aforementioned technical problems, this application adopts the following technical solution: a weakly supervised segmentation method combining multidimensional uncertainty assessment and anomaly correction, comprising:

[0008] S1. Image-level multidimensional features are extracted using prompt-free SAM and extremely sparse doodle semantic information is propagated to construct an initial pseudo-label map with physical boundary constraints, which serves as the basic supervision signal for online training;

[0009] S2. Input the preprocessed image into the ResNet-101+DeepLabV3 network, and output a pixel-level class probability distribution map through multi-scale feature extraction and dilated convolution decoding;

[0010] S3. By comprehensively calculating pixel semantic similarity, spatial consistency and temporal stability through multidimensional uncertainty assessment, a full-image adaptive uncertainty map is generated in real time.

[0011] S4. Utilize the anomaly interception mechanism to compare deep features with class prototypes and associate them with MADUE weights. Through label remapping or confidence masking, dynamically refine the pseudo-labels.

[0012] S5. Introducing a dynamic temperature coefficient maps uncertainty to pixel-level weights. Combining refined pseudo-labels, prototype constraints, and graffiti supervision, a weighted loss function is constructed. The parameters are iteratively updated, and a high-precision binary segmentation map is output.

[0013] Furthermore, the method in step S1 includes:

[0014] S11. Load the pre-trained SAM model and start the automatic mask generator. Traverse the original image and generate a set of image target masks in the prompt-free mode.

[0015] S12. Extract the spectral mean, gray-level co-occurrence matrix texture and geometric features of each mask region, match the overlap between the mask and the sparse graffiti labels, allocate the optimal category label according to the overlap area to achieve feature propagation, and construct an initial pseudo-label map with physical boundary constraints.

[0016] Furthermore, the method in step S2 includes:

[0017] The preprocessed raw Mars image is input into a segmentation network with ResNet-101 as the encoder and DeepLabV3 as the decoder. The encoder extracts high-dimensional multi-scale semantic features, and the receptive field is expanded through the dilated convolutional layer of the decoder. The pixel-level class probability distribution map is output during forward propagation, providing probability input for subsequent uncertainty measurement.

[0018] Furthermore, the method in step S3 includes:

[0019] S31. Calculate the original semantic uncertainty based on the cosine similarity between the current features and the semantic prototype;

[0020] S32. Use average pooling to detect the distribution difference between pixels in the initial pseudo-label image and the 8-neighbor labels in the initial pseudo-label image, and calculate the spatial consistency uncertainty.

[0021] S33. Align the current forecast map with the results recorded in the historical forecast buffer by size and class, and calculate the time stability score based on EMA to quantify the time uncertainty;

[0022] S34. The original semantic uncertainty, spatial consistency uncertainty and temporal uncertainty indicators are weighted and fused to generate a full-image adaptive MADUE uncertainty map, thereby accurately quantifying the noise risk of each pixel in the initial pseudo-label.

[0023] Furthermore, the method for monitoring and updating the time stability score in step S33 includes:

[0024] During the model initialization phase, a stability score storage space and a historical prediction storage space are registered. In each training iteration, the current pixel-level prediction result is aligned with the corresponding historical prediction result in size. By comparing the consistency between the two in pixel category, dynamic weight fusion update is performed using an adaptive update rate to maintain the stability score of each image sample in real time. The data loader is simultaneously set to a fixed sequence mode to ensure consistent matching between the image index and the sample information in the storage space.

[0025] Furthermore, the method in step S4 includes:

[0026] S41. During the iteration process, start the Anomaly Interception and Repair Pipeline (ARAR) to extract the current predicted features in real time and compare them with the similarity of the rock and background prototypes;

[0027] S42. If a pixel is classified as rock in the initial pseudo-label, but its feature similarity to the prototype is lower than a preset threshold and its spatial uncertainty in the MADUE uncertainty map is higher than a preset level, then perform "hard correction" to flip it to the background category.

[0028] S43. If a pixel is the background in the initial pseudo-label, but the similarity between the feature and the prototype is higher than the preset threshold and the total uncertainty of MADUE is higher than the preset level, then perform "soft masking" to set it as an ignored label, thereby dynamically generating high-quality refined pseudo-labels.

[0029] Furthermore, the method in step S5 includes:

[0030] S51. Introduce a temperature coefficient t that is dynamically adjusted with the training cycle. By setting different coefficient values, control the smoothness of weight distribution and construct a dynamic temperature-weighted cross-entropy loss.

[0031] S52. Based on the exponential decay function with temperature coefficient t as parameter, the full-image adaptive uncertainty is mapped to pixel-level audit weights, and a multi-dimensional weighted cross-entropy loss is constructed.

[0032] S53. Based on the effective pixel mask, pixel regions with weights higher than the preset reliability threshold and not ignored labels are selected in real time for gradient aggregation to eliminate the interference of high noise regions on model updates.

[0033] S54. Input the image to be tested, combine the sparse graffiti supervision loss and the prototype constraint loss to drive the network parameter iteration, and when the validation set performance index reaches the preset optimal condition, lock the model weights and output a high-precision binary segmentation map.

[0034] Furthermore, the method for outputting a high-precision binarized segmentation map in step S54 includes:

[0035] S541. Input the image to be tested into the pre-saved optimal performance model to perform forward inference and obtain the pixel-level class probability distribution;

[0036] S542. Based on the category probability distribution, perform confidence normalization and execute a category competition mechanism to extract the category attribute with the highest response value for each pixel;

[0037] S543. Based on the preset category mapping rules, the pixels are determined to be either target rocks or background areas, and finally a pixel-level binary segmentation image reflecting the physical distribution characteristics of the target to be identified is generated.

[0038] The beneficial effects of this application are: by extracting current features in real time and comparing them with class prototypes for deep similarity, combined with spatial island detection logic, this application can perform "hard correction" or "soft masking" on noise in the initial pseudo-labels. This mechanism enables the initial object-level mask to be dynamically refined to high-fidelity pixel-level labels during training, significantly improving the model's accuracy in depicting fine boundaries and complex textures of rocks.

[0039] This application not only considers the similarity of semantic features, but also innovatively introduces spatial local consistency evaluation based on 8 neighborhoods and EMA temporal stability monitoring based on historical prediction buffers. By quantizing the logical coherence of pixel predictions in spatial distribution and their convergence state during the training period using MADUE, this application can more comprehensively and accurately identify ambiguous regions in pseudo-labels, significantly enhancing the system's robustness to interference from complex environments. Attached Figure Description

[0040] Figure 1 This is an overall flowchart of an embodiment of the weakly supervised segmentation method combining multidimensional uncertainty assessment and anomaly correction of this application;

[0041] Figure 2 yes Figure 1 A flowchart illustrating step S1 of an embodiment;

[0042] Figure 3 This is a network architecture diagram of an embodiment of the weakly supervised segmentation method combining multidimensional uncertainty assessment and anomaly correction in this application;

[0043] Figure 4 yes Figure 1 A flowchart illustrating step S3 in one embodiment;

[0044] Figure 5 This is a schematic diagram of the structure of a weakly supervised segmentation method combining multidimensional uncertainty assessment and anomaly correction according to an embodiment of this application;

[0045] Figure 6 yes Figure 1 A flowchart illustrating step S4 in one embodiment;

[0046] Figure 7 yes Figure 1 A flowchart illustrating step S5 of an embodiment;

[0047] Figure 8 This is a visualization comparison of the TWMARS dataset results of an embodiment of the weakly supervised segmentation method combining multidimensional uncertainty assessment and anomaly correction proposed in this application.

[0048] Figure 9 This is a visualization comparison of the MarsData-V2 dataset results of an embodiment of the weakly supervised segmentation method combining multidimensional uncertainty assessment and anomaly correction proposed in this application.

[0049] Figure 10 This is a visualization comparison of the MSLRock dataset, representing an embodiment of the weakly supervised segmentation method combining multidimensional uncertainty assessment and anomaly correction proposed in this application. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with specific embodiments.

[0051] Many specific details are set forth in the following description in order to provide a full understanding of this application. However, this application may also be implemented in other ways than those described herein, and therefore this application is not limited to the specific embodiments disclosed in the following specification.

[0052] See Figure 1 , Figure 1 This is an overall flowchart of an embodiment of a weakly supervised segmentation method combining multidimensional uncertainty assessment and anomaly correction, as proposed in this application. The method includes:

[0053] S1. Image-level multidimensional features are extracted using prompt-free SAM and extremely sparse doodle semantic information is propagated to construct an initial pseudo-label map with physical boundary constraints, which serves as the basic supervision signal for online training.

[0054] Specifically, the algorithm utilizes prompt-free SAM to generate image-level object candidate regions, extracts spectral, texture, and geometric features, propagates extremely sparse graffiti semantic information, and constructs an initial pseudo-label map with physical boundary constraints as the basic supervision signal for online training.

[0055] See Figure 2 The method of step S1 includes:

[0056] S11. Load the pre-trained SAM model and start the automatic mask generator. Traverse the original image and generate a set of image target masks in prompt-free mode.

[0057] Specifically, in this embodiment, the experimental data comes from three publicly available datasets, covering diverse surface scenes collected by different Mars rovers. All datasets are uniformly processed into a binary segmentation task containing two categories: "rocks" and "background," and simulated extremely sparse linear graffiti annotations as weak supervision signals.

[0058] The TWMARS dataset, acquired by the navigation and terrain camera of the Mars rover Zhurong, contains 336 high-resolution RGB images with an original resolution of 1024×1024 pixels, showcasing the topographic features of Utopia Plain on Mars.

[0059] MarsData-V2 dataset: Collected by the Mastcam camera of the Curiosity rover, this dataset contains 839 raw, unenhanced images with a resolution of 512×512 pixels, primarily featuring large rocky targets. MSLRock dataset: Also from the Curiosity mission, this dataset retained 318 high-quality, clearly labeled images after quality screening, with a resolution of 640×640 pixels.

[0060] To ensure the scientific rigor of the experiment, the datasets were randomly divided into three sets: 80% training set, 10% validation set, and 10% test set. For example... Figure 3 The OPG strategy steps are as follows: Before inputting the image into the network, the image is resized and normalized. For the pre-processed image, a pre-trained ViT-L version of SAM, balancing accuracy and computational cost, is used as a general segmenter. A promptless full-image automatic generation mode is employed for the input image. Perform a gridded scan to automatically generate a set of candidate object masks covering the entire image. ,in This represents the total number of objects segmented. This step ensures that the initial mask has high-precision boundary constraints that closely match the physical edges of the rock, overcoming the shortcomings of blurred boundaries in traditional superpixel segmentation.

[0061] S12. Extract the spectral mean, gray-level co-occurrence matrix texture and geometric features of each mask region, match the overlap between the mask and the sparse graffiti label, allocate the optimal category label according to the overlap area to achieve feature propagation, and construct the initial pseudo label with physical boundary constraints.

[0062] Specifically, this involves generating a set of candidate masks for objects that cover the entire image. Each object mask in Three types of physically interpretable features are extracted separately and then integrated into a multi-dimensional complementary object-level feature representation. The three features that are physically interpretable are as follows:

[0063] Spectral characteristics : Calculate the average intensity of all pixels in the mask area across the R, G, and B channels to reflect the color reflectivity information of the rock surface.

[0064] Texture features The contrast and difference within the masked area are calculated based on the gray-level co-occurrence matrix, which is used to describe the roughness and texture pattern of the rock surface.

[0065] Geometric features Calculates the area, perimeter, and roundness of the mask region. Used to distinguish rocks from irregular backgrounds.

[0066] Finally, the above features are concatenated to form an object-level feature representation: .

[0067] The process of semantic propagation and initial pseudo-tag construction and anchor point allocation is as follows:

[0068] The manually annotated sparse graffiti labels are projected onto the object mask generated by SAM. If the mask... If there is spatial overlap with the graffiti lines, the mask is identified as an "anchor object" and directly inherits the graffiti's category label, i.e., rock or background. For unlabeled objects not covered by graffiti, the feature vector is used as the basis for... Calculate the Euclidean distance between the unlabeled object and all anchor objects in the feature space. Employ a nearest neighbor search strategy to classify the unlabeled object into the category of the anchor object with the closest feature distance. Finally, map the predicted categories of all objects back to the pixel space to generate a dense initial pseudo-label map. While this pseudo-label may contain some semantic noise, it possesses accurate physical boundary information, providing a solid foundation of supervision signals for the subsequent online training steps S2-S5.

[0069] S2. Input the preprocessed image into the ResNet-101+DeepLabV3 network, and output a pixel-level class probability distribution map through multi-scale feature extraction and dilated convolution decoding.

[0070] Specifically, the preprocessed raw Mars image is input into the ResNet-101+DeepLabV3 network. The encoder extracts multi-scale semantic features, and the decoder expands the receptive field through dilated convolution, outputting a pixel-level category probability distribution map to provide input for uncertainty auditing.

[0071] See Figure 3 As shown in the online training module, this step constructs a deep neural network architecture for online training, which includes three parts: data augmentation, encoder feature extraction, and decoder multi-scale fusion. The method of step S2 is as follows:

[0072] S21. During the training iteration process, the original Mars image as input is first processed. Online data augmentation was performed. A random cropping strategy was adopted to crop the original images of different resolutions to a fixed size to unify the network input. Subsequently, random horizontal flipping and photometric distortion operations were performed to simulate the changes in lighting conditions and shooting angles on Mars, enhancing the model's robustness to environmental noise. Finally, Z-score normalization was applied to the images, subtracting the mean of the RGB channels of the dataset and dividing by the standard deviation to normalize the pixel values ​​to a stable distribution range, accelerating network convergence.

[0073] S22. The backbone network trained online uses ResNet-101 as the feature extraction encoder.

[0074] The network comprises four main residual stages (Layer 1 to Layer 4), each consisting of stacked bottleneck residual blocks. After initial dimensionality reduction via 7×7 convolutions and pooling layers, the image passes through these residual stages sequentially. Residual connections are used to address the vanishing gradient problem in deep networks, enabling the network to learn hierarchical features from low-level texture to high-level semantics. To preserve more spatial detail, the downsampling stride of Layer 4 is removed and replaced with dilated convolutions, maintaining the spatial resolution of the final high-dimensional feature map at 1 / 8 or 1 / 16 of the input image. This feature map contains rich semantic context, used not only for prediction generation but also directly transmitted to subsequent multidimensional uncertainty evaluation modules for calculating semantic uncertainty and anomaly interception and repair modules for feature comparison.

[0075] S23. The backbone network trained online uses DeepLabV3 as the decoder.

[0076] The core component of the decoder is the dilated spatial pyramid pooling module, which contains five parallel branches: a 7×7 convolutional branch for dimensionality reduction; three 3×3 dilated convolutional branches for capturing multi-scale features of unstructured targets under different receptive fields; and a global average pooling branch for integrating image-level global contextual information. The output features of these five branches are concatenated along the channel dimension and fused using 1×1 convolutions. Finally, bilinear interpolation is used to upsample to the original image size. The final output is a pixel-level class prediction probability distribution map, where the value of each pixel represents the confidence score of belonging to the corresponding class at that location. This probability map not only serves as the current segmentation result but also as a key input for calculating the time uncertainty in step S3.

[0077] S3. By comprehensively calculating pixel semantic similarity, spatial consistency and temporal stability through multidimensional uncertainty assessment (MADUE), an adaptive uncertainty map of the entire image is generated in real time.

[0078] For details, please refer to Figure 4-5 The method of step S3 includes:

[0079] S31. Calculate the original semantic uncertainty based on the cosine similarity between the current features and the semantic prototype.

[0080] Specifically, semantic dimension uncertainty assessment This metric measures confidence based on distribution distance in the feature space. Using the high-dimensional feature map extracted by S2, the cosine similarity between the feature vector of each pixel and the "rock" or "background" prototype in the class prototype cache is calculated. The class prototypes are the cluster centers of each category in the feature space and are continuously updated during training. If a pixel's feature is far from the prototype of its category, it indicates that the pixel's semantic features are ambiguous or deviate from the distribution center, thus assigning it high semantic uncertainty.

[0081] S32. Utilize average pooling to detect the distribution differences between pixels in the initial pseudo-label image and the 8-neighbor labels in the initial pseudo-label image, and calculate the spatial consistency uncertainty.

[0082] Specifically, spatial dimension uncertainty assessment This metric aims to identify spatially isolated noise points in the initial pseudo-labels. The initial pseudo-label map is generated from input S1. The consistency of each pixel is detected using an 8-neighborhood convolution audit mechanism. The 8-neighborhood refers to the set of eight adjacent pixels centered at the target pixel (i,j), encompassing all directly adjacent pixels horizontally, vertically, and diagonally. In this step, a 3×3 average pooling operator is used as a spatial detector to perform convolution calculations on the initial pseudo-labels to obtain the average label value of the target pixel and its 8-neighborhood. By analyzing the label consistency within the local neighborhood, if the average label value of a pixel's 8-neighborhood is close to 0.5, it indicates that rock and background pixels are mixed within the neighborhood, thus indicating high spatial inconsistency in that area; if the average value is close to 0 or 1, it indicates high consistency in the neighborhood categories, resulting in low spatial uncertainty.

[0083] The formula for calculating the spatial uncertainty fraction is:

[0084] .

[0085] S33. Align the current forecast map with the results recorded in the historical forecast buffer by size and class, and calculate the time stability score based on EMA to quantify the time uncertainty.

[0086] Specifically, uncertainty assessment in the time dimension This metric aims to capture the oscillations in model predictions during training. A historical prediction buffer is introduced to store the prediction results of the entire image in the previous round. In the current training round, the current predicted probability map output by S2 is compared pixel-by-pixel with the records in the historical buffer. The stability score of the entire image is updated using the EMA mechanism. If the prediction result of a pixel changes frequently between consecutive rounds, its stability score decreases, and the corresponding temporal uncertainty increases.

[0087] The method for monitoring and updating the time stability score in step S33 includes:

[0088] S331. During the model initialization phase, register the stability score storage space and the historical prediction storage space;

[0089] S332. In each training iteration, the current pixel-level prediction result is aligned with the corresponding historical prediction result in size. By comparing the consistency between the two in pixel category, dynamic weight fusion update is performed using an adaptive update rate to maintain the stability score of each image sample in real time.

[0090] S333. Synchronously set the data loader to fixed sequence mode to ensure consistent matching between the image index and the sample information in the storage space.

[0091] S34. The original semantic uncertainty, spatial consistency uncertainty and temporal uncertainty indicators are weighted and fused to generate a full-image adaptive MADUE uncertainty map, thereby accurately quantifying the noise risk of each pixel in the initial pseudo-label.

[0092] Specifically, the spatial uncertainty calculated above... Time uncertainty and semantic uncertainty Adaptive weighted fusion is performed to generate the final total uncertainty map of the entire graph. This graph quantifies the unreliability of each pixel in the image and serves as a key control signal passed to the subsequent anomaly interception and repair module and loss calculation module.

[0093] The fusion formula is as follows:

[0094] ;

[0095] in, , , These are the weight parameters.

[0096] S4. Utilize the Anomaly Interception (ARAR) mechanism to compare deep features with class prototypes and associate them with MADUE weights. Through label remapping or confidence masking, achieve dynamic refinement of pseudo-labels.

[0097] Specifically, by combining the deep features in the category probability distribution map with the comparison results of the class prototype, and associating them with the multidimensional uncertainty weights of the full-map adaptive uncertainty map, an anomaly interception mechanism is established to identify isolated noise and semantically confused regions in pseudo-labels. By performing forced label remapping or confidence masking, the dynamic refinement of pseudo-labels is achieved.

[0098] See Figure 5-6This step executes the anomaly interception and repair pipeline. This module receives the initial pseudo-labels generated in S1, the high-dimensional feature map extracted in S2, and the spatial uncertainty and total uncertainty maps calculated in S3 as input. It dynamically optimizes the label quality through logical rules. The method for step S4 is as follows:

[0099] S41. During the iteration process, start the Anomaly Interception and Repair Pipeline (ARAR) to extract the current predicted features in real time and compare them with the similarity of the rock and background prototypes.

[0100] Specifically, the high-dimensional feature map output from the category probability distribution map is compared with the "rock" prototype vector in the class prototype cache using cosine similarity calculation to generate a rock similarity map for the entire image. This metric reflects how much the current pixel resembles a "rock" in semantic features, providing a semantic basis for subsequent error correction.

[0101] S42. If a pixel is classified as rock in the initial pseudo-label, but its feature similarity to the prototype is below a preset threshold and its spatial uncertainty in the MADUE uncertainty map is above a preset level, then perform a "hard correction" to flip it to the background category.

[0102] Specifically, a hard correction is performed on "isolated noise points," a mechanism designed to remove obvious mislabels. Pixels across the entire image are evaluated point-by-point. If an initial pseudo-label is determined to be "rock," but its features have extremely low similarity to a rock prototype, and the spatial uncertainty of the pixel is extremely high, then the system determines that the point is a mislabeled background noise point. In practice, the label of that pixel is forcibly flipped to "background."

[0103] S43. If a pixel is the background in the initial pseudo-label, but the similarity between the feature and the prototype is higher than the preset threshold and the total uncertainty of MADUE is higher than the preset level, then perform "soft masking" to set it as an ignored label, thereby dynamically generating high-quality refined pseudo-labels.

[0104] A soft masking mechanism is applied to "class confusion regions," designed to handle ambiguous boundary areas. If the initial pseudo-label is determined to be "background," but its features are highly similar to the rock prototype, and the model has high overall uncertainty regarding this area, it indicates that the region may be an unlabeled rock or a difficult-to-distinguish shadow. To avoid the negative impact of incorrect supervision on the network gradient, the label of this pixel is set to "ignore," preventing it from participating in gradient backpropagation and parameter updates in step S5.

[0105] Combining the results of the hard correction and soft masking processes, the final refined pseudo-labels are generated. These labels not only eliminate high-confidence noise interference but also mask potential ambiguous regions, thereby significantly improving the semantic consistency and boundary accuracy of the pseudo-labels and providing high-quality supervision signals for subsequent weighted loss calculations.

[0106] S5. Introducing a dynamic temperature coefficient maps uncertainty to pixel-level weights. Combining refined pseudo-labels, prototype constraints, and graffiti supervision, a weighted loss function is constructed. The parameters are iteratively updated, and a high-precision binary segmentation map is output.

[0107] Specifically, a dynamic temperature coefficient t that varies with the number of rounds is introduced to map the uncertainty graph into pixel-level weights. A weighted cross-entropy loss is constructed by combining refined pseudo-labels, and parameters are updated by superimposing sparse graffiti supervision and prototype constraint loss. After training balance, the optimal weights are locked, and a high-precision binarized segmentation map is output.

[0108] See Figure 7 The method for step S5 is as follows:

[0109] S51. Introduce a temperature coefficient t that is dynamically adjusted with the training cycle. By setting different coefficient values, control the smoothness of weight distribution and construct a dynamic temperature-weighted cross-entropy loss.

[0110] S52. Based on the exponential decay function with temperature coefficient t as parameter, the full-image adaptive uncertainty is mapped to pixel-level audit weights, and a multi-dimensional weighted cross-entropy loss is constructed.

[0111] Specifically, in the early stages of training, a higher temperature value is set to make the weight distribution smoother, tolerate a certain degree of uncertainty, and prevent the model from converging to erroneous samples too early; in the later stages of training, the temperature value is lowered to enter the "strict audit" mode, forcing the model to focus only on high-confidence regions.

[0112] Using formula The total uncertainty map generated in step S3 is mapped to a pixel-level weight matrix W. Simultaneously, an effective pixel masking strategy is applied to filter out invalid pixels with excessively low weights, and the multi-dimensional weighted cross-entropy loss between the refined pseudo-label and the predicted probability is calculated only for reliable regions.

[0113] S53. Based on the effective pixel mask, pixel regions with weights higher than the preset reliability threshold and not ignored labels are selected in real time for gradient aggregation to eliminate the interference of high noise regions on model updates.

[0114] S54. Input the image to be tested, combine the sparse graffiti supervision loss and the prototype constraint loss to drive the network parameter iteration, and when the validation set performance index reaches the preset optimal condition, lock the model weights and output a high-precision binary segmentation map.

[0115] Specifically, the mixed objective function is calculated and backpropagation is performed, resulting in the total loss function. It consists of three parts:

[0116] (1) Loss of sparse graffiti Calculate the cross-entropy between the predicted probability map and the original sparse graffiti labels, and use it as an absolute truth constraint to prevent semantic drift of the model during the learning process of unlabeled regions;

[0117] (2) Dynamic weighted loss : It dominates pixel-level classification learning of the entire image and automatically suppresses noise interference by using uncertain weights;

[0118] (3) Prototype constraint loss : Calculate the distance constraint between the high-dimensional feature map and the class prototype, so that similar features are closely clustered in the embedding space, while dissimilar features are separated from each other.

[0119] The total loss formula is:

[0120] ;

[0121] in, , , These are the weight parameters.

[0122] By minimizing this total loss, the stochastic gradient descent algorithm is used to perform backpropagation of the error, thereby iteratively updating the weight parameters of the deep neural network.

[0123] At the end of each training iteration, an auxiliary cache update operation is performed synchronously. This involves updating the class prototype cache based on the feature mean of the current batch and overwriting the historical prediction buffer based on the current prediction results. Once the training reaches a preset number of iterations or the validation set metrics stabilize, the optimal model weights are locked. During the inference phase, the Mars image to be tested is input into the trained network to obtain the predicted probability map. A maximum value indexing operation is performed on this map, and the final high-precision binarized rock segmentation mask is output.

[0124] The method for outputting a high-precision binarized segmentation map in step S54 includes:

[0125] S541. Input the image to be tested into the pre-saved optimal performance model to perform forward inference and obtain the pixel-level class probability distribution;

[0126] S542. Based on the category probability distribution, perform confidence normalization and execute a category competition mechanism to extract the category attribute with the highest response value for each pixel;

[0127] S543. Based on the preset category mapping rules, the pixels are determined to be either target rocks or background areas, and finally a pixel-level binary segmentation image reflecting the physical distribution characteristics of the target to be identified is generated.

[0128] In one embodiment, extensive quantitative comparative experiments were conducted based on three public datasets: TWMARS, MarsData-V2, and MSLRock. Regarding the experimental setup, this application is implemented using the PyTorch deep learning framework and trained and inferred on a computing platform equipped with an NVIDIA GeForce RTX 4090 GPU. The specific hardware and algorithm environment configuration is as follows: The network architecture adopts a segmentation model with ResNet-101 as the encoder and DeepLabV3 as the decoder. The encoder is loaded with pre-trained weights and utilizes a multi-grid structure and dilated convolutions to extract high-dimensional features. For the training strategy, the total number of training epochs is set to 200, using a stochastic gradient descent (SGD) optimizer with an initial learning rate of 0.01, momentum of 0.9, and weight decay of 0.0005. The learning rate adjustment adopts a Poly strategy with its power parameter set to 0.9.

[0129] To comprehensively evaluate the model's performance in pixel-level classification, target region overlap, and boundary fitting, this embodiment selects pixel accuracy (PA), rock class intersection-over-union ratio (IoU-rock), average intersection-over-union ratio (MIoU), rock class F1 score (F1-rock), and rock class boundary intersection-over-union ratio (BIoU-rock) as core evaluation metrics. The specific definitions and calculation formulas for these five core evaluation metrics are as follows:

[0130] (1) Pixel accuracy: used to measure the proportion of correctly predicted pixels to the total number of pixels in the image, reflecting the overall classification accuracy.

[0131] ;

[0132] (2) Rock crossover ratio: This measure is used to measure the degree of overlap between the predicted area of ​​the foreground category "rock" and the actual labeled area. It is a key indicator for evaluating the quality of rock segmentation.

[0133] ;

[0134] (3) Average Intersection over Union (IoU): The average IoU of the two categories “rock” and “background” is calculated to evaluate the model’s segmentation performance on the overall scene.

[0135] ;

[0136] (4) Rock F1 score: The precision and recall of rock segmentation are used to reflect the robustness of the model in identifying rock targets.

[0137] ;

[0138] (5) Rock boundary intersection-over-union ratio: used to evaluate the segmentation quality of rock edges. The overlap between the predicted rock boundary and the actual boundary within a certain distance range is calculated to measure the model's ability to capture irregular rock contours.

[0139] ;

[0140] Where: TP represents true positives, TN represents true negatives, FP represents false positives, and FN represents false negatives, which respectively represent the number of classification statistics at the pixel level;

[0141] and These represent the set of boundary pixels of the rock region in the prediction result and the actual label, respectively.

[0142] To further verify the effectiveness of each core module in the proposed framework and its universality under different geological scenarios, Tables 1 to 3 summarize the ablation experiment results of the proposed method on the TWMARS, MarsData-V2, and MSLRock datasets. This experiment employs a rigorous controlled variable method, constructing a baseline model and embedding each core component to explore the contribution logic of each module to performance gains. The experiment focuses on evaluating the joint auditing effect of Multidimensional Uncertainty Evaluation (MADUE) in semantic space, local spatial consistency, and temporal stability, as well as the degree of improvement in pseudo-label quality by the Anomaly Interception Mechanism (ARAR) under "hard correction" and "soft shielding" strategies. Through cross-dataset comparative observation, the ablation experiment quantitatively analyzes the perception accuracy of MADUE for low-quality supervisory signals from multiple dimensions, and how ARAR achieves label self-evolution under the guidance of uncertainty. The necessity of MADUE and ARAR working together is demonstrated, showcasing the strong robustness and generalization potential of the framework in dealing with dramatic changes in Martian surface illumination, dust interference, and heterogeneous targets at different scales.

[0143] Table 1 shows the ablation experiments on the TWMARS dataset.

[0144]

[0145] Table 2 shows the ablation experiments on the MarsData-V2 dataset.

[0146]

[0147] Table 3 shows the ablation experiments on the MSLRock dataset.

[0148]

[0149] To objectively verify the superiority of the proposed method in weakly supervised segmentation tasks, this embodiment selects seven current mainstream graffiti-based weakly supervised semantic segmentation frontier algorithms as comparison benchmarks, specifically including: DBFNet, URSS, TEL, AGMM-SASS, PFA, CC4S, and SOUL. Among them, SOUL is the current best-performing representative method in this field, and this application further introduces a multi-dimensional uncertainty assessment and anomaly correction mechanism on its basis. Tables 4 to 6 summarize and show the quantitative comparative experimental results of the proposed method and the above-mentioned benchmark models on the TWMARS, MarsData-V2, and MSLRock datasets. By comparing the values ​​of each evaluation index horizontally, the significant advantages of this application in improving segmentation accuracy and boundary quality can be intuitively reflected.

[0150] Table 4 shows the comparative experimental results of the TWMARS dataset.

[0151]

[0152] Table 4 presents the quantitative comparative experimental results on the TWMARS dataset. This dataset contains a large number of small and densely distributed rock targets, posing a significant challenge to the boundary capture capabilities of segmentation algorithms. As shown in Table 4, our proposed method (Ours) outperforms the benchmark in all key metrics. Specifically, the IoU-rock metric, reflecting rock segmentation accuracy, reaches 58.90%, a 3.31% improvement over the previous best method, SOUL's 55.59%, and a more significant improvement over AGMM-SASS's 35.81%. In the BIoU-rock metric, which measures boundary fit, our method achieves 43.45%, an improvement of approximately 3.5 percentage points compared to SOUL's 39.97%. This result directly verifies the effectiveness of the MADUE module introduced in our application in handling the problem of blurred edges of small targets. Through rigorous auditing of the consistency of the 8-neighborhood space, it effectively guides ARAR to accurately correct spatially isolated noise points. Furthermore, the F1-rock index improved to 59.43%, and the MIoU reached 78.61%, indicating that the model, driven by the synergy of MADUE and ARAR, achieved a good balance between precision and recall, and can effectively improve the segmentation quality of rock targets.

[0153] Table 5 shows the comparative experimental results of the MarsData-V2 dataset.

[0154]

[0155] Table 5 presents the quantitative comparative experimental results on the MarsData-V2 dataset. This dataset mainly contains large-sized rock targets with clear textures, and the challenge lies in maintaining the consistency of predictions within the rocks. As shown in Table 5, our method (Ours) outperforms the benchmark in all metrics. Specifically, in the IoU-rock metric, which reflects the quality of rock segmentation, our method achieves 76.51%, surpassing the previous high-performing benchmark SOUL's 75.80%. This improvement validates the ARAR introduced in our application, which effectively identifies and corrects class confusion regions within large-sized rocks by extracting current features in real time and performing deep similarity comparison with class prototypes, combined with MADUE, thus improving the consistency of internal predictions. Meanwhile, the MIoU metric, which measures overall segmentation accuracy, reaches 84.69%, and the BIoU-rock metric, which reflects edge precision, reaches 51.81%. This indicates that even for large-scale targets, our method can accurately depict their edge contours, improving upon the common boundary overflow or local under-segmentation problems in traditional methods. Furthermore, the F1-rock score reached 83.24%, further validating the model's robustness in handling rock targets in complex terrain.

[0156] Table 6 shows the comparative experimental results of the MSLRock dataset.

[0157]

[0158] Table 6 presents the quantitative comparative experimental results on the MSLRock dataset. This dataset has extremely complex terrain textures and contains a large amount of sand and shadow interference that resembles the appearance of rocks, making it a highly challenging testing scenario. As shown in Table 6, our proposed method (Ours) demonstrates advantages in all key metrics. In the IoU-rock metric, which reflects the quality of rock segmentation, our method achieves 46.51%, an improvement of nearly 6 percentage points compared to the previous benchmark method SOUL's 40.61% which performed well on this dataset. This improvement validates the effectiveness of the EMA time stability monitoring and dynamic temperature-weighted loss in the MADUE mechanism introduced in our application. By quantifying the cross-cycle consistency of predictions during training iterations, it effectively suppresses prediction oscillations under complex background interference and reduces the misclassification rate caused by high-similarity backgrounds. Meanwhile, the MIoU metric, which measures overall segmentation accuracy, reaches 67.36%, while the BIoU-rock metric, which specifically reflects the edge accuracy of difficult samples, reaches 28.24%, outperforming other mainstream weakly supervised benchmark methods. This demonstrates that even in challenging scenarios with severe texture obfuscation, the proposed MADUE strategy can identify subtle differences between rock edges and background noise. Combined with the ARAR mechanism, it effectively improves upon the edge blurring or semantic alignment failure issues of traditional methods in complex environments. Furthermore, the F1-rock metric reaches 52.82%, and the pixel accuracy PA reaches 89.31%, further confirming the model's robustness and segmentation capabilities in complex geological environments.

[0159] See Figure 8-10This paper presents a visualization comparison of the proposed method and the SOUL weakly supervised segmentation technique in a complex Martian surface scene. The Ground Truth (GT) is a manually annotated reference label map, with the red box highlighting small rocks, shadowed areas, and edge details that are easily missed or misjudged in the Martian surface scene. The comparison results show that, under the complex background interference of Mars, while all the comparative models can identify larger rock bodies, the proposed method (Ours) significantly outperforms SOUL and other comparative methods in terms of the accuracy of rock edge depiction and overall structural integrity. In the red box area, the SOUL model often exhibits segmentation omissions and oversegmentation when facing small targets or the boundaries between rock shadows. In contrast, the proposed method, benefiting from multidimensional uncertainty assessment and anomaly interception and repair mechanisms, can better restore the continuity of fine structures and effectively avoid missed detections. Meanwhile, Martian dust and complex lighting often cause existing methods to generate false targets and produce false detections in non-rocky areas. This application's method effectively suppresses these artifacts by performing consistency audits of the spatial and temporal dimensions, demonstrating stronger generalization ability and robustness in complex scenes. Observing the details within the red box reveals that the mask generated by this application closely matches the ground truth map at the boundaries, verifying the significant improvement in BIoU-rock in the experimental data and solving the boundary overflow or local blurring problems common in traditional methods. In summary, the visualization results intuitively demonstrate the technical superiority of this application in maintaining the authenticity of rock physical distribution and suppressing environmental noise interference.

[0160] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A weakly supervised segmentation method combining multidimensional uncertainty assessment and anomaly correction, characterized in that, include: S1. Image-level multidimensional features are extracted using prompt-free SAM and extremely sparse doodle semantic information is propagated to construct an initial pseudo-label map with physical boundary constraints, which serves as the basic supervision signal for online training; S2. Input the preprocessed image into the ResNet-101+DeepLabV3 network, and output a pixel-level class probability distribution map through multi-scale feature extraction and dilated convolution decoding; S3. By comprehensively calculating pixel semantic similarity, spatial consistency and temporal stability through multidimensional uncertainty assessment, a full-image adaptive uncertainty map is generated in real time. S4. Use the anomaly interception mechanism to compare deep features with class prototypes and associate them with MADUE weights. Obtain refined pseudo-labels through label remapping or confidence masking. S5. Introduce a dynamic temperature coefficient to map uncertainty into pixel-level weights, combine the refined pseudo-labels, prototype constraints and graffiti supervision to construct a weighted loss function, iteratively update the parameters and output a high-precision binarized segmentation map.

2. The method according to claim 1, characterized in that, The method of step S1 includes: S11. Load the pre-trained SAM model and start the automatic mask generator. Traverse the original image and generate a set of image target masks in the prompt-free mode. S12. Extract the spectral mean, gray-level co-occurrence matrix texture and geometric features of each mask region, match the overlap between the mask and the sparse graffiti labels, allocate the optimal category label according to the overlap area to achieve feature propagation, and construct an initial pseudo-label map with physical boundary constraints.

3. The method according to claim 1, characterized in that, The method of step S2 includes: The preprocessed original Mars image is input into a segmentation network with ResNet-101 as the encoder and DeepLabV3 as the decoder. The encoder extracts high-dimensional multi-scale semantic features, and the dilated convolutional layer of the decoder expands the receptive field. During forward propagation, a pixel-level class probability distribution map is output, providing probability input for subsequent uncertainty measurement.

4. The method according to claim 1, characterized in that, The method of step S3 includes: S31. Calculate the original semantic uncertainty based on the cosine similarity between the current features and the semantic prototype; S32. Utilize average pooling operation to detect the distribution difference between pixels in the initial pseudo-label image and the 8-neighbor labels in the initial pseudo-label image, and calculate the spatial consistency uncertainty; S33. Align the current forecast map with the results recorded in the historical forecast buffer by size and class, and calculate the time stability score based on EMA to quantify the time uncertainty; S34. The original semantic uncertainty, the spatial consistency uncertainty, and the temporal uncertainty are weighted and fused to generate a full-image adaptive MADUE uncertainty map, thereby accurately quantifying the noise risk of each pixel in the initial pseudo-label.

5. The method according to claim 4, characterized in that, The method for monitoring and updating the time stability score in step S33 includes: During the model initialization phase, a stability score storage space and a historical prediction storage space are registered. In each training iteration, the current pixel-level prediction result is aligned with the corresponding historical prediction result in size. By comparing the consistency between the two in pixel category, dynamic weight fusion update is performed using an adaptive update rate to maintain the stability score of each image sample in real time. The data loader is simultaneously set to a fixed sequence mode to ensure consistent matching between the image index and the sample information in the storage space.

6. The method according to claim 1, characterized in that, The method of step S4 includes: S41. During the iteration process, start the Anomaly Interception and Repair Pipeline (ARAR) to extract the current predicted features in real time and compare them with the similarity of the rock and background prototypes; S42. If a pixel is classified as rock in the initial pseudo-label, but the similarity between its features and the prototype is lower than a preset threshold, and the spatial uncertainty of the similarity between its features and the prototype in the MADUE uncertainty map is higher than a preset level, then perform "hard correction" to flip it to the background category. S43. If a pixel is the background in the initial pseudo-label, but the feature similarity to the prototype is higher than the preset value, and the total uncertainty of MADUE is higher than the preset level, then perform "soft masking" to set it as an ignored label, thereby dynamically generating high-quality refined pseudo-labels.

7. The method according to claim 1, characterized in that, The method of step S5 includes: S51. Introduce a temperature coefficient t that is dynamically adjusted with the training cycle. By setting different coefficient values, control the smoothness of weight distribution and construct a dynamic temperature-weighted cross-entropy loss. S52. Based on the exponential decay function with the temperature coefficient t as a parameter, the full-image adaptive uncertainty is mapped to pixel-level audit weights, and a multi-dimensional weighted cross-entropy loss is constructed. S53. Based on the effective pixel mask, pixel regions with weights higher than the preset reliability threshold and not ignored labels are selected in real time for gradient aggregation to eliminate the interference of high noise regions on model updates. S54. Input the image to be tested, combine the sparse graffiti supervision loss and the prototype constraint loss to drive the network parameter iteration, and when the performance index of the validation set reaches the preset optimal condition, lock the model weights and output a high-precision binary segmentation map.

8. The method according to claim 7, characterized in that, The method for outputting a high-precision binarized segmentation map in step S54 includes: S541. Input the image to be tested into the pre-saved optimal performance model to perform forward inference and obtain the pixel-level category probability distribution; S542. Based on the category probability distribution, perform confidence normalization and execute a category competition mechanism to extract the category attribute with the highest response value for each pixel; S543. Based on the preset category mapping rules, the pixels are determined to be either target rocks or background areas, and finally a pixel-level binary segmentation image reflecting the physical distribution characteristics of the target to be identified is generated.

Images