Semi-supervised semantic segmentation method and system based on dual-view consistency perception

Abstract

The application discloses a kind of based on dual view angle consistency perception's semi-supervised semantic segmentation method and system, belong to computer vision and semantic segmentation technical field, including: the same structure is constructed teacher and student model, weak enhancement and strong enhancement are respectively applied to training data;Teacher model generates pseudo label based on weak enhancement unlabeled image, student model simultaneously predicts strong enhancement and weak enhancement image.By calculating the consistency difference between pseudo label and student model weak enhancement prediction, the hardness value of each unlabeled sample is obtained, and the unlabeled loss is weighted by this.Finally, the student model is optimized by combining the supervision loss of labeled data and the weighted unlabeled loss, and the teacher model is updated by exponential moving average.Based on the updated student model and teacher model, semantic segmentation is completed.The application realizes more efficient, more stable, more accurate semi-supervised semantic segmentation under the condition of a small amount of labeled samples through the synergistic effect of the above technical means.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and semantic segmentation technology, and particularly relates to a semi-supervised semantic segmentation method and system based on dual-view consistency perception. Background Technology

[0002] Semantic segmentation is a key technology in computer vision, aiming to assign a semantic category label to each pixel in an image, enabling pixel-level understanding of a scene. This technology is widely used in fields such as autonomous driving, medical image analysis, remote sensing image interpretation, and industrial visual inspection, and forms the foundation for many intelligent systems to achieve environmental perception and decision-making.

[0003] Currently, high-performance semantic segmentation models typically rely on large-scale, high-quality pixel-level labeled data for fully supervised training. However, acquiring such labeled data requires significant human, time, and financial resources. For example, on the publicly available Cityscapes dataset, fine-grained annotation of a single image takes an average of over 1.5 hours. In specialized fields such as medical imaging and remote sensing, data annotation relies even more heavily on the knowledge and experience of domain experts, resulting in extremely high annotation thresholds and costs. This severely restricts the application and development of semantic segmentation technology in data-scarce scenarios.

[0004] To alleviate the reliance on labeled data, semi-supervised learning methods have been introduced into semantic segmentation tasks. The core idea of ​​these methods is to train the model using both a small amount of labeled data and a large amount of readily available unlabeled data. Mainstream methods typically employ consistency regularization strategies, which involve applying different perturbations or enhancements to the unlabeled data and constraining the consistency of the model's predictions across different perturbation versions, thereby enabling the model to learn useful feature representations from the unlabeled data.

[0005] Despite this, existing semi-supervised semantic segmentation methods still have significant limitations. First, regarding data augmentation strategies, existing methods often directly adopt strong augmentation schemes designed for supervised learning. Such augmentations may involve fixed combinations of operations or random local perturbations, which can easily lead to excessive image distortion in a semi-supervised setting, disrupting semantic consistency and generating unreliable supervision signals. This limits the effective use of unlabeled data and may even cause model confirmation bias. Second, in terms of loss function design, existing methods generally impose equal weights on all unlabeled samples, ignoring the inherent differences in learning difficulty between different samples. This equalization makes model training susceptible to interference from difficult samples with unstable predictions and high noise levels, leading to fluctuations in the training process and ultimately affecting the model's generalization ability and segmentation accuracy.

[0006] To address the aforementioned problems in existing technologies, there is an urgent need to propose a semi-supervised semantic segmentation method and system based on dual-view consistency perception. Summary of the Invention

[0007] To address the aforementioned technical problems, this invention provides a semi-supervised semantic segmentation method and system based on dual-view consistency perception. This method jointly introduces a mixture of random intensity enhancement and sparse dual-view saliency on unlabeled samples to construct a joint strong data augmentation mechanism with semantic constraints, thereby enhancing sample diversity. Simultaneously, this invention employs a teacher-student model architecture, guiding and constraining the learning process of unlabeled samples by calculating the weighted intersection-union ratio (IUU) between the teacher model's prediction results and the student model's weakly enhanced branch prediction results, thus achieving dynamic modeling of sample reliability. Through these technical means, this invention can effectively mitigate the adverse effects of noisy pseudo-labels, improve the stability and generalization ability of model training, and achieve higher-precision semantic segmentation results.

[0008] This invention proposes a semi-supervised semantic segmentation method based on dual-view consistency awareness, comprising the following steps:

[0009] Obtain a training dataset containing both labeled and unlabeled images;

[0010] Construct teacher and student models with the same network structure and initialize the model parameters;

[0011] Weak data augmentation is applied to both labeled and unlabeled images in the training dataset to obtain corresponding weakly augmented labeled images and weakly augmented unlabeled images;

[0012] The weakly enhanced unlabeled image is subjected to enhanced data augmentation to obtain a strongly enhanced unlabeled image;

[0013] The weakly enhanced unlabeled image is input into the teacher model to obtain the corresponding pseudo-label;

[0014] The strongly enhanced unlabeled image and the weakly enhanced unlabeled image are respectively input into the student model to obtain the first prediction result and the second prediction result, respectively.

[0015] Based on the pseudo-label and the second prediction result, calculate the hardness value of each unlabeled image sample;

[0016] The unlabeled image loss is calculated based on the pseudo-labels and the prediction results of the student model for the strongly enhanced unlabeled image.

[0017] The weighted unlabeled image loss is obtained by weighting the hardness value on the unlabeled image loss.

[0018] The labeled image loss is calculated based on the true labels of the labeled images and the prediction results of the weakly enhanced unlabeled images;

[0019] A total loss is constructed based on the labeled image loss and the weighted unlabeled image loss. The parameters of the student model are updated using the total loss, and the parameters of the teacher model are updated using an exponential moving average.

[0020] Semantic segmentation is completed based on the updated student model and the updated teacher model.

[0021] Optionally, the strong data augmentation includes random intensity augmentation and sparse dual-view saliency hybrid augmentation;

[0022] The strong data augmentation includes: performing two independent random intensity augmentations on the same unlabeled image to generate a first augmented view image and a second augmented view image, and then sending the first augmented view image and the second augmented view image of the unlabeled images in the same batch to the sparse dual-view saliency hybrid augmentation for parallel processing.

[0023] The sparse dual-view saliency blending enhancement includes: for images in a batch, performing saliency analysis on a portion of the sequentially input first enhanced view images through a first view enhancement branch to obtain a first saliency region; performing saliency analysis on a portion of the sequentially input second enhanced view images through a second view enhancement branch to obtain a second saliency region; and blending the first enhanced view images and the second enhanced view images according to the first saliency region and the second saliency region to generate a final strongly enhanced image.

[0024] Optionally, the first enhanced viewpoint image and the second enhanced viewpoint image are blended based on the first salience region and the second salience region, specifically as follows:

[0025] Saliency analysis is performed on the odd-order images in the first enhanced viewpoint images input sequentially to obtain the first salient region; saliency analysis is performed on the even-order images in the second enhanced viewpoint images input sequentially to obtain the second salient region;

[0026] Extract the image patch corresponding to the first salient region from the first enhanced viewpoint image in the current order, and superimpose it onto the corresponding position of the first salient region in the second enhanced viewpoint image in the current order to generate a first strongly enhanced image;

[0027] Extract the image patch corresponding to the second salient region from the second enhanced viewpoint image in the current order, and superimpose it onto the corresponding position of the second salient region in the second enhanced viewpoint image in the next order to generate a second strongly enhanced image;

[0028] The first strongly enhanced image and the second strongly enhanced image constitute the final strongly enhanced image.

[0029] Optionally, the hardness value for each unlabeled image sample is calculated, including:

[0030] The proportion of pixels whose maximum predicted probability exceeds a preset confidence threshold in the total image pixels is taken as the first high confidence proportion.

[0031] The proportion of pixels whose predicted probability exceeds the preset confidence threshold in the prediction results of the student model is taken as the second highest confidence proportion.

[0032] The category-weighted intersection-union ratio (IUU) between the teacher model prediction results and the student model prediction results is calculated. The category-weighted IUU is obtained by weighting the IUU of each semantic category by summing the IUU of each category.

[0033] The hardness value is calculated using a symmetric evaluation function based on the first high confidence ratio, the second high confidence ratio, and the category-weighted crossover ratio.

[0034] Optionally, the hardness value is used to weight the loss of the unlabeled image, specifically as follows:

[0035] Based on the calculated hardness value of each unlabeled image sample, obtain the corresponding weight coefficient;

[0036] Based on the weighting coefficients, the loss of the unlabeled image is scaled to obtain a weighted loss value.

[0037] Optionally, both the unlabeled image loss and the labeled image loss are calculated using the cross-entropy loss function.

[0038] Optionally, the total loss is a weighted sum of the labeled image loss and the weighted unlabeled image loss, wherein the weight of the weighted unlabeled image loss in the total loss is adjusted by a hyperparameter.

[0039] Optionally, the parameters of the teacher model are updated using an exponential moving average method, specifically:

[0040] Based on the latest parameters of the student model and the current parameters of the teacher model, as well as the preset momentum coefficient, the update amount of the teacher model parameters is calculated.

[0041] Based on the update amount, adjust the parameters of the teacher model to obtain the updated teacher model.

[0042] This invention also proposes a semi-supervised semantic segmentation system based on dual-view consistency awareness, used to implement the aforementioned semi-supervised semantic segmentation method based on dual-view consistency awareness, comprising:

[0043] The data acquisition module is used to acquire a training dataset containing both labeled and unlabeled images;

[0044] The model building and initialization module is used to build teacher and student models with the same network structure and initialize model parameters;

[0045] The data augmentation module is used to apply weak data augmentation to both labeled and unlabeled images in the training dataset to obtain corresponding weakly augmented labeled images and weakly augmented unlabeled images; and to apply strong data augmentation to the weakly augmented unlabeled images to obtain strongly augmented unlabeled images.

[0046] The pseudo-label generation module is used to input the weakly enhanced unlabeled image into the teacher model and generate pseudo-labels;

[0047] The model prediction module is used to input the strongly enhanced unlabeled image and the weakly enhanced unlabeled image into the student model respectively, and obtain a first prediction result and a second prediction result respectively;

[0048] The hardness assessment module is used to calculate the hardness value of each unlabeled image sample based on the pseudo-label and the second prediction result.

[0049] The loss calculation and weighting module is used to calculate the loss of labeled images and the loss of unlabeled images, and to weight the loss of unlabeled images using the hardness value;

[0050] The model update module is used to update the student model parameters based on the total loss and update the teacher model parameters using an exponential moving average.

[0051] The semantic segmentation module is used to perform semantic segmentation based on the updated student model and the updated teacher model.

[0052] The present invention also proposes an electronic device, including a memory, a processor, and a computing program stored in the memory and executable on the processor, wherein the processor is configured to implement the semi-supervised semantic segmentation method based on dual-view consistency awareness when executing the computing program.

[0053] Compared with the prior art, the present invention has the following advantages and technical effects:

[0054] This invention constructs a dual-view consistency learning framework. This approach introduces rich and diverse semantic-preserving perturbations through strong augmentation to expand the effective training samples, while providing stable prediction references through weak augmentation. This enables the model to learn more robust and generalized feature representations from different augmentation perspectives of the same unlabeled sample, thereby effectively mitigating the problem of pseudo-label noise caused by single or excessive data augmentation and improving the efficiency of utilizing unlabeled data.

[0055] This invention achieves a precise quantitative assessment of the learning difficulty of samples by dynamically calculating the hardness value of each unlabeled sample based on the pseudo-labels generated by the teacher model and the prediction results of the student model for weakly augmented views. This method, by simultaneously considering the confidence level of the pseudo-labels themselves and the consistency between the predictions of the teacher and student models, can automatically and reliably distinguish between easy-to-learn and difficult-to-learn samples, thus providing a precise and adaptive basis for subsequent implementation of differentiated weight allocation and training strategies.

[0056] This invention guides the model to prioritize learning samples with higher-quality pseudo-labels by assigning greater loss weights to high-reliability (low-hardness) samples; simultaneously, it reduces the weights of high-difficulty (high-hardness) samples to suppress the negative impact of noisy pseudo-labels on model optimization. This mechanism significantly improves the stability of the training process and avoids training oscillations or overfitting caused by difficult samples.

[0057] This invention establishes a stable collaborative training and knowledge distillation mechanism. This method utilizes a student model for exploratory learning and parameter updates, while simultaneously generating pseudo-labels using an exponential moving average to obtain a teacher model with smoother updates and more stable representations. This design allows the quality of pseudo-labels to steadily improve during training, providing continuous supervision signals for the student model and forming a virtuous cycle of self-training. Ultimately, this significantly improves the overall segmentation performance and generalization ability of the model under limited labeled data conditions.

[0058] In summary, through the synergistic effect of the above-mentioned technical means, this invention achieves more efficient, more stable, and higher-precision semi-supervised semantic segmentation with a small number of labeled samples. Attached Figure Description

[0059] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0060] Figure 1 A schematic diagram of the overall method flow of the semi-supervised semantic segmentation method based on dual-view consistency awareness provided in an embodiment of the present invention;

[0061] Figure 2This is a schematic diagram of the data augmentation process for unlabeled images provided in an embodiment of the present invention;

[0062] Figure 3 This is a network model system architecture diagram of an embodiment of the semi-supervised semantic segmentation system based on dual-view consistency awareness provided by the present invention. Detailed Implementation

[0063] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0064] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0065] Example 1

[0066] This embodiment provides a semi-supervised semantic segmentation method based on dual-view consistency awareness, including the following steps:

[0067] Obtain a training dataset containing both labeled and unlabeled images;

[0068] Construct teacher and student models with the same network structure and initialize the model parameters;

[0069] Weak data augmentation is applied to both labeled and unlabeled images in the training dataset to obtain corresponding weakly augmented labeled images and weakly augmented unlabeled images;

[0070] The weakly enhanced unlabeled image is subjected to enhanced data augmentation to obtain a strongly enhanced unlabeled image;

[0071] The weakly enhanced unlabeled image is input into the teacher model to obtain the corresponding pseudo-label;

[0072] The strongly enhanced unlabeled image and the weakly enhanced unlabeled image are respectively input into the student model to obtain the first prediction result and the second prediction result, respectively.

[0073] Based on the pseudo-label and the second prediction result, calculate the hardness value of each unlabeled image sample;

[0074] The unlabeled image loss is calculated based on the pseudo-labels and the prediction results of the student model for the strongly enhanced unlabeled image.

[0075] The weighted unlabeled image loss is obtained by weighting the hardness value on the unlabeled image loss.

[0076] The labeled image loss is calculated based on the true labels of the labeled images and the prediction results of the weakly enhanced unlabeled images;

[0077] A total loss is constructed based on the labeled image loss and the weighted unlabeled image loss. The parameters of the student model are updated using the total loss, and the parameters of the teacher model are updated using an exponential moving average.

[0078] Semantic segmentation is completed based on the updated student model and the updated teacher model.

[0079] As a specific implementation method, this invention proposes a semi-supervised semantic segmentation model based on dual-view consistency perception.

[0080] like Figure 1 The diagram shown is a schematic of the overall process of the method in this embodiment. It is used in semi-supervised semantic segmentation tasks to address the problems of insufficient perturbation of unlabeled samples and difficulty in distinguishing sample difficulty differences. It achieves highly reliable pixel-level pseudo-label generation and stable consistency learning under the condition of a small amount of labeled data.

[0081] S1: Regarding model construction, the method proposed in this embodiment adopts a pluggable network architecture design, enabling it to be flexibly integrated into various mainstream convolutional neural network semantic segmentation frameworks. To verify the applicability and effectiveness of the method in the convolutional neural network (CNN) system, this embodiment selects ResNet-101 or ResNet-50 as the encoder network and combines it with DeepLabV3+ as the decoder network to construct a complete semantic segmentation model.

[0082] The encoder section employs a ResNet network structure based on residual learning to extract multi-scale semantic features from the input image layer by layer. Specifically, the input image first undergoes preliminary feature extraction through a convolutional layer with a 7×7 kernel size. This is followed by batch normalization layers and ReLU activation functions to enhance the stability and non-linear expressive power of the feature representation. After initial feature extraction, the feature mapping is encoded sequentially through four stages of residual block structures. Each stage's residual block consists of 1×1, 3×3, and 1×1 convolutional layers in sequence, and the input features are directly introduced to the output through residual connections, forming a residual learning path. This effectively alleviates the gradient vanishing problem that may occur during deep network training and enhances the network's ability to model high-level semantic features. The number of residual blocks and channel size in the four stages are set according to the standard configuration of ResNet-50 or ResNet-101 to ensure the universality and stability of the network structure.

[0083] The decoder employs the decoding structure from the DeepLabV3+ network, performing stepwise spatial resolution recovery on the high-level semantic features output by the encoder. Through multi-layer upsampling operations, high-level features are progressively mapped back to a spatial scale consistent with the input image, thus preserving global semantic information while also considering target boundaries and local details, ultimately outputting pixel-level semantic segmentation results. This decoding structure effectively improves the boundary accuracy and overall consistency of segmentation results in complex scenes.

[0084] To enhance the model's feature representation capabilities and accelerate the convergence speed of the training process, the encoder network is pre-trained on the ImageNet dataset and introduced as initialization parameters. During the model training phase, this embodiment employs the stochastic gradient descent (SGD) algorithm to optimize the model parameters and combines it with a multinomial decay strategy to dynamically adjust the learning rate, with the update form as follows:

[0085] ;

[0086] Where base_lr represents the initial learning rate, iter represents the current iteration number, max_iter represents the maximum number of iterations during training, and power is a hyperparameter controlling the learning rate decay rate. In this embodiment, power is set to 0.9, and the weight decay coefficient is set to 1×102. −4 By combining the SGD optimizer with a multinomial learning rate decay strategy, the training process can be kept stable while avoiding training oscillations caused by excessive learning rates. This provides the model with sufficient optimization time to gradually converge along the direction of loss function descent, thereby achieving better semantic segmentation results.

[0087] Furthermore, in the semi-supervised learning framework of this embodiment, the model consists of a teacher model and a student model. The teacher and student models maintain a consistent network structure, both using ResNet-101 or ResNet-50 as the encoder and DeepLabV3+ as the decoder. During model initialization, the teacher and student models are constructed as independent network instances, but their parameter initialization methods are consistent, meaning both load the same pre-trained parameters as initial weights.

[0088] S2: Before inputting the images from the training dataset into the model for training, this embodiment first performs data augmentation processing on the input images, such as... Figure 2 As shown, by applying data augmentation of different intensities to the same image, the segmentation prediction results under different augmentation perspectives are kept consistent, thereby improving the robustness and generalization ability of the model during training. Specifically, let labeled image samples be... Unlabeled image samples are For labeled images In addition to the unlabeled images input to the teacher model, this embodiment employs a weak data augmentation operator. (⋅) is used to process the data, introducing appropriate perturbations while preserving the original semantic structure of the image as much as possible; for unlabeled images input to the student model, a strong data augmentation operator is used. (⋅) is processed to obtain an enhanced perspective with greater appearance changes. By performing the above random and diverse data augmentation operations on the dataset images, weakly augmented labeled images are obtained respectively. ( Weakly enhanced unlabeled images ( ) and strongly enhanced unlabeled images ( In subsequent training, the weakly enhanced images are mainly used to provide stable supervision and pseudo-supervision signals, while the strongly enhanced images are used to constrain the model's prediction consistency under different enhancement perspectives, thereby prompting the model to learn more robust feature representations.

[0089] Furthermore, the strong data augmentation includes random intensity augmentation and sparse dual-view saliency hybrid augmentation;

[0090] The random intensity enhancement includes: performing random intensity enhancement twice independently on the same unlabeled image to generate a first enhanced viewpoint image and a second enhanced viewpoint image, and then sending the first enhanced viewpoint image and the second enhanced viewpoint image of the unlabeled images in the same batch to the sparse dual-view saliency hybrid enhancement for parallel processing.

[0091] The sparse dual-view saliency blending enhancement includes: for images in a batch, performing saliency analysis on a portion of the sequentially input first enhanced view images through a first view enhancement branch to obtain a first saliency region; performing saliency analysis on a portion of the sequentially input second enhanced view images through a second view enhancement branch to obtain a second saliency region; and blending the first enhanced view images and the second enhanced view images according to the first saliency region and the second saliency region to generate a final strongly enhanced image.

[0092] Furthermore, the first enhanced viewpoint image and the second enhanced viewpoint image are blended based on the first salience region and the second salience region, specifically as follows:

[0093] Saliency analysis is performed on the odd-order images in the sequentially input first enhanced viewpoint images to obtain the first salient region; saliency analysis is performed on the even-order images in the sequentially input second enhanced viewpoint images to obtain the second salient region; an image patch corresponding to the first salient region in the current order of the first enhanced viewpoint image is extracted and superimposed on the corresponding position in the current order of the second enhanced viewpoint image to generate a first strongly enhanced image; an image patch corresponding to the second salient region in the current order of the second enhanced viewpoint image is extracted and superimposed on the corresponding position in the next order of the second enhanced viewpoint image to generate a second strongly enhanced image; the first strongly enhanced image and the second strongly enhanced image constitute the final strongly enhanced image. This superposition method allows for cross-mixing of adjacent images in the original unlabeled dataset, resulting in higher quality image data.

[0094] Specifically, strong enhancement methods mainly include Random Intensity Enhancement (RIA) and Sparse Dual-View Saliency Hybrid Enhancement (SDVS). RIA is implemented by selecting K candidate enhancement operations from a pre-defined random enhancement pool, and randomly selecting [1, K] enhancement operations during training, sequentially applying them to image samples after weak enhancement processing to generate enhanced viewpoints with significant appearance differences. The random enhancement pool includes operations such as automatic contrast adjustment, histogram equalization, Gaussian filtering, contrast adjustment, sharpening, color balance, brightness adjustment, tone dithering, tone separation, and exposure adjustment. Through these methods, the randomness and diversity of image enhancement can be significantly improved while maintaining semantic consistency.

[0095] The implementation process of Sparse Dual-View Saliency Hybridization Enhancement (SDVS) is as follows: First, for the same unlabeled image samples... Two random intensity enhancement RIAs were applied independently to generate two sets of unlabeled images with different enhancement perspectives, denoted as follows:

[0096] ;

[0097] ;

[0098] Subsequently, from the set and set Each batch of samples with a size of B is selected and denoted as:

[0099] ={s1,b∶b∈[1,B]},

[0100] ={s2,b∶b∈[1,B]},

[0101] Furthermore, from At the same time from The set is formed by selecting samples with even indices. Specifically, when b is odd, for Unsupervised saliency analysis is performed on the image samples to generate corresponding saliency maps. The local region with the largest response value is selected from the saliency map, and a corresponding original image patch is obtained by cropping the region based on its spatial location in the original image. Then, this original image patch is pasted into... The corresponding spatial location in the image is used to complete a saliency-guided cross-view blending operation. When b is even, for Unsupervised saliency analysis is performed on the image samples to generate corresponding saliency maps. The local region with the largest response value is selected from the saliency map, and a corresponding original image patch is obtained by cropping the region based on its spatial location in the original image. Then, this original image patch is pasted into... The corresponding spatial locations in the image are used to complete the symmetric saliency-guided blending operation. This process is repeated within a batch until all samples have been blended, ultimately resulting in a set of strongly enhanced images, denoted as . :

[0102] ;

[0103] in, Represents a binary or soft mask generated from a saliency map, used to indicate the location of saliency regions; the symbol ⊙ represents a pixel-by-pixel multiplication operation; Indicates from set The original image patch obtained by cropping. Indicates from set The original image patch obtained by cropping is used. Through the above-mentioned SDVS strong enhancement method, while maintaining the consistency of key semantic regions, it is possible to introduce local structural perturbations across perspectives and samples, thereby further improving the utilization efficiency and robustness of unlabeled data in the semi-supervised training process.

[0104] S3: Based on the weakly enhanced labeled image ( ) and their corresponding real tags Construct a label image loss function And through the label image loss function For student models Supervised training is performed to enable the student model to learn pixel-level semantic information of labeled samples, using the labeled image loss function. The expression is as follows:

[0105]

[0106] Among them, B l |B represents a training batch consisting of labeled images. l | represents the number of labeled images in the training batch; H and W represent the height and width of the input image, respectively, and j represents the pixel position index in the image; Student model The prediction result at pixel position j for the i-th labeled image after weak enhancement processing; This represents the actual label value corresponding to the pixel position j; (⋅) represents the pixel-level cross-entropy loss function.

[0107] S4: During training, through the student model Teacher Model Perform initialization and utilize the teacher model. Predictions are made on the weakly augmented unlabeled images to generate pseudo-labels for training. Specifically, let the unlabeled image samples be... Its weakness is enhanced (⋅) after processing, we obtain ( Input it into the teacher model. Perform forward reasoning to obtain the prediction result. ( ( The prediction result is then used as the pseudo-label for the corresponding unlabeled sample.

[0108] Specifically, in this example, the parameters of the student model are updated via backpropagation, while the parameters of the teacher model do not participate in gradient backpropagation. Instead, the teacher model's parameters are dynamically updated using Exponential Moving Average (EMA). This allows the teacher model to represent the stable evolutionary state of the student model during training, providing a reliable reference for the subsequent generation of pseudo-labels for unlabeled samples and consistency constraints. In the initial training phase, the student model can... Copy the parameters to the teacher model This completes the initialization of the teacher model. Through the above method, the teacher model can be initialized. It possesses feature representation capabilities consistent with the student model from the early stages of training, thus enabling relatively stable processing of unlabeled datasets. Make predictions. Then, use the initialized teacher model. For unlabeled datasets Inference and prediction are made based on the samples in the dataset.

[0109] Specifically, teacher model For each unlabeled image Weak enhancement perspective ( Output the predicted probability distribution of each category at the pixel level, denoted as . ( ( The predicted probability distribution ( ( The labels ()) are considered pseudo-labels for the unlabeled samples and are used to guide the student model during subsequent training. For unlabeled datasets The learning process. Although the pseudo-labels are not real manually labeled labels, they can reflect the semantic category information of pixels in the image to a large extent, thus providing a basis for the student model. It provides approximate supervision signals. With the help of this pseudo-label mechanism, the student model can gradually optimize its parameters with the assistance of unlabeled data, achieve effective learning of unlabeled samples, and further improve the overall semantic segmentation performance.

[0110] By analyzing unlabeled images Perform strong and weak data augmentation to obtain unlabeled images. Strong data augmentation results ( ) and weak data augmentation results ( Using the student model Simultaneously enhance the results of strong data. ( ) and weak data augmentation results ( Predictions were made, and the first prediction results were obtained respectively. ( ( )) and the second prediction result ( ( The strong enhancement branch is used to introduce larger appearance perturbations to improve the robustness of the model, while the weak enhancement branch is used to maintain the main semantic structure of the samples, providing a stable reference for subsequent difficulty assessment and consistency constraints.

[0111] Furthermore, the hardness value of each unlabeled image sample is calculated, including:

[0112] The proportion of pixels whose maximum predicted probability exceeds a preset confidence threshold in the teacher model's prediction results is defined as the first high-confidence proportion. Similarly, the proportion of pixels whose maximum predicted probability exceeds the preset confidence threshold in the student model's prediction results is defined as the second high-confidence proportion. A category-weighted intersection-union ratio (IUU) is calculated between the teacher model's and student model's prediction results. This IUU is obtained by weighting the IUU of each semantic category by summing the IUU of each category. Based on the first high-confidence proportion, the second high-confidence proportion, and the category-weighted IUU, the hardness value is calculated using a symmetric evaluation function.

[0113] Specifically, based on the prediction results of the teacher model Prediction results of the weak processing branch of the student model Calculate the hardness value γ. The specific calculation steps are as follows:

[0114]

[0115] ,

[0116] in, This represents the i-th unlabeled image. This represents the prediction results of the student model on weakly enhanced unlabeled images. This represents the prediction results of the teacher model on weakly enhanced unlabeled images; and These represent the proportions of high-confidence pixels in the student model's prediction results and the teacher model's prediction results, respectively. This represents the threshold used to determine high-confidence predictions.

[0117] To further measure the consistency between the teacher model and the student model in spatial prediction distribution, an intersection-union ratio (IU) metric based on the percentage of pixels belonging to each category is introduced. Its definition is as follows:

[0118]

[0119] in, (⋅,⋅) represents the intersection-union ratio (IUU) weighted by the percentage of pixels in each category, which measures the degree of consistency between the two sets of prediction results in spatial distribution. Indicates mask The set of pixels belonging to category c. This represents the number of pixels in that category. By introducing the category-to-pixel ratio as a weight, the consistency evaluation bias problem caused by the imbalance in category distribution in semantic segmentation tasks can be effectively alleviated, making the consistency measure focus more on the prediction differences between minority and difficult categories.

[0120] Based on the aforementioned high-confidence ratio and class-weighted intersection-union ratio, a sample-level symmetric hardness evaluation function is constructed to quantify the learning difficulty of unlabeled samples. Specifically, the hardness value of the i-th unlabeled sample... Defined as:

[0121]

[0122] in, This represents the hardness value of the i-th unlabeled sample. Its magnitude is positively correlated with the degree of inconsistency between the student model and the teacher model's predictions on that sample; a higher hardness value indicates a more difficult sample. Hardness Value The value range is [0,1], and its magnitude is positively correlated with the degree of inconsistency between the teacher model and the student model's predictions on this sample. The larger the value, the more difficult the sample; conversely, the smaller the value, the easier the sample.

[0123] The specific steps are as follows: For each unlabeled image sample, its weakly enhanced image is input into the student model and the teacher model respectively to obtain the corresponding pixel-level segmentation prediction results. Based on the prediction results, the maximum statistical prediction probability is not less than a preset confidence threshold. The number of pixels is used as the percentage of high-confidence pixels, and this percentage is divided into the total number of pixels in the image. The student model and teacher model correspond to... and This step, by introducing a confidence threshold mechanism, effectively filters out noise interference that may be caused by low-confidence pixels, providing a reliable input basis for subsequent consistency evaluation and ensuring the robustness of the hardness assessment. Based on this, a category-weighted intersection-union ratio (IUU) metric is used to calculate the degree of consistency between the student model's prediction results and the teacher model's prediction results. The category-weighted IUU is obtained by weighting and summing the IUU corresponding to each category using the pixel proportion of each semantic category in the reference mask as weights. Further, based on the proportion of high-confidence pixels and the category-weighted IUU, a symmetric sample hardness assessment function is constructed, which weights and fuses the consistency results from the student model to the teacher model and from the teacher model to the student model, obtaining a sample hardness value ranging from [0,1]. Among them, the hardness value This method is used to characterize the learning difficulty of unlabeled samples. It achieves a dynamic quantitative assessment of the learning difficulty of unlabeled instances, with the hardness value adaptively adjusted according to the model training state, providing an accurate quantitative basis for subsequent adaptive data augmentation and loss weighting.

[0124] The loss function is weighted according to the hardness value γ of each sample. If the hardness value... A larger value indicates that the sample was a difficult sample to learn during the training phase, and its false label has high uncertainty; if the hardness value... A smaller value indicates that the pseudo-label of the sample is relatively stable and reliable, making it an easy-to-learn sample. During the loss function calculation, (1-γ) is used as the weight to weight the consistency loss of each sample, thus assigning higher weights to samples with low hardness values ​​and lower weights to samples with high hardness values. This design allows the training process to prioritize samples with high pseudo-label reliability for model updates, avoiding training instability caused by excessive noise introduced by high-difficulty samples. As model training progresses, the sample hardness value γ is dynamically adjusted according to changes in the model's prediction consistency, allowing unlabeled samples of varying difficulty to gradually participate in model optimization during training.

[0125] S5: Based on pseudo-tags and Constructing an unlabeled image loss function Then, the unlabeled image loss function is constructed by normalizing the hardness value γ (1-γ) and weighting it. And based on the labeled image loss function Weighted unlabeled image loss function Construct a total loss function L, and then further train the student model using the total loss function L. Multiple trained student models were obtained. .

[0126] Calculating the weighted unlabeled image loss function includes the following steps:

[0127] The standard second loss function is obtained through the following operation. :

[0128]

[0129] The weighted unlabeled image loss function is obtained through the following operation. :

[0130] ,

[0131] in, This indicates the batch size of the unlabeled samples; H and W represent the height and width of the unlabeled image, respectively. Let represent the predicted probability distribution of the student model at pixel position j of the i-th unlabeled sample; This represents the predicted probability distribution of the teacher model on the j-th pixel in weakly enhanced unlabeled samples; This represents the pseudo-label of the j-th pixel after the prediction result from the teacher model is further enhanced by SDVS. (⋅) is an indicator function, where the maximum predicted class probability of a pixel is not less than a threshold. The value is 1 if the condition is met, and 0 otherwise. (⋅,⋅) represents the cross-entropy loss function; This represents the hardness value of the i-th unlabeled sample calculated in step S6. A larger value indicates higher prediction uncertainty for that sample; 1- This is used to adaptively weight unlabeled samples of varying difficulty, thereby reducing the negative impact of noisy pseudo-labels in difficult samples on model training.

[0132] S6: Obtain the total loss function L through the following operation:

[0133] ;

[0134] in, This represents the labeled image loss function calculated based on labeled samples; This represents a hyperparameter used to adjust the weight of the unlabeled consistency loss in the total loss function; in this embodiment, it is set to 1.

[0135] On the other hand, this embodiment also provides a semi-supervised semantic segmentation system based on dual-view consistency awareness, used to implement the method, including:

[0136] The data acquisition module is used to acquire a training dataset containing both labeled and unlabeled images;

[0137] The model building and initialization module is used to build teacher and student models with the same network structure and initialize model parameters;

[0138] The data augmentation module is used to apply weak data augmentation to both labeled and unlabeled images in the training dataset to obtain corresponding weakly augmented labeled images and weakly augmented unlabeled images; and to apply strong data augmentation to the weakly augmented unlabeled images to obtain strongly augmented unlabeled images.

[0139] The pseudo-label generation module is used to input the weakly enhanced unlabeled image into the teacher model and generate pseudo-labels;

[0140] The model prediction module is used to input the strongly enhanced unlabeled image and the weakly enhanced unlabeled image into the student model respectively, and obtain a first prediction result and a second prediction result respectively;

[0141] The hardness assessment module is used to calculate the hardness value of each unlabeled image sample based on the pseudo-label and the second prediction result.

[0142] The loss calculation and weighting module is used to calculate the loss of labeled images and the loss of unlabeled images, and to weight the loss of unlabeled images using the hardness value;

[0143] The model update module is used to update the student model parameters based on the total loss and update the teacher model parameters using an exponential moving average.

[0144] The semantic segmentation module is used to perform semantic segmentation based on the updated student model and the updated teacher model.

[0145] In a feasible preferred approach, a model architecture diagram of a semi-supervised semantic segmentation system based on dual-view consistency awareness is shown below. Figure 3 As shown.

[0146] This embodiment provides a semi-supervised semantic segmentation method and system based on dual-view consistency awareness. It can fully mine the potential semantic information in unlabeled data under conditions of limited labeled sample quantity, achieving stable and reliable pixel-level semantic prediction. This method introduces a joint strong data augmentation mechanism with semantic constraints, combined with a consistency learning strategy that adaptively adjusts sample hardness. While ensuring the stability of the training process, it effectively improves the reliability of pseudo-labels and the generalization ability of the model, overcoming the shortcomings of existing semi-supervised segmentation methods in terms of strong augmentation distortion and noisy supervision.

[0147] Based on the above technical solution, this embodiment can significantly reduce the dependence on large-scale, high-precision pixel-level labeled data in practical applications, and is especially suitable for application scenarios with high labeling costs and difficult data acquisition. This method can serve as a general-purpose semi-supervised semantic segmentation engine, flexibly integrated into existing deep learning training frameworks, and used in fields such as autonomous driving perception systems, intelligent medical image analysis, remote sensing image interpretation, and industrial visual inspection, achieving high-precision segmentation modeling under limited labeling conditions.

[0148] Example 2

[0149] This embodiment is based on the semi-supervised semantic segmentation model described in Embodiment 1, and its effectiveness is verified on a publicly available benchmark dataset. Addressing the issues of insufficient perturbation of unlabeled samples and difficulty in distinguishing sample difficulty differences under limited labeled data conditions, the model achieves highly reliable pixel-level pseudo-label generation and stable consistency learning through a teacher-student collaborative training mechanism.

[0150] This embodiment selects the PASCAL VOC 2012 and Cityscapes datasets as validation objects. The PASCAL VOC 2012 dataset contains 21 semantic categories and is trained using a hybrid training set with SBD extended annotations. The Cityscapes dataset contains 19 city street view semantic categories and provides finely annotated training and validation sets. Corresponding training parameters are set for each dataset. For the PASCAL VOC 2012 dataset, the initial learning rate is 0.001, the crop size is 513×513, the batch size is 8, and the training epochs are 80. For the Cityscapes dataset, the initial learning rate is 0.01, the crop size is 800×800, the batch size is also 8, and the training epochs are 240. All training and inference processes are performed using the PyTorch deep learning framework.

[0151] During the model evaluation phase, the trained student models are applied to test set images for inference and prediction, and the mean Intersection over Union (mIoU) is used as the performance evaluation metric. mIoU is obtained by calculating the intersection over union ratio between the predicted results and the ground truth annotations for each semantic category and averaging the results. It is used to comprehensively measure the overall performance of the model in multi-class semantic segmentation tasks. Based on this metric, the student models with the best segmentation performance during training are selected as the final models.

[0152] To further evaluate the model's segmentation capabilities in complex scenes, especially on high-resolution urban street view datasets such as Cityscapes, this embodiment introduces a sliding window evaluation strategy during the testing phase. By performing block-based inference on the test image and gradually covering the entire image, the segmentation performance of different local regions can be obtained, thus more accurately reflecting the model's segmentation effect in small objects and complex structural regions. Experimental results show that the model exhibits good convergence stability and segmentation performance on different datasets, and can effectively improve semantic segmentation performance under conditions with few annotations.

[0153] This embodiment relates to a model for semi-supervised semantic segmentation. Under conditions of limited labeled data, this model addresses the problems of insufficient perturbation in unlabeled samples and difficulty in distinguishing sample difficulty by employing a teacher-student collaborative training mechanism to achieve highly reliable pixel-level pseudo-label generation and difficulty-aware consistency learning. During training, unlabeled samples are weakly augmented, and the teacher model generates pseudo-labels. The student model then learns from these augmented samples to improve the model's adaptability to complex perturbations. Simultaneously, a difficulty-aware consistency loss weight adjustment mechanism is introduced to adaptively train on unlabeled samples of varying difficulty, thereby suppressing pseudo-label noise accumulation and improving model training stability and overall segmentation performance. Experimental results show that the model exhibits good convergence characteristics and stability on both the PASCAL VOC and Cityscapes datasets. Table 1 compares the method in this embodiment with the state-of-the-art method on the classic PASCAL VOC dataset under different partitioning protocols. Labeled images are sampled from the original training set, and scores represent the proportion of labeled data used for training. Best performance is highlighted in bold. "SupOnly" indicates a supervised baseline network trained solely on labeled images.

[0154] Table 1

[0155]

[0156] Table 1 compares the method of this embodiment with state-of-the-art methods on the classic PASCALVOC2012 benchmark. On the ResNet-101 backbone, DSC-HCL outperforms the supervised baseline SupOnly by +28.25%, +16.52%, +12.57%, +9.59%, and +4.26% at 1 / 16, 1 / 8, 1 / 4, 1 / 2, and full splits, respectively. Compared to MGCT, DSC-HCL maintains an advantage of +4.03%, +1.17%, +0.94%, +1.81%, and +0.05% across the 1 / 16 to full settings range. On the ResNet-50 backbone, this method again demonstrates continuous improvements over the supervised baseline SupOnly by +17.26%, +18.20%, +13.70%, +10.48%, and +5.64%, showcasing its scalability to lightweight architectures.

[0157] Table 2 compares the method of this embodiment with state-of-the-art methods on the Blender PASCALVOC dataset under different partitioning protocols. Labeled images were sampled from the original training set, and scores represent the proportion of labeled data used for training. Best performance is highlighted in bold. "SupOnly" indicates a supervised baseline network trained solely on labeled images.

[0158] Table 2

[0159]

[0160] Table 2 shows the comparison results on the hybrid PASCALVOC dataset. Among all partitioning protocols, the method proposed in this embodiment outperforms the state-of-the-art methods on the Blender PASCALVOC dataset under different partitioning protocols in most cases. Using the ResNet-101 backbone, it achieves improvements of +8.16%, +4.92%, and +2.62% over the SupOnly baseline at 1 / 16, 1 / 8, and 1 / 4 labeling settings, respectively. Compared to MGCT, this method further achieves gains of +1.55%, +0.83%, and +1.21% at 1 / 16, 1 / 8, and 1 / 4 splits, respectively. A similar trend was observed with the ResNet50 backbone, where the method achieved performance improvements of +10.34%, +6.57%, and +3.47% compared to SupOnly, confirming its effectiveness across different model capacities.

[0161] Table 3 compares the method of this embodiment with state-of-the-art methods for urban landscapes under different zoning protocols. Labeled images were sampled from the original training set, and scores represent the proportion of labeled data used for training. Best performance is highlighted in bold. "SupOnly" indicates a supervised baseline network trained solely on labeled images.

[0162] Table 3

[0163]

[0164] Table 3 shows the performance comparison on the Cityscapes validation set. Using the ResNet-101 backbone, DSC-HCL achieves performance improvements of +7.94%, +4.67%, +3.45%, and +1.60% over the supervised baseline in partitions of 1 / 16, 1 / 8, 1 / 4, and 1 / 2, respectively, and improvements of +1.55%, +1.42%, +1.81%, and None over MGCT under the same protocol. Compared to SupOnly, DSC-HCL also shows consistent performance gains using ResNet-50, with improvements of +10.61%, +6.03%, +4.00%, and +3.99%, further demonstrating its versatility across different network capacities.

[0165] Table 4 shows the ablation experiments conducted on the effectiveness of different modules in this embodiment on the PASCALVOC dataset:

[0166] Table 4

[0167]

[0168] As shown in Table 4, the contribution of each core component in the method of this embodiment to PASCAL VOC 2012 was evaluated. Compared with the Mean Teacher (MT) baseline, the mIoU of Random Intensity-Based Augmentation (RIA), Sparse Dual-View Saliency Mixture (SDVS), and Hardness-Aware Consistency Loss Weights (HCL) were improved by +2.15%, +3.25%, and +0.47%, respectively. Notably, SDVS itself outperformed RIA, demonstrating the benefits of fusing dual augmented views. When all three components were combined, the method of this embodiment achieved its highest performance, confirming the synergistic effect of each augmentation and hardness-weighted mechanism to enhance semi-supervised segmentation.

[0169] The values ​​of the unsupervised loss weight λ and the confidence threshold τ were determined experimentally. In this embodiment, λ=1 was used for all experiments. The confidence threshold was set to τ=0.95 on the PASCAL VOC dataset and τ=0.7 on the Cityscapes dataset. This setting reflects the more complex scenarios in the Cityscapes dataset, requiring a lower confidence threshold to fully utilize the unlabeled samples.

[0170] Table 5 shows the different maximum number of ablation experiments based on random intensity enhancement on the PASCALVOC and Cityscapes datasets.

[0171] Table 5

[0172]

[0173] As shown in Table 5, the maximum number of intensity enhancements k for different strengths were evaluated for the method of this embodiment, where k=0 indicates that no enhancement based on random intensity is applied. The results of this embodiment show that optimal performance is achieved when k=2 due to the random selection mechanism. At the same time, the random intensity enhancement method also outperforms the fixed enhancement method.

[0174] Table 6 shows the ablation experiments on the effects of different enhancement designs on the PASCAL VOC dataset:

[0175] Table 6

[0176]

[0177] As shown in Table 6, the method of this invention evaluates individual and combined strong data augmentations (SDAs), including RandAugment (RA), random intensity-based augmentation (RIA), standard SaliencyMix (SS), and sparse dual-view SaliencyMix (SDVS). SaliencyMix is ​​a method of saliency mixing. Individually, RA and RIA produce modest improvements relative to the MT baseline, while SS and SDVS provide significant gains due to their spatially aware perturbations. Performance is further enhanced when combined with random intensity-based augmentation. In particular, combining RIA and SDVS achieves the highest improvement, indicating that the intensity level and saliency-aware perturbation are complementary. These results highlight the importance of synergistic augmentation design, where integrating different augmentation strategies can significantly enhance semi-supervised semantic segmentation.

[0178] Hard instances are defined as the average hardness calculated for samples with hardness values ​​exceeding the average instance value, while simple instances correspond to the average hardness of samples with hardness values ​​below the average instance value. As model performance improves, the average hardness of unlabeled samples gradually decreases during training. Specifically, simple instances exhibit lower hardness from the beginning, while cases with hardness values ​​fluctuate throughout training but eventually decrease.

[0179] To investigate the impact of strong perturbations on the method of this embodiment, the SDVS method of this invention was compared with three strong perturbation sample generation strategies: standard SaliencyMix (SS), dual-view and same sample SaliencyMix strategy (DSS), and dual-view and cross-sample SaliencyMix (DCS) in additional experiments. Compared with SS, DSS, and DCS, SDVS became the most efficient variant. The inclusion of SDVS resulted in the highest mIoU gain, highlighting its advantages in enhancing semantic consistency and cross-domain generalization, which illustrates the importance of appropriate perturbations in consistent regularization-based SSL.

[0180] Table 7 shows the ablation studies of EMA transfer weight on the PASCAL VOC dataset:

[0181] Table 7

[0182]

[0183] Table 7 shows the segmentation performance of the EMA with respect to different momentum parameters for the proposed framework. The results indicate that the setting of α=0.999 yields the best results.

[0184] This embodiment is visualized on the PASCAL VOC 2012 dataset and compared with baseline methods and other state-of-the-art methods. Based on the comparison results on the PASCAL VOC 2012 dataset, the method in this embodiment performs exceptionally well in object category recognition, boundary segmentation, and detail feature preservation, outperforming baseline methods and other state-of-the-art methods in most cases. The segmentation results are more accurate, especially in complex scenes, such as those containing animals or people, where it better preserves the integrity of objects and improves the clarity of segmented regions, significantly reducing background noise interference. Baseline methods, on the other hand, often suffer from blurred boundaries in complex scenes, particularly in the detailed segmentation of people or animals.

[0185] This embodiment visualizes the data on the Cityscapes dataset and compares it with baseline methods and other state-of-the-art methods. The comparison results show that the method in this embodiment further demonstrates its superior segmentation performance under complex conditions on the Cityscapes dataset. While baseline methods and other state-of-the-art methods can identify major objects in a scene, they are insufficient in detail processing, such as segmenting human outlines and small objects, resulting in unclear boundaries, especially in scenes with multiple objects or complex backgrounds. Compared to baseline methods and other state-of-the-art methods, the method in this embodiment shows higher segmentation accuracy in all images. It provides clearer boundary delineation between people, vehicles, and environmental elements such as roads and buildings, avoiding the blurring and overlap problems common in baseline methods in complex scenes. The method in this embodiment demonstrates significant advantages in improving model generalization ability, detail fidelity, and segmentation accuracy in complex scenes, providing a new solution for semi-supervised semantic segmentation tasks.

[0186] Example 3

[0187] This embodiment also provides an electronic device, including a memory, a processor, and a computing program stored in the memory and executable on the processor, wherein the processor implements the method when executing the computing program.

[0188] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A semi-supervised semantic segmentation method based on dual-view consistency perception, characterized in that, Includes the following steps: Obtain a training dataset containing both labeled and unlabeled images; Construct teacher and student models with the same network structure and initialize the model parameters; Weak data augmentation is applied to both labeled and unlabeled images in the training dataset to obtain corresponding weakly augmented labeled images and weakly augmented unlabeled images; The weakly enhanced unlabeled image is subjected to enhanced data augmentation to obtain a strongly enhanced unlabeled image; The weakly enhanced unlabeled image is input into the teacher model to obtain the corresponding pseudo-label; The strongly enhanced unlabeled image and the weakly enhanced unlabeled image are respectively input into the student model to obtain the first prediction result and the second prediction result, respectively. Based on the pseudo-label and the second prediction result, calculate the hardness value of each unlabeled image sample; The unlabeled image loss is calculated based on the pseudo-labels and the prediction results of the student model for the strongly enhanced unlabeled image. The weighted unlabeled image loss is obtained by weighting the hardness value on the unlabeled image loss. The labeled image loss is calculated based on the true labels of the labeled images and the prediction results of the weakly enhanced unlabeled images; A total loss is constructed based on the labeled image loss and the weighted unlabeled image loss. The parameters of the student model are updated using the total loss, and the parameters of the teacher model are updated using an exponential moving average. Semantic segmentation is performed based on the updated student model and the updated teacher model. The strong data augmentation includes random strength augmentation and sparse dual-view saliency hybrid augmentation; The random intensity enhancement includes: performing random intensity enhancement twice independently on the same unlabeled image to generate a first enhanced viewpoint image and a second enhanced viewpoint image, and sending the first enhanced viewpoint image and the second enhanced viewpoint image of the unlabeled images in the same batch to the sparse dual-view saliency hybrid enhancement for parallel processing. The sparse dual-view saliency blending enhancement includes: for images within a batch, performing saliency analysis on a portion of the sequentially input first enhanced view images through a first view enhancement branch to obtain a first saliency region; performing saliency analysis on a portion of the sequentially input second enhanced view images through a second view enhancement branch to obtain a second saliency region; and blending the first enhanced view images and the second enhanced view images based on the first and second saliency regions to generate the final strongly enhanced image. The first enhanced viewpoint image and the second enhanced viewpoint image are blended based on the first salience region and the second salience region, specifically as follows: Saliency analysis is performed on the odd-order images in the first enhanced viewpoint images input sequentially to obtain the first salient region; saliency analysis is performed on the even-order images in the second enhanced viewpoint images input sequentially to obtain the second salient region; Extract the image patch corresponding to the first salient region from the first enhanced viewpoint image in the current order, and superimpose it onto the corresponding position of the first salient region in the second enhanced viewpoint image in the current order to generate a first strongly enhanced image; Extract the image patch corresponding to the second salient region from the second enhanced viewpoint image in the current order, and superimpose it onto the corresponding position of the second salient region in the second enhanced viewpoint image in the next order to generate a second strongly enhanced image; The first strongly enhanced image and the second strongly enhanced image constitute the final strongly enhanced image.

2. The semi-supervised semantic segmentation method based on dual-view consistency perception according to claim 1, characterized in that, Calculate the hardness value for each unlabeled image sample, including: In the prediction results of the statistical teacher model, the proportion of pixels whose predicted probability exceeds the preset confidence threshold to the total number of image pixels is taken as the first high confidence proportion. In the prediction results of the statistical student model, the proportion of pixels whose maximum predicted probability exceeds the preset confidence threshold to the total number of image pixels is taken as the second highest confidence proportion. The category-weighted intersection-union ratio (IUU) between the teacher model prediction results and the student model prediction results is calculated. The category-weighted IUU is obtained by weighting the IUU of each semantic category by the pixel proportion of each category. The hardness value is calculated using a symmetric evaluation function based on the first high confidence ratio, the second high confidence ratio, and the category-weighted crossover ratio.

3. The semi-supervised semantic segmentation method based on dual-view consistency perception according to claim 1, characterized in that, The loss of the unlabeled image is weighted using the hardness value, specifically as follows: Based on the calculated hardness value of each unlabeled image sample, obtain the corresponding weight coefficient; Based on the weighting coefficients, the loss of the unlabeled image is scaled to obtain a weighted loss value.

4. The semi-supervised semantic segmentation method based on dual-view consistency perception according to claim 1, characterized in that, Both the loss of the unlabeled image and the loss of the labeled image are calculated using the cross-entropy loss function.

5. The semi-supervised semantic segmentation method based on dual-view consistency perception according to claim 1, characterized in that, The total loss is a weighted sum of the labeled image loss and the weighted unlabeled image loss, wherein the weight of the weighted unlabeled image loss in the total loss is adjusted by a hyperparameter.

6. The semi-supervised semantic segmentation method based on dual-view consistency perception according to claim 1, characterized in that, The parameters of the teacher model are updated using an exponential moving average method, specifically as follows: Based on the latest parameters of the student model and the current parameters of the teacher model, as well as the preset momentum coefficient, the update amount of the teacher model parameters is calculated. Based on the update amount, adjust the parameters of the teacher model to obtain the updated teacher model.

7. A semi-supervised semantic segmentation system based on dual-view consistency perception, characterized in that, A method for implementing the semi-supervised semantic segmentation method based on dual-view consistency awareness as described in any one of claims 1-6 includes: The data acquisition module is used to acquire a training dataset containing both labeled and unlabeled images; The model building and initialization module is used to build teacher and student models with the same network structure and initialize model parameters; The data augmentation module is used to apply weak data augmentation to both labeled and unlabeled images in the training dataset to obtain corresponding weakly augmented labeled images and weakly augmented unlabeled images; and to apply strong data augmentation to the weakly augmented unlabeled images to obtain strongly augmented unlabeled images. The pseudo-label generation module is used to input the weakly enhanced unlabeled image into the teacher model and generate pseudo-labels; The model prediction module is used to input the strongly enhanced unlabeled image and the weakly enhanced unlabeled image into the student model respectively, and obtain a first prediction result and a second prediction result respectively; The hardness assessment module is used to calculate the hardness value of each unlabeled image sample based on the pseudo-label and the second prediction result. The loss calculation and weighting module is used to calculate the loss of labeled images and the loss of unlabeled images, and to weight the loss of unlabeled images using the hardness value; The model update module is used to update the student model parameters based on the total loss and update the teacher model parameters using an exponential moving average. The semantic segmentation module is used to perform semantic segmentation based on the updated student model and the updated teacher model.

8. An electronic device comprising a memory, a processor, and a computing program stored in the memory and executable on the processor, characterized in that, The processor is used to implement the semi-supervised semantic segmentation method based on dual-view consistency awareness as described in any one of claims 1-6 when executing the computing program.

Images