Unsupervised domain adaptation medical image segmentation method based on multi-view alignment and pseudo label optimization

Abstract

The application discloses an unsupervised domain adaptation medical image segmentation method based on multi-view alignment and pseudo label optimization, aiming at solving the problems of insufficient segmentation accuracy and low quality of pseudo labels caused by domain shift. The technical scheme is as follows: first, an image-level alignment is performed through a frequency domain smoothing fusion module to generate a class target domain image; a segmentation network is pre-trained using the image to generate initial pseudo labels; then, a two-stage optimization process is used to improve the integrity and structure rationality of the pseudo labels through prototype-based latent foreground completion and structure-awareness enhancement based on SAM; finally, based on the optimized high-quality pseudo labels, a multi-view prototype contrast learning framework is constructed for final feature-level alignment training. The application improves the segmentation accuracy of the model in the unlabeled target domain, and provides an effective solution to the challenge of the scarcity of labeled data in medical image segmentation.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of artificial intelligence, and particularly relates to an unsupervised domain adaptation medical image segmentation method based on multi-view alignment and pseudo-label optimization, which can be used for automatic segmentation of unlabeled target domain medical images. BACKGROUND

[0002] Medical image segmentation is an important task in medical image processing, aiming to accurately divide different tissues or structures in the image to improve the accuracy and efficiency of clinical diagnosis. In recent years, deep learning-based methods have shown good performance in medical image segmentation. However, deep learning models are highly dependent on large-scale, high-quality pixel-level labeled data. In the medical field, obtaining such labeled data is a huge challenge: due to the professional nature of medical image data, labeling labels usually require experts in the field, and the already labeled medical image dataset is very scarce, making it difficult for supervised learning-based medical image segmentation methods to achieve good segmentation results. In order to solve this problem, many studies in recent years have focused on exploring how to improve segmentation performance under unlabeled data conditions.

[0003] Unsupervised domain adaptation (UDA) technology uses labeled source domain data to adapt the model to unlabeled target domain data, reducing the dependence on large-scale labeled data sets, and simultaneously achieving model knowledge transfer between source and target domains with different distributions. However, implementing effective domain adaptation faces a core challenge - domain shift. Due to differences in imaging equipment, scanning parameters, patient population, and other factors, medical images in the source and target domains differ significantly in visual style and data distribution, leading to a sharp decline in model performance when directly transferred.

[0004] Existing UDA methods mainly try to solve the domain shift problem from the following three aspects. First, through image-to-image conversion technology, the style of source domain images is converted to target domain style, making them visually closer to target data. However, this method may introduce unnatural artifacts or damage fine anatomical structures critical to segmentation during the conversion process, resulting in loss of semantic information. Second, by measuring the difference between features in the feature space, the model is forced to learn domain-invariant feature representations. However, this method often performs global macro-alignment, which can easily ignore subtle differences between classes, especially when foreground and background features have high similarity, resulting in poor alignment results. Third, the model generates pseudo-labels for target domain images, and these pseudo-labels are used for supervised training. However, due to domain shift, the quality of the initial pseudo-labels generated by the model on the target domain is usually poor, with problems such as missed foreground targets and incomplete structures. In the subsequent training process, these incorrect supervision signals are learned and accumulated by the model, ultimately affecting the performance ceiling of the model. SUMMARY

[0005] In order to solve the problems of the prior art, the present application provides an unsupervised domain adaptation medical image segmentation method based on multi-view alignment and pseudo label optimization, so as to improve the medical image segmentation precision of the target domain by using an improved image and feature double-layer alignment strategy and a two-stage pseudo label optimization method.

[0006] In order to achieve the above-mentioned purpose, the technical scheme adopted by the present application comprises the following steps:

[0007] An unsupervised domain adaptation medical image segmentation method based on multi-view alignment and pseudo label optimization is used to segment the unlabeled target domain medical image, comprising the following steps:

[0008] S1: Perform an image-level alignment process.

[0009] Through a style transfer module of frequency domain smoothing fusion, the source domain image is converted into a target domain image, so as to realize the image-level alignment of the source domain and the target domain.

[0010] S2: Use the target domain image to perform preliminary model training and prediction.

[0011] The target domain image is input into an image segmentation network, and the source domain real label is used for pre-training; after pre-training, the real target domain image is input, and the initial pseudo label is output.

[0012] S3: Perform a two-stage pseudo label optimization process.

[0013] Through two-stage pseudo label optimization, the initial pseudo label is processed to improve its structural integrity and boundary accuracy, and the optimized pseudo label is obtained.

[0014] S4: Perform feature-level alignment.

[0015] Based on the real label of the source domain and the optimized pseudo label, a multi-view prototype contrast learning framework is constructed, and the image segmentation network is trained by using the framework, and the medical image segmentation is performed by using the image segmentation network obtained by training.

[0016] Compared with the prior art, the present application has the following advantages:

[0017] The present application aims at the problem of lack of high-quality label data for target domain medical images, and provides an unsupervised domain adaptation medical image segmentation method based on multi-view alignment and pseudo label optimization. First, the method solves the domain shift problem through the cooperative alignment strategy of multiple views. At the image level, a style transfer module with frequency domain smoothing fusion is used, which can effectively convert the image style and avoid the high-frequency artifacts caused by hard replacement, and maximize the preservation of the original image structure and semantic details which are crucial for segmentation. At the feature level, a multi-view prototype contrast learning framework containing independent boundary prototypes is constructed. This not only realizes the intra-class compactness and inter-class separation of the main area, but also forces the network to accurately learn and distinguish the ambiguous tissue boundaries through explicit modeling of the boundary area, significantly improving the accuracy of the segmentation result at the fine structure.

[0018] The present application designs a two-stage, from completion to enhancement, pseudo label optimization process, which improves the quality and structural rationality of the pseudo label. In the first stage, the reconstruction-based potential foreground completion mechanism is used to guide the reverse mining of the region misjudged as background. This method actively solves the common missed detection problem in pseudo labels, significantly improving the completeness of pseudo labels. In the second stage, SAM is introduced as a structure-aware enhancement branch, which uses its strong zero-shot segmentation capability to correct the structure of the completed pseudo label. This effectively repairs the common problems of internal cavities and boundary fractures in pseudo labels, ensuring that the pseudo label is not only pixel accurate, but also has anatomical structural rationality. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 is the implementation flowchart of the present application.

[0020] Figure 2 is the structure diagram of the first stage of constructing the pseudo label optimization strategy in the present application, the structure diagram of the reconstruction-based potential foreground completion module.

[0021] Figure 3 is the structure diagram of the second stage of constructing the pseudo label optimization strategy in the present application, the structure diagram of the structure-aware enhancement strategy.

[0022] Figure 4 is the schematic diagram of constructing a multi-view prototype contrast learning framework in the present application.

[0023] Figure 5 is the segmentation visualization result diagram on the ETIS adaptation CVC-EndoScene task.

[0024] Figure 6 is the segmentation visualization result diagram on the ETIS adaptation Kvasir task. DETAILED DESCRIPTION

[0025] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings.

[0026] The method and process are as follows Figure 1 As shown, the implementation steps of this invention are as follows:

[0027] Step 1: Perform image-level alignment between the source domain image and the target domain image.

[0028] In unsupervised domain-adaptive medical image segmentation, the core challenge lies in the domain shift problem caused by factors such as imaging equipment and scanning parameters. This domain shift is not a single-level problem, but exists simultaneously in the low-level visual wind and the high-level feature table of the image. Due to the influence of domain shift, the performance of a model trained on the source domain will drop sharply when directly applied to the target domain.

[0029] To mitigate domain shift, several methods have been developed to transfer image styles using image transformation techniques. However, these methods, while pursuing global distribution alignment, often neglect crucial anatomical structural information in medical images. Specifically, image-level style transfer processes are prone to introducing artifacts, disrupting tissue boundary integrity, and losing fine structures, resulting in segmentation results with blurred boundaries and structural discontinuities in the target domain. Therefore, this invention proposes a frequency domain smooth fusion style transfer module that converts the source domain image into a target domain-like image, achieving image-level alignment between the source and target domains, thereby stably eliminating the visual differences between the two domains. Specifically, this includes:

[0030] (1a) Source domain image I s and target domain image I t The source domain image I is obtained by converting from the spatial domain to the frequency domain using Fast Fourier Transform. s Amplitude spectrum A s and the corresponding phase spectrum P s and target domain image I t Amplitude spectrum A t and the corresponding phase spectrum P t .

[0031] (1b) Design a center-weighted smooth mask M, whose weights decrease smoothly from the low-frequency center to the high-frequency region. The calculation formula is as follows:

[0032]

[0033] Where d(i,j) is the distance from point (i,j) to the center of the mask (c). x ,c y The Euclidean distance of ) d max To start from the center of the mask (c x ,c y) the distance from the mask to the farthest corner, cos(·) is the cosine function.

[0034] (1c) using the smooth mask M to weight the magnitude spectrum of the source domain image I s and the target domain image I t to generate a new magnitude spectrum:

[0035] A new = M o A t + (1-M) o A s

[0036] (1d) combine the new magnitude spectrum A new with the phase spectrum P s of the source domain image I s and map it back to the image space through inverse Fourier transform to obtain the final target-like source image, i.e., the target-like domain image I s→t , as follows:

[0037]

[0038] where, denotes the inverse Fourier transform.

[0039] Step 2, using the target-like domain image to perform preliminary model training and prediction.

[0040] The target-like domain image and its corresponding source domain real label are input into the image segmentation network. The source domain real label is used as supervision to pre-train, and the weighted cross-entropy loss and the weighted intersection-over-union loss are used for supervision. The combination of this loss function is designed to optimize the classification accuracy at the pixel level and the overlap degree of the segmentation region in the overall structure, which is particularly effective for dealing with the common foreground and background class imbalance problem in medical images. Since the target-like domain image is highly similar in style to the real target domain, this pre-training process enables the network to initially adapt to the visual characteristics of the target domain. After pre-training, the real target domain image is input into the network for forward inference, and the output is the initial pseudo label of the target domain.

[0041] The implementation of the image segmentation network and the supervision loss in this step includes:

[0042] (2a) the encoder module is composed of several convolutional blocks, the last convolutional block only contains convolutional operations, and the remaining convolutional blocks each contain convolutional operations and max-pooling operations, which are used to gradually extract deep semantic features of the image.

[0043] For example, in the present embodiment, the encoder module is composed of 4 convolutional blocks, the last convolutional block only contains 2 consecutive 3x3 convolutional operations, and the remaining convolutional blocks each contain 2 consecutive 3x3 convolutional operations and 1 2x2 max-pooling operation.

[0044] Each time a convolutional block is passed, the feature map size is halved and the number of channels is doubled; the source domain image I s→t and the target domain image I t After inputting the encoder, the output is the target domain high-level semantic feature f s and the target domain high-level semantic feature f t .

[0045] (2b) The decoder receives the high-level semantic feature output by the encoder, which is symmetrical in structure and contains several deconvolutional blocks (4 in this embodiment), each of which doubles the feature map size through deconvolutional operation (2x2 deconvolution in this embodiment) and fuses with the feature map of the corresponding level of the encoder through jump connection to supplement the bottom-level detail information.

[0046] (2c) The output layer receives the feature map output by the decoder and maps its channel dimension to the number of categories related to the task through 1x1 convolutional operation. Finally, the class probability of each pixel is output through the softmax function; wherein the class probability of the source domain image is the prediction result y s , and the class probability of the target domain image is the initial pseudo label y t .

[0047] (2d) The loss function in the pre-training stage is the weighted cross-entropy loss and the weighted intersection-over-union loss:

[0048]

[0049] wherein, represents the weighted cross-entropy loss, represents the weighted intersection-over-union loss, N represents the number of pixels, w is the category weight, y i represents the i-th pixel value of the real label, p i represents the i-th pixel value of the prediction probability.

[0050] Step 3, perform a two-stage pseudo label optimization process.

[0051] Since the domain shift problem has not been completely solved at the feature level, the initial pseudo label inevitably has quality problems, especially in the form of missed detection of the foreground area and loss of anatomical structure details. If these low-quality pseudo labels are directly used for subsequent training, it will lead to the accumulation of false signals. Therefore, before the final feature alignment training, these initial pseudo labels must be refined and optimized to improve their structural integrity and boundary accuracy to obtain the optimized pseudo label.

[0052] In this application, a two-stage pseudo label optimization process is proposed. The first stage is the reconstruction-based latent foreground completion, as Figure 2The first stage is the reconstruction-based potential foreground completion, which aims to solve the problem of missing detection of pseudo labels. The core idea is to use the existing and relatively reliable foreground prototype information of the model to review the areas preliminarily judged as background and mine potential foreground that may be misclassified. The second stage is the structure perception-based enhancement. After completion, the integrity of the pseudo label is improved, but the boundary may still be inaccurate, and the overall structure may be unreasonable. To solve this problem, a pre-trained SAM model is introduced as a structure perception branch, which uses the strong zero-shot generalization ability of SAM to generate segmentation hypotheses with fine boundaries and complete structures, as shown in FIG. 6. Through the consistency checking module, high-fidelity results that are consistent with the network's own prediction are filtered and retained as structured supervision signals, and the optimized pseudo labels are finally obtained. Figure 3

[0053] The first stage of the two-stage optimization strategy in S3 is the reconstruction-based potential foreground completion, which includes the following steps:

[0054] (31a) Under the guidance of the initial pseudo label y t of the target domain, the high-level semantic feature f t of the target domain obtained in step (2a) is combined to calculate the class centroid as the initial prototype

[0055]

[0056] wherein, is the total number of pixels of the c-th class in the target domain, H is the height of the feature map, W is the width of the feature map, f t (i) is the i-th pixel value of the high-level semantic feature f t of the target domain, y t (i) is the i-th pixel value of the initial pseudo label y t .

[0057] (31b) Based on the initial pseudo label y t output by the image segmentation network, a background-guided mask M bg is constructed to locate the areas preliminarily judged as background by the model, and the mask is defined as: M bg = 1-argmax(y t ).

[0058] (31c) Feature fusion is performed between the background-guided mask M bg and the high-level semantic feature f t of the target domain to obtain a background-weighted feature representation

[0059]

[0060] wherein,​ Element-wise multiplication.

[0061] (31d) Calculate the similarity between local features of each pixel and prototypes of each class. When the similarity between a certain pixel feature and a foreground prototype exceeds a preset threshold, the pixel is identified as a potential foreground. This process can be formulated as:

[0062]

[0063] where sim(·) is used to calculate the cosine similarity, β is a preset threshold, C is the total number of classes, is the final pixel class determination result.

[0064] (31e) Integrate all pixel class determination results into the initial pseudo-label to correct the misjudged foreground in the original segmentation result, and obtain a coarsely optimized pseudo-label as follows:

[0065]

[0066] where ∪ is the union of two masks.

[0067] Further, the second stage of the two-stage optimization strategy in S3 is a structure-aware enhancement strategy, which includes:

[0068] (32a) Introduce a pre-trained Segment Anything Model (SAM) as a structure-aware branch.

[0069] (32b) From the coarsely optimized pseudo-label with foreground completion, a grid sparse sampling strategy is used to extract a point hint set S, as follows:

[0070] Divide the image into k×k grids, and in each grid, select the center of the grid as a foreground point or a background point according to the proportion of foreground pixels. Collect all sampled points to form a point hint set:

[0071] S = {(i, j, l) | l ∈ {0, 1}}

[0072] where (i, j) represents the pixel coordinates in the foreground region; l = 1 indicates that the coordinate is a foreground point, and l = 0 indicates that the coordinate is a background point.

[0073] (32c) The SAM model generates a segmentation prediction map M sam with complete structure under the guidance of the point hint set S.

[0074] ​(32d) filtering segmentation hypotheses by consistency checking module, retaining those consistent with coarse optimization pseudo-labels forming consensus regions to obtain the optimized pseudo-labels:

[0075]

[0076] wherein, is the optimized pseudo-label, M sam (i) is the i-th pixel value of the segmentation prediction map M sam is the i-th pixel value of the coarse optimization pseudo-label

[0077] (32e) introducing structural loss constraint to optimize the pseudo-labels and the structure predicted by the segmentation network is consistent:

[0078]

[0079] wherein, is the loss function, SSIM(·,·) is the structural loss; λ is the weight coefficient.

[0080] Step 4, constructing a multi-view prototype contrast learning framework to perform feature-level alignment.

[0081] Using the optimized pseudo-labels obtained in step 3, perform the final feature-level alignment training. Existing methods usually rely on global information for construction, and this strategy inevitably dilutes the key detail features of the boundary region, thereby weakening the modeling ability of the model for the boundary region. Therefore, the present application constructs a multi-view prototype contrast learning framework, as shown in Figure 4 Specifically, based on the source domain real label and the target domain optimized pseudo-label, the respective class prototypes are calculated and a hybrid global class prototype is generated. At the same time, to solve the problem that boundary information is easily diluted by main feature, the boundary region is extracted by morphological operation, and an independent boundary prototype is constructed. Finally, through the contrast loss function, the feature representation learned by the network is forced to be able to distinguish different semantic categories and the transition region of the main body and the boundary, so as to realize the compactness of intra-class features, the separability of inter-class features and the explicit enhancement of boundary features.

[0082] The multi-view prototype contrast learning framework implemented in this step includes:

[0083] (4a) under the guidance of the real label of the source domain and the optimized pseudo-label of the target domain, the source domain prototype and the target domain prototype ​​

[0084]

[0085] wherein, is the total number of the c-th class pixels in the source domain; is the total number of the c-th class pixels in the target domain, f s (i) is the i-th pixel value of the high-level semantic feature f s (i) is the i-th pixel value of the high-level semantic feature f t (i) is the i-th pixel value of the high-level semantic feature f t (i) is the i-th pixel value of the high-level semantic feature f is the i-th pixel value of the real label f is the i-th pixel value of the real label f is the i-th pixel value of the optimized pseudo label f is the i-th pixel value of the optimized pseudo label f

[0086] (4b) performing dynamic iterative mixed updating on the source domain prototype f and the target domain prototype f to generate the mixed domain prototype of the current iteration:

[0087]

[0088] wherein, a is a weighting coefficient, used to adjust the contribution degree of the source domain prototype and the target domain prototype to the updating of the global prototype, is the mixed domain prototype of the t-th iteration, is the mixed domain prototype of the t-1-th iteration, is the source domain prototype of the t-th iteration, is the target domain prototype of the t-th iteration.

[0089] (4c) constructing a contrastive learning loss, which aims to pull the distance between the global pixel feature output by the network and the mixed domain prototype of its corresponding category, while pushing away the distance between the global pixel feature output by the network and the mixed domain prototypes of other categories, represented as:

[0090]

[0091] wherein, exp(·) is an exponential function, used to convert the similarity into a probability distribution, sim(·) is a cosine similarity, used to calculate the similarity between feature vectors, and τ is a temperature parameter, used to smooth the probability distribution, f i is the i-th pixel value of the semantic feature of the source domain or the target domain.

[0092] Further, the multi-view prototype contrastive learning framework in S4 further comprises:

[0093] (5a) extracting boundary region masks B from the real label of the source domain and the optimized pseudo label respectively through morphological operations:

[0094] B = dilate(y, k) - erode(y, k)

[0095] wherein dilate(y, k) is a dilation operation with kernel size k, and erode(y, k) is an erosion operation with kernel size k.

[0096] (5b) Calculate the centroid of the pixel features in the boundary region to construct an independent boundary prototype

[0097]

[0098] wherein, is the total number of the c-th class of pixels in the source domain, and f i is the i-th pixel value of the semantic feature of the source domain or the target domain; B i is the i-th pixel value of the boundary mask.

[0099] (5c) Add the boundary prototype to the contrast learning process to enhance the modeling ability of the model for the fine structure of the boundary, which is a supplement to the global contrast learning, and reuse the contrast learning loss function form in step (4c), only replace the input prototype with the boundary prototype.

[0100] In order to show the advantages of the present application, the specific implementation effect can be further illustrated by the following simulation results:

[0101] I. Simulation conditions

[0102] The segmentation method designed in the simulation process is based on Python 3.10 version and Pytorch 1.11 version framework; the machine used to train the segmentation network in the simulation process is equipped with Intel(R) Core(TM), 2.5GHz CPU, 24GB RAM and 1 GeForce RTX 3090 GPU;

[0103] The simulation uses the polyp image and label data set obtained from the public ETIS, CVC-EndoScene, Kvasir data set.

[0104] II. Simulation content

[0105] ​Under the above simulation conditions, the polyp images obtained are segmented by the method of the application and the label data set, and the segmentation evaluation indexes are Dice coefficient and mean intersection over union (mIoU). The Dice coefficient is a region overlap-based metric used to quantify the similarity between the model's predicted segmentation results and the true labels. The higher the value of this index, the more accurate the segmentation result; the mean intersection over union is also an index for measuring the degree of region overlap, which calculates the ratio of the intersection and union of the predicted region and the true region. The value of this index ranges from 0 to 1, and the higher the value, the better the performance.

[0106] Tables 1 and 2 respectively show the performance comparison of the method of the application and the current mainstream unsupervised domain adaptation medical image segmentation method on the ETIS adaptation CVC-EndoScene task and the ETIS adaptation Kvasir task.

[0107] Table 1 Performance comparison of different UDA methods on the ETIS adaptation CVC-EndoScene task

[0108] Model Dice (%)↑ mIoU (%)↑ No domain adaptation 61.87 72.21 FDA 67.98 75.44 MPA-DA 72.08 75.33 GFDA 75.13 80.76 DCLPS 80.02 83.25 MLDB 81.46 ]]> ​ 84.23 ]]> ​ Ours 82.14 84.50

[0109] Table 2 Performance comparison of different UDA methods on the ETIS adaptation Kvasir task

[0110]

[0111]

[0112] As shown in Tables 1 and 2, the application achieves the best results in each performance indicator on two different domain adaptation tasks. Specifically, in the ETIS to CVC-EndoScene adaptation task, the Dice coefficient of the application reaches 82.14%, which is about 0.7% higher than the suboptimal method. In the ETIS to Kvasir adaptation task, the Dice coefficient of the application reaches 83.16%, which is about 2% higher than the suboptimal method. This improvement indicates that the application has a clear advantage in the overall segmentation accuracy of the target region, and can more accurately identify the decision boundary between the foreground and the background. In addition, in the mIoU index of the two tasks, the application also shows a comprehensive performance advantage.

[0113] In order to more intuitively show the performance advantage of the application, Figure 5 and Figure 6 the visualization results on the polyp segmentation data set are presented. In these segmentation visualization comparisons, the application shows excellent results. Specifically, in the ETIS to CVC-EndoScene adaptation task, Figure 5 and Figure 6In the middle, the foreground target size is large, and the color of the foreground region is similar to the surrounding background. In this scenario, most contrast methods are difficult to accurately define the target boundary, not only the under-segmentation situation, but also the background region is incorrectly identified as foreground. In contrast, the present application can accurately capture the overall profile, and no false segmentation results are produced.

Claims

1. An unsupervised domain-adaptive medical image segmentation method based on multi-view alignment and pseudo-label optimization, used for segmenting unlabeled target domain medical images, characterized in that... Includes the following steps: S1: The style transfer module, which uses frequency domain smooth fusion, converts the source domain image into a target domain-like image, achieving image-level alignment between the source and target domains; S2: Input the target domain image into the image segmentation network and perform pre-training with the source domain real labels as supervision; after pre-training is completed, input the real target domain image and output the initial pseudo labels; S3: Perform a two-stage pseudo-label optimization process to process the initial pseudo-labels to improve their structural integrity and boundary accuracy, and obtain optimized pseudo-labels; S4: Perform feature-level alignment. Based on the true labels of the source domain and the optimized pseudo labels, construct a multi-view prototype contrastive learning framework. Specifically, based on the true labels of the source domain and the optimized pseudo labels of the target domain, calculate their respective category prototypes and fuse them to generate a hybrid global category prototype. At the same time, extract boundary regions through morphological operations and construct independent boundary prototypes. Use this framework to train the image segmentation network and perform medical image segmentation using the trained image segmentation network. The first stage of the two-stage optimization strategy in S3 is based on reconstruction-based potential prospect completion, including: (31a) In the initial pseudo-label Under the guidance of the algorithm, the centroid of the class is calculated as the initial prototype. : in, For the target domain The total number of pixels of each class For feature map height, The width of the feature map. High-level semantic features of the target domain The pixel value, For initial pseudo-tags The Each pixel value; (31b) Initial pseudo-labels based on the output of the image segmentation network Construct a background boot mask The area initially determined by the positioning model, among which ; (31c) Background guiding mask High-level semantic features of the target domain Feature fusion is performed to obtain a background-weighted feature representation. : in, For element-wise multiplication; (31d) Calculation Local features and various prototypes The similarity between pixels is determined by a preset threshold; when the similarity between a pixel feature and a foreground prototype exceeds a certain threshold, the pixel is identified as a potential foreground. The formula is as follows: in, Used to calculate cosine similarity For a preset threshold, C It is the total number of categories. This is the result of classifying all pixels; (31e) Determine the category of all pixels These are integrated into the initial pseudo-labels to correct misclassified foreground elements in the original segmentation results, resulting in coarsely optimized pseudo-labels. ,as follows: in, To obtain the union of the two masks; The second stage of the two-stage optimization strategy in S3 is a structure-aware enhancement strategy, including: (32a) Introduce a pre-trained segmentation model as a structure-aware branch; (32b) From the coarse-optimized pseudo-labels after foreground completion In the middle, a grid sparse sampling strategy is used to extract the point cue set. ; (32c) Segment all models in the point hint set Guided by this, a segmentation prediction map with structural integrity is generated. ; (32d) Filter the segmentation hypothesis through the consistency verification module, retaining the pseudo-labels from the coarse optimization. The region where consensus is reached is used to obtain the optimized pseudo-label: In the formula, For optimized pseudo-tags, For segmentation prediction map The pixel value, To coarsely optimize pseudo tags The Each pixel value; (32e) Pseudo-labels optimized by introducing structural loss constraints Structure and consistency with segmentation network predictions: in, For loss function, For structural loss; These are the weighting coefficients.

2. The method as described in claim 1, characterized in that, In S1, the processing procedure of the style transfer module for frequency domain smooth fusion includes: (1a) Source domain image and target domain image The source domain image is obtained by converting it from the spatial domain to the frequency domain using Fast Fourier Transform. amplitude spectrum and the corresponding phase spectrum and target domain image amplitude spectrum and the corresponding phase spectrum ; (1b) Design center-weighted smoothing mask Its weight decreases smoothly from the low-frequency center to the high-frequency region, and the calculation formula is: in, For point To the center of the mask Euclidean distance, , To from the center of the mask Distance to the farthest point of the mask It is a cosine function; (1c) Utilizing the smoothing mask For source domain images and target domain image The amplitude spectra are weighted and fused to generate a new amplitude spectrum: (1d) The new amplitude spectrum With source domain image phase spectrum By combining these images and mapping them back to the image space using inverse Fourier transform, we obtain the final source image resembling the target, i.e., the target-domain image. The formula is as follows: in, This represents the inverse Fourier transform.

3. The method as described in claim 1, characterized in that, The image segmentation network in S2 adopts an encoder-decoder architecture, wherein: (2a) The encoder module consists of several convolutional blocks. The last convolutional block contains only convolution operations, while each of the remaining convolutional blocks contains both convolution and max-pooling operations to progressively extract deep semantic features from the image. After each convolutional block, the feature map size is halved and the number of channels is doubled; class-object domain image. and target domain image After inputting into the encoder, the output is the high-level semantic features of the target domain. and target domain high-level semantic features ; (2b) The decoder receives the features output by the encoder. Its structure is symmetrical with the encoder and contains several deconvolution blocks. Each deconvolution block doubles the size of the feature map through deconvolution operation and fuses it with the feature map of the corresponding level of the encoder through skip connections to supplement the low-level detailed information. (2c) The output layer receives the feature map output by the decoder and uses a 1×1 convolution operation to map its channel dimensions to the number of task-related classes. Finally, it outputs the class probability of each pixel through the softmax function; where the class probability of the target domain image is used as the prediction result. The class probability of the target domain image is used as the initial pseudo-label. .

4. The method as described in claim 3, characterized in that, The encoder module consists of four convolutional blocks. The last convolutional block contains only two consecutive 3×3 convolutional operations, while each of the remaining convolutional blocks contains two consecutive 3×3 convolutional operations and one 2×2 max pooling operation.

5. The method as described in claim 1, 3, or 4, characterized in that, The pre-training process employs a loss function that combines weighted cross-entropy loss and weighted intersection-over-union (IoU) loss. in, This represents the weighted cross-entropy loss. This indicates the weighted average loss. Indicates the number of pixels. For category weights, The first one representing the true label pixel value, The first digit representing the predicted probability Each pixel value.

6. The method as described in claim 1, characterized in that, The point cue set is extracted using a sparse grid sampling strategy. The implementation method is as follows: Divide the image into sizes. The grid is used to select the center of each grid as the foreground or background point based on the proportion of foreground pixels. All sampled points are then combined to form a cue set. in, Represents the pixel coordinates within the foreground region; This indicates that the coordinates represent the foreground point. This indicates that the coordinates are the background point.

7. The method as described in claim 1, characterized in that, The multi-view prototype contrastive learning framework in S4 includes: (4a) Real labels in the source domain Pseudo-tags optimized for the target domain Under the guidance of the relevant authorities, source domain prototypes were generated respectively. and target domain prototype : in, For the source domain The total number of pixels of each class; For the target domain The total number of pixels of each class The first high-level semantic feature of the target domain pixel value, High-level semantic features of the target domain The pixel value, For source domain real tags The pixel value, Pseudo-tags optimized for the target domain The Each pixel value; (4b) For the source domain prototype and target domain prototype Perform dynamic iterative hybrid updates to generate the hybrid domain prototype for the current iteration: in, These are weighting coefficients used to adjust the contribution of the source domain prototype and the target domain prototype to the global prototype update. For the first The prototype of the hybrid domain in the next iteration. For the first The prototype of the hybrid domain in the next iteration. For the first The source domain prototype of the next iteration. For the first The target domain prototype for the next iteration; (4c) Construct a contrastive learning loss, the goal of which is to bring the global pixel features output by the network closer to the mixed domain prototypes of their corresponding classes, while increasing their distance from the mixed domain prototypes of other classes, expressed as: in, It is an exponential function used to convert similarity into a probability distribution. Cosine similarity is used to calculate the similarity between feature vectors. It is a temperature parameter used to smooth the probability distribution. The first semantic feature of the source or target domain Each pixel value.

8. The method as described in claim 7, characterized in that, The multi-view prototype contrastive learning framework also includes: (5a) Extract boundary region masks from the real labels of the source domain and the optimized pseudo labels respectively through morphological operations. : in, For the core size is The expansion operation, For the core size is Corrosion operation; (5b) Calculate the centroid of pixel features within the boundary region to construct an independent boundary prototype. : in, For the source domain The total number of pixels of each class The first semantic feature of the source or target domain Each pixel value; For the boundary mask of the first Each pixel value; (5c) The boundary prototype A contrastive learning process is added to enhance the model’s ability to model fine boundary structures. This process complements global contrastive learning by reusing the contrastive learning loss function form in step (4c) and replacing the input prototype with the boundary prototype.

Images