Semi-supervised pericardial fat segmentation method based on multi-model cross teaching

Abstract

The application belongs to the technical field of medical image processing, and particularly relates to a semi-supervised pericardial fat segmentation method based on multi-model cross teaching. First, a lightweight SAM model is used to extract ROI of pericardial fat tissue to obtain a rough ROI; the lightweight SAM model comprises an image encoder and a visual Transformer decoder, and an adapter layer is embedded in each Transformer block of the image encoder; the rough ROI is subjected to morphological processing to obtain a fine ROI; the fine ROI is mapped to an input image and cropped to obtain a fine ROI image; then, a segmentation model composed of an encoder, a bottleneck processing module and a decoder is built based on a TransUNet, the bottleneck processing module comprises an ASPP module and a plurality of WTS blocks; finally, the segmentation model, a UNeXt model and a Mamba-UNet model are subjected to semi-supervised training based on multi-model cross teaching, and the trained segmentation model is used for pericardial fat segmentation. The method can better capture edge and detail information of pericardial fat and improve segmentation precision.

Description

Technical Field

[0001] This invention belongs to the field of medical image processing technology, specifically a semi-supervised pericardial fat segmentation method based on multi-model cross-teaching. Background Technology

[0002] Pericardial adipose tissue is a specialized tissue that surrounds the epicardium of the heart and coronary arteries, consisting of pericardial adipose tissue and epicardial adipose tissue. Pericardial adipose tissue, located in the outer layer of the pericardium, has been extensively studied and is closely related to various heart diseases. Its unique structure not only participates in energy supply and mechanical protection but also regulates local and systemic metabolism through the secretion of cytokines, thereby directly affecting cardiac function and health. Specifically, increased volume of pericardial adipose tissue not only increases the secretion of pro-inflammatory adipokines but may also accelerate endothelial damage through local inflammatory responses, promoting the formation of atherosclerotic plaques and fibrosis of the vascular wall, thus increasing the risk of coronary artery disease. Epicardial adipose tissue, located between the epicardium and the inner pericardium, is mainly concentrated around the ventricles and coronary arteries. Increased epicardial adipose tissue thickness is positively correlated with the occurrence of hypertrophic cardiomyopathy and acute myocardial infarction. Therefore, accurate segmentation of pericardial adipose tissue is crucial for assessing cardiac risk, supporting early diagnosis, and personalized treatment.

[0003] With the rapid development of deep learning technology, it has made significant progress in medical image segmentation tasks. However, due to the generally small size of existing clinical databases, it is difficult to effectively train models using fully supervised methods. Semi-supervised training, on the other hand, can reduce reliance on large amounts of manually labeled data while maintaining high accuracy, providing a new solution for the automatic segmentation of pericardial fat tissue. In semi-supervised learning methods, consistency regularization methods, especially teacher-student architecture-based model frameworks, have been widely used. This framework updates the teacher model parameters using the exponential moving average (EMA) method, thereby establishing a close connection between the teacher and student models. This design requires the teacher model to reflect changes in the student model in real time to maintain training stability and consistency. Although this method improves segmentation accuracy and robustness, the flexibility of model structure and learning strategies is somewhat limited because the teacher and student models need to maintain a high degree of consistency, especially when dealing with highly diverse, blurred, and irregularly distributed pericardial fat features.

[0004] Pericardial fat segmentation is a significant challenge in medical image processing, primarily due to several factors: First, pericardial fat shares density and texture similarities with adjacent structures such as myocardium and coronary vessels, making accurate differentiation difficult. Second, noise, low resolution, and motion blur caused by heartbeats degrade image quality, further increasing segmentation complexity. Third, the distribution and thickness of pericardial fat vary among individuals, adding to the complexity of segmentation. Current research largely relies on U-Net, Transformer, and their variants. While these models have achieved good results in image segmentation, they still have limitations in feature extraction and information propagation. The U-Net model excels at capturing local features but is limited in handling complex issues such as blurred boundaries between pericardial fat and surrounding tissues, and long-range dependencies. The Transformer model effectively captures global features and long-range dependencies, but it demands high computational resources, especially when processing high-resolution medical images, easily straining memory and computing resources. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the technical problem this invention aims to solve is to propose a semi-supervised pericardial fat segmentation method based on multi-model cross-teaching.

[0006] The present invention solves the aforementioned technical problem by adopting the following technical solution:

[0007] A semi-supervised pericardial fat segmentation method based on multi-model cross-teaching is characterized by the following steps:

[0008] S1. Use the lightweight SAM model to extract the ROI of pericardial adipose tissue in cardiac MR images to obtain a coarse ROI.

[0009] The lightweight SAM model includes an image encoder and a visual Transformer decoder. Each Transformer block of the image encoder embeds an adapter layer, and the adapter layer is located after the multi-head attention layer of the Transformer block.

[0010] S2. Perform morphological processing on the coarse ROI to obtain the fine ROI; map the fine ROI onto the input image and crop it to obtain the fine ROI image;

[0011] S3. Build a segmentation model based on TransUNet; the segmentation model includes an encoder, a decoder, and a bottleneck processing module that connects the encoder and decoder. The bottleneck processing module includes an ASPP module and multiple WTS blocks.

[0012] In the WTS block, the input features are normalized to obtain normalized input features. These normalized input features are then fed into two branches. In one branch, the input features pass through a linear layer, a depthwise separable convolutional layer, an SS2D module, and a normalization layer. This normalized input features are then multiplied pixel-by-pixel with the features obtained from the linear layer. The resulting multiplicative features are then passed through another linear layer to obtain the output features of that branch. In the other branch, the normalized input features pass through a frequency domain feature extraction module, a normalization layer, and a multilayer perceptron to obtain the output features of that branch. Finally, the output features of both branches are added pixel-by-pixel to the input features of the WTS block to obtain the output features of the WTS block.

[0013] S4. Based on multi-model cross-teaching, the segmentation model, UNeXt and Mamba-UNet models are semi-supervised and trained. The trained segmentation model is then used for pericardial fat segmentation.

[0014] Furthermore, in step S4, the refined ROI image x i The inputs are fed into the Mamba-UNet model and the segmentation model respectively to generate segmentation maps. Pseudo-labels are then generated from the segmentation maps. The pseudo-labels are represented as follows:

[0015]

[0016] In the formula, The segmentation map is generated by the Mamba-UNet model. The obtained pseudo-labels The segmentation map is generated by the segmentation model. The obtained pseudo-labels, Softmax(·) is the Softmax function, and T(t) is the temperature parameter at the t-th training round;

[0017] Consistency loss of unlabeled data L con Defined as:

[0018]

[0019] In the formula, L softdice (·) denotes a variant of the Dice loss, where N is the number of images in the current batch. Represents a segmentation graph The segmentation result at position (h,w) in the middle. Indicates pseudo-tags The result at position (h,w);

[0020] Supervision loss L of labeled data sup Defined as:

[0021] L sup =w CE L CE(G,P)+w Dice L Dice (G,P)+w BD L BD (G,P) (18)

[0022]

[0023] L BD (G,P)=L mse (sobel(G),sobel(P)) (21)

[0024] In the formula, L CE (G,P) represents the cross-entropy loss, and L... Dice (G,P) represents the Dice loss, L BD (G,P) represents the boundary loss, w CE w Dice and w BD Both are weighting factors, G is the label, and P is the segmentation result. c and P c These represent the true label and predicted probability of category c, respectively, where C is the number of categories, and L is the number of categories. mse (·) is the mean squared error loss, and sobel(·) is the Sobel edge operator.

[0025] The loss between the Mamba-UNet model and the segmentation model L part1 Represented as:

[0026] L part1 =L sup +λL con (twenty two)

[0027] Similarly, the loss L between the UNeXt model and the segmentation model is obtained. part2 The total loss is then expressed as:

[0028] L total =w1L part1 +w2L part2 (twenty three)

[0029] In the formula, λ, w1, and w2 are all weighting factors.

[0030] Furthermore, in the adapter layer, the input features are processed by average pooling and then enter two processing branches. In one processing branch, the features obtained by average pooling are pre-activated, multiplied by the input features of the adapter layer, and then passed through a Transformer block to obtain the output features of that processing branch. In the other processing branch, the features obtained by average pooling are sequentially processed by depthwise convolution, ReLU activation, deconvolution, and Sigmoid activation to obtain the output features of that processing branch. The output features of the two processing branches are added together, and then sequentially processed by convolution, ReLU activation, deconvolution, and ReLU activation before being added to the input features of the adapter layer to obtain the output features of the adapter layer.

[0031] Furthermore, for the morphological processing of coarse ROIs, the coarse ROIs are first normalized and then binarized; then, the binarized ROIs are converted into arrays and connected component labels are applied to obtain the maximum connected component, thus obtaining ROIs containing only the maximum connected component; dilation convolution is performed on the ROIs containing only the maximum connected component to obtain fine ROIs.

[0032] Furthermore, in the frequency domain feature extraction module, the normalized input features are decomposed by wavelet to obtain low-frequency sub-bands, horizontal high-frequency sub-bands, vertical high-frequency sub-bands, and diagonal high-frequency sub-bands. Each sub-band is multiplied pixel-by-pixel with its corresponding learnable weight matrix to obtain weighted low-frequency sub-bands, weighted horizontal high-frequency sub-bands, weighted vertical high-frequency sub-bands, and weighted diagonal high-frequency sub-bands. Each weighted sub-band is then subjected to inverse wavelet transform to obtain reconstructed low-frequency sub-bands, reconstructed horizontal high-frequency sub-bands, reconstructed vertical high-frequency sub-bands, and reconstructed diagonal high-frequency sub-bands. All reconstructed sub-bands are then fused to obtain the output features of the frequency domain feature extraction module.

[0033] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0034] 1. To achieve accurate segmentation of pericardial fat, the SAM model was improved to obtain a lightweight SAM model. This lightweight SAM model was fine-tuned using a small amount of labeled ROI data. Combined with the large number of pre-trained parameters of the original SAM model, the lightweight SAM model can accurately extract ROIs containing pericardial fat tissue from cardiac MR images. An adapter layer was introduced into the image encoder to help it locate the desired ROI, enhancing its ROI extraction capabilities while maintaining its own feature extraction capabilities. Compared to traditional center point or rectangular bounding box methods, the lightweight SAM model can more intelligently exclude irrelevant regions, focusing attention on key areas related to pericardial fat tissue. This reduces the interference of irrelevant regions on segmentation accuracy, improving the performance of the segmentation model and reducing the waste of subsequent computational resources.

[0035] 2. To address the problem that traditional methods cannot accurately handle complex boundaries, this invention further introduces morphological processing to refine the coarse ROI after extracting it. Morphological processing can effectively remove noise and fill the empty areas inside the ROI, making the ROI boundary clearer. This allows the segmentation model to be trained and inferred on a cleaner and more accurate fine ROI image, which is beneficial to improving segmentation accuracy.

[0036] 3. An innovative multi-model cross-teaching strategy is employed to train the segmentation model, enhancing segmentation performance through a semi-supervised learning framework combining pseudo-labels and consistency regularization. Specifically, the pseudo-label method generates labels for unlabeled data, expanding the scale of the training data; while consistency regularization improves the model's generalization ability and robustness through data augmentation. The cross-teaching strategy effectively mitigates the uncertainty introduced by pseudo-labels and reduces the sensitivity of consistency regularization to noise, thereby achieving more stable and efficient segmentation results.

[0037] 4. A segmentation model is built based on TransUNet, and a Spatial-Channel Compression and Activation Module (ASPP) with an attention mechanism is introduced to enhance the global receptive field and improve the model's ability to perceive key regions of the image. Simultaneously, a wavelet transform-based frequency domain feature extraction and Mamba feature extraction module (WTS block) is used instead of the Transformer block, effectively capturing global features while focusing on local features, and significantly reducing the number of parameters, thereby optimizing computational resources. Compared with traditional segmentation models such as UNet and Transformer, UNeXt and Mamba-UNet are introduced as auxiliary models. The backbone model (i.e., the segmentation model) is trained through multi-model cross-teaching, enabling the backbone model to better capture the edge and detail information of pericardial fat, thus improving segmentation accuracy. Furthermore, multi-model cross-teaching offers better training efficiency and stability, reducing the training difficulty with limited labeled data. Attached Figure Description

[0038] Figure 1 This is an overall flowchart of the present invention;

[0039] Figure 2 This is a structural diagram of the lightweight SAM model of the present invention;

[0040] Figure 3 This is a structural diagram of the segmentation model of the present invention;

[0041] Figure 4 This is a diagram of the semi-supervised training framework for multi-model cross-teaching in this invention. Detailed Implementation

[0042] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments, but this does not limit the scope of protection of this application.

[0043] This invention provides a semi-supervised pericardial fat segmentation method based on multi-model cross-teaching (hereinafter referred to as the method, see below). Figures 1-4 The process includes the following steps:

[0044] S1. Use the lightweight SAM model to extract the ROI of pericardial adipose tissue in cardiac MR images to obtain a coarse ROI.

[0045] The lightweight SAM model includes an image encoder and a visual Transformer decoder (ViT). The image embedding features obtained by the image encoder are input into the visual Transformer decoder for decoding to obtain a coarse ROI. The visual Transformer decoder replaces the cue encoder and mask decoder of the original SAM to achieve lightweighting. An adapter layer is introduced into the image encoder, enabling it to learn more parameters and helping it locate the ROI to be extracted. This enhances the image encoder's ability to extract ROIs while maintaining its own feature extraction capabilities. Figure 3 As shown, each Transformer block of the image encoder embeds an adapter layer, which is located after the multi-head attention layer of the Transformer block. In the adapter layer, the input features are processed by average pooling and then enter two processing branches. In one processing branch, the features obtained by average pooling are pre-activated and multiplied with the input features of the adapter layer. The resulting feature is then processed by the Transformer block to obtain the output features of that processing branch. In the other processing branch, the features obtained by average pooling are sequentially processed by depthwise convolution, ReLU activation, deconvolution, and Sigmoid activation to obtain the output features of that processing branch. The output features of the two processing branches are added together, and then sequentially processed by convolution, ReLU activation, deconvolution, and ReLU activation before being added to the input features of the adapter layer to obtain the output features of the adapter layer.

[0046] S2. Perform morphological processing on the coarse ROI to obtain the fine ROI; map the fine ROI onto the input image and crop it to obtain the fine ROI image;

[0047] Rough ROIs have relatively rough contours and may also contain noise and holes. To reduce the impact of these problems on the segmentation results, ROIs are refined through morphological processing.

[0048] First, the coarse ROI is normalized by using the Sigmoid activation function to map pixel values ​​to the range of 0 to 1. Then, binarization is performed, setting pixel values ​​greater than 0.5 to 1 and pixel values ​​equal to or less than 0.5 to 0. The specific process is as follows:

[0049]

[0050] In the formula, R represents the roughened ROI, and R' represents the binarized ROI;

[0051] The binarized ROI is converted into an array, and its connected components are labeled using a connected component labeling algorithm. This process is defined as follows:

[0052] (L,N)=cdla(R′,connectivity=2) (2)

[0053] In the formula, L represents the ROI after connected domain labeling, N represents the number of connected domains, cala (Connected Domain Labeling Algorithm) represents the connected domain labeling algorithm, and connectivity represents adjacency.

[0054] If there is more than one connected component, the number of pixels in each connected component is calculated to identify the largest connected component, thus obtaining the ROI containing only the largest connected component. A 3×3 dilated convolution operation is performed on the ROI containing only the largest connected component to eliminate holes in the ROI, resulting in a refined ROI. The binary image corresponding to the refined ROI is multiplied pixel by pixel with the input image to map the refined ROI onto the input image, resulting in the ROI-mapped input image. The ROI-mapped input image is then cropped to obtain the refined ROI image, which is the output of the morphological processing.

[0055] S3. A segmentation model (WTS-UNet) is built based on TransUNet. The segmentation model includes an encoder, a decoder, and a bottleneck processing module connecting the encoder and decoder. The bottleneck processing module includes an ASPP module (dilated spatial convolutional pooling pyramid) and multiple WTS blocks (12 in this embodiment). The ASPP module combines spatial attention and channel attention mechanisms, significantly expanding the receptive field and enhancing attention to key regions, especially performing well in handling complex or small object segmentation tasks. It optimizes feature representation through compression and activation operations, effectively reducing redundant information and improving the responsiveness to useful features, thereby capturing richer feature information with low computational cost and thus improving segmentation accuracy.

[0056] In the WTS block, the input features are normalized to obtain normalized input features. These normalized input features are then fed into two branches. In one branch, the input features pass through a linear layer, a depthwise separable convolutional layer, an SS2D module, and a normalization layer. This normalized input features are then multiplied pixel-by-pixel with the features obtained after the linear layer. The resulting multiplicative features are then passed through another linear layer to obtain the output features of that branch. In the other branch, the normalized input features pass through a frequency domain feature extraction module, a normalization layer, and a multilayer perceptron to obtain the output features of that branch. The output features of both branches are then added pixel-by-pixel to the input features of the WTS block to obtain the output features of the WTS block. The frequency domain feature extraction module extracts frequency domain features based on Discrete Wavelet Transform (DWT). First, the normalized input features are decomposed into four sub-bands by a wavelet decomposition layer: low-frequency sub-band (LL), horizontal high-frequency sub-band (LH), vertical high-frequency sub-band (HL), and diagonal high-frequency sub-band (HH), represented as follows:

[0057] LL[m,n]=∑ i ∑ j I[i,j]·g[i-2m]·g[j-2n] (3)

[0058] LH[m,n]=∑ i ∑ j I[i,j]·g[i-2m]·h[j-2n] (4)

[0059] HL[m,n]=∑ i ∑ j I[i,j]·h[i-2m]·g[j-2n] (5)

[0060] HH[m,n]=∑ i ∑ j I[i,j]·h[i-2m]·h[j-2n] (6)

[0061] In the formula, I[i,j] represents the normalized input feature; LL[m,n] represents the pixel value of the low-frequency sub-band, used to reflect the global features or contour information of the image; LH[m,n] represents the pixel value of the horizontal high-frequency sub-band, used to reflect the detail information in the horizontal direction, i.e., the horizontal edge features; HL[m,n] represents the pixel value of the vertical high-frequency sub-band, used to reflect the detail information in the vertical direction, i.e., the vertical edge features; HH[m,n] represents the pixel value of the diagonal high-frequency sub-band, used to reflect the detail information in the diagonal direction, i.e., fine textures and noise; g[·] is a low-pass filter, using a scaling function, used to extract the smooth part of the image; h[·] is a high-pass filter, using a wavelet function, used to extract the detail part of the image; 2m and 2n represent indices;

[0062] Each subband is multiplied pixel-by-pixel by its corresponding learnable weight matrix to obtain the weighted low-frequency subband LL'[m,n], the weighted horizontal high-frequency subband LH'[m,n], the weighted vertical high-frequency subband HL'[m,n], and the weighted diagonal high-frequency subband HH'[m,n]. Taking the low-frequency subband as an example, the weighted low-frequency subband LL'[m,n] is represented as follows:

[0063] LL′[m,n]=W LL ·LL[m,n] (7)

[0064] In the formula, W LL The learnable weight matrix represents the low-frequency subband;

[0065] By combining learned weights with the DWT formula, the features are decomposed into multiple levels of wavelet decomposition, thereby extracting detailed information at different resolution levels, which helps to capture local structural features.

[0066] Each weighted subband is input into the wavelet reconstruction layer, and after inverse wavelet transform, the reconstructed low-frequency subband I' is obtained. LL [i,j]、Reconstructing the horizontal high-frequency subband I' LH [i,j]、Reconstructing the vertical high-frequency subband I' HL [i,j] and reconstructed diagonal high-frequency subband I' HH [i,j]; These sub-bands are then fused to obtain frequency domain features, achieving feature reconstruction. The process is as follows:

[0067] I' LL [i,j]=∑ m ∑ n LL′[m,n]·g[i-2m]·g[j-2n] (8)

[0068] I' LH [i,j]=Σ m Σ n LH′[m,n]·g[i-2m]·h[j-2n] (9)

[0069] I' HL [i,j]=Σ m Σ n HL′[m,n]·h[i-2m]·g[j-2n] (10)

[0070] I' HH [i,j]=∑ m ∑ n HH′[m,n]·h[i-2m]·h[j-2n] (11)

[0071] I′[i,j]=I′ LL [i,j]+I′LH [i,j]+I′ HL [i,j]+I′ HH [i,j] (12)

[0072] In the formula, I'[i,j] represents the frequency domain feature.

[0073] S4. Based on multi-model cross-teaching, WTS-UNet, UNeXt and Mamba-UNet models are semi-supervised and trained, and the trained segmentation models are used for pericardial fat segmentation.

[0074] The UNeXt model effectively captures detailed information and microstructures in images through a local focusing mechanism, while the Mamba-UNet model achieves a deep understanding of global contextual information in images. Figure 4 As shown, WTS-UNet is used as the backbone model, while UNeXt and Mamba-UNet are used as auxiliary models for cross-teaching to improve the performance of the backbone model. For unlabeled data, the prediction results of UNeXt and Mamba-UNet models guide the learning process of the backbone model, while the prediction results of the backbone model optimize the parameters of the auxiliary models. This cross-teaching mechanism allows the backbone model to combine the advantages of the two auxiliary models in capturing details and understanding global contextual information, which is beneficial to improving the segmentation effect.

[0075] For a given fine ROI image x i The Mamba-UNet and WTS-UNet models generate segmentation maps respectively, and the process is as follows:

[0076]

[0077] In the formula, and These represent the Mamba-UNet model. and WTS-UNet model The generated segmentation map;

[0078] Since the Mamba-UNet and WTS-UNet models belong to different learning paradigms, the segmentation maps they generate inherently have different characteristics; therefore, pseudo-labels are generated based on the segmentation results, and the generation process is as follows:

[0079]

[0080] In the formula, and These are respectively composed of segmentation maps and The generated pseudo-labels, Softmax(·) is the Softmax function, T(t) is the temperature parameter at the t-th training round, T base It is the initial temperature parameter, T final This is the lowest temperature parameter at the end of training; decay_rate controls the decay rate; t total It represents the total number of training rounds.

[0081] The smoothness of the segmentation result distribution is controlled by dynamically adjusting the temperature parameter T(t). As training progresses, the temperature parameter T(t) gradually decays exponentially, which not only promotes the model's exploration of multiple possible label distributions in the early stage of training, but also promotes the model's exploration of multiple possible label distributions in the later stage of training, thereby enhancing the model's deterministic learning of specific patterns and structures.

[0082] Segmentation maps generated by Mamba-UNet and WTS-UNet models and With pseudo-tags and No gradient backpropagation is performed between them to preserve perturbations during training. For unlabeled data, consistency loss is used to evaluate the consistency of segmentation results between the Mamba-UNet and WTS-UNet models. This requires no additional constraints and effectively avoids error accumulation that may result from prediction errors in a single model, thus improving overall robustness and accuracy. The consistency loss for unlabeled data is defined as follows:

[0083]

[0084] In the formula, L con L represents the consistency loss of unlabeled data. softdice (·) denotes a variant of the Dice loss, where N is the number of images in the current batch. Represents a segmentation graph The segmentation result at position (h,w) in the middle. Indicates pseudo-tags The result at position (h,w); The calculation process is the same as that of equation (17).

[0085] For labeled data, the model performs supervised learning directly using the real labels. The supervised loss for labeled data is then defined as:

[0086] L sup =w CE L CE (G,P)+w Dice L Dice (G,P)+w BD L BD (G,P) (18)

[0087]

[0088] L BD (G,P)=L mse (sobel(G),sobel(P)) (21)

[0089] In the formula, L sup L represents the supervised loss of labeled data. CE (G,P) represents the cross-entropy loss, which accelerates the initial learning of the model by imposing a high penalty on incorrect predictions; L Dice (G,P) represents the Dice loss, which addresses the data imbalance problem in segmentation tasks by optimizing the overlap between the segmented region and the ground truth region; L BD (G,P) represents the boundary loss, which optimizes the model's segmentation of the target boundary, especially for irregular boundaries, significantly improving the accuracy of segmentation; w CE w Dice and w BD Both are weighting factors, G is the label, and P is the segmentation result. c and P c These represent the true label and predicted probability of category c, respectively, where C is the number of categories, and L is the number of categories. mse (·) is the mean squared error loss, and sobel(·) is the Sobel edge operator.

[0090] The loss between the Mamba-UNet model and the WTS-UNet model L part1 Supervised loss L from labeled data sup Consistency loss L with unlabeled data con The composition is shown in Equation (22); similarly, the loss L between the UNeXt model and the WTS-UNet model is obtained. part2 ;

[0091] L part1 =L sup +λL con (twenty two)

[0092] Ultimately, the total loss is expressed as:

[0093] L total =w1L part1 +w2L part2 (twenty three)

[0094] In the formula, λ is a weighting factor, defined by a time-dependent Gaussian warm-up function: w1 and w2 are both weight factors that change over time. In the early stages of training, the Mamba-UNet model focuses more on global features, so its initial weights are larger, while the UNeXt model focuses more on local information, so its initial weights are smaller and gradually increase during training until they reach a stable value.

[0095] Example

[0096] First, the lightweight SAM model is fine-tuned simultaneously using 300 manually labeled ROIs and the ACDC dataset. Then, the results of downstream segmentation tasks (segmentation maps generated by UNeXt or Mamba-UNet models) are overlaid with the corresponding refined ROIs. The resulting refined ROI images are then used as pseudo-labels and fed back into the lightweight SAM model for further training. The model is fine-tuned again, and its parameters are continuously updated during the segmentation model training process to improve the accuracy of ROI extraction.

[0097] The lightweight SAM model was used for ROI extraction, and IoU and Re were used as evaluation indicators. The results showed that IoU and Re reached 88.19% and 95.63% respectively, which achieved satisfactory results. This indicates that it is feasible to use the lightweight SAM model for ROI extraction by fine-tuning it with a small amount of data.

[0098] For segmentation tasks, this invention employs five-fold cross-validation to verify the robustness of the segmentation model, training the model five times for each fold and taking the average and standard deviation. Simultaneously, this invention's method is compared with the existing semi-supervised method DCT. [1] ICT [2] EM [3] UAMT [4] CCT [5] CPS [6] and URPC [7] For comparison, all methods used the same parameter configuration and training strategy, and all employed ROI extraction and morphological processing to control variables. Precision (Pr), Jaccard index, F1 score, and High Definition (HD) were selected as evaluation metrics. A higher Pr indicates higher accuracy in predicting positive samples; higher Jaccard index and F1 score indicate better precision and recall; and a lower HD indicates better performance in capturing the boundaries and microstructures of segmented targets. The performance of different methods' five-fold cross-validation results on the four evaluation metrics is shown in Tables 1-4.

[0099] Table 1. Results of Pr↑ in five-fold cross-validation.

[0100]

[0101] Table 2 shows the results of F1↑ in the five-fold cross-validation.

[0102]

[0103]

[0104] Table 3 shows the results of Jaccard↑ in the five-fold cross-validation.

[0105]

[0106] Table 4. Results of HD↓ in five-fold cross-validation.

[0107]

[0108]

[0109] As shown in Tables 1, 2, and 3, the segmentation model of this invention outperforms other methods in terms of Pr, Jaccard index, and F1 score, indicating that the model has significant advantages in terms of region coverage and prediction accuracy, and can achieve higher consistency between the target region and the real region. Meanwhile, Table 4 shows that the segmentation model of this invention also outperforms other methods in terms of HD index, indicating that the model has stronger performance in boundary fitting accuracy and detail capture, and can better achieve boundary and detail segmentation of the target region. In summary, this invention effectively improves the model's ability in global feature extraction and detail segmentation through a multi-model cross-teaching strategy. Moreover, the segmentation model effectively enhances its ability to segment pericardial fat edges and details by integrating the ASPP module and WTS block for feature extraction.

[0110] The sources of the aforementioned existing methods are as follows:

[0111] [1]Qiao S, Shen W, Zhang Z, et al.Deep Co-Training for Semi-SupervisedImage Recognition[M / OL] / / Computer Vision–ECCV 2018,Lecture Notes in ComputerScience.2018:142-159.

[0112] [2]Verma V,Kawaguchi K,Lamb A,et al.Interpolation consistencytraining for semi-supervised learning[J / OL].Neural Networks, 2022:90-106.

[0113] [3]Vu T H,Jain H,Bucher M,et al.ADVENT:Adversarial EntropyMinimization for Domain Adaptation in Semantic Segmentation[C / OL].2019IEEE / CVF Conference on Computer Vision and Pattern Recognition(CVPR),Long Beach,CA,USA,2019,pp.2512-2521.

[0114] [4]Yu L,Wang S,Li X,et al.Uncertainty-Aware Self-ensembling Model forSemi-supervised 3D Left Atrium Segmentation[M / OL] / / Lecture Notes in ComputerScience,Medical Image Computing and Computer Assisted Intervention–MICCAI2019.2019:605-613.

[0115] [5]Ouali Y,Hudelot C,Tami M.Semi-Supervised Semantic SegmentationWith Cross-Consistency Training[C / OL] / / 2020IEEE / CVF Conference on ComputerVision and Pattern Recognition(CVPR),Seattle,WA,USA.2020.pp.12674–12684.

[0116] [6]Chen X,Yuan Y,Zeng G,et al.Semi-Supervised Semantic Segmentationwith Cross Pseudo Supervision[C / OL] / / 2021IEEE / CVF Conference on ComputerVision and Pattern Recognition(CVPR),Nashville,TN,USA.2021.pp.2613–2622.

[0117] [7]Luo X, Wang G, Liao W, et al. Semi-supervised medical imagesegmentation via uncertainty rectified pyramid consistency[J]. Medical ImageAnalysis 80(2022)102517.

[0118] Any aspects not covered in this invention are applicable to existing technologies.

Claims

1. A semi-supervised pericardial fat segmentation method based on multi-model cross-teaching, characterized in that, Includes the following steps: S1. Use the lightweight SAM model to extract the ROI of pericardial adipose tissue in cardiac MR images to obtain a coarse ROI. The lightweight SAM model includes an image encoder and a visual Transformer decoder. Each Transformer block of the image encoder embeds an adapter layer, and the adapter layer is located after the multi-head attention layer of the Transformer block. S2. Perform morphological processing on the coarse ROI to obtain the fine ROI; map the fine ROI onto the input image and crop it to obtain the fine ROI image; S3. Build a segmentation model based on TransUNet; the segmentation model includes an encoder, a decoder, and a bottleneck processing module that connects the encoder and decoder. The bottleneck processing module includes an ASPP module and multiple WTS blocks. In the WTS block, the input features are normalized to obtain normalized input features; The normalized input features are fed into two branches. In one branch, the input features pass through a linear layer, a depthwise separable convolutional layer, an SS2D module, and a normalization layer in sequence. Then, they are multiplied pixel-by-pixel with the normalized input features after passing through the linear layer. The resulting feature then passes through a linear layer to obtain the output feature of that branch. In the other branch, the normalized input features pass through a frequency domain feature extraction module, a normalization layer, and a multilayer perceptron in sequence to obtain the output feature of that branch. The output features of the two branches are added pixel by pixel to the input features of the WTS block to obtain the output features of the WTS block; S4. Based on multi-model cross-teaching, the segmentation model, UNeXt and Mamba-UNet models are semi-supervised and trained. The trained segmentation model is then used for pericardial fat segmentation. x i The inputs are fed into the Mamba-UNet model and the segmentation model respectively to generate segmentation maps. Pseudo-labels are then generated from the segmentation maps. The pseudo-labels are represented as follows: In the formula, The segmentation map is generated by the Mamba-UNet model. The obtained pseudo-labels The segmentation map is generated by the segmentation model. The obtained pseudo-labels, Softmax(·) is the Softmax function, and T(t) is the temperature parameter at the t-th training round; Consistency loss of unlabeled data L con Defined as: In the formula, L softdice (·) denotes a variant of the Dice loss, where N is the number of images in the current batch. Represents a segmentation graph The segmentation result at position (h,w) in the middle. Indicates pseudo-tags The result at position (h,w); Supervision loss L of labeled data sup Defined as: L sup =w CE L CE (G,P)+w Dice L Dice (G,P)+w BD L BD (G,P) (18) L BD (G,P)=L mse (sobel(G),sobel(P)) (21) In the formula, L CE (G,P) represents the cross-entropy loss, and L... Dice (G,P) represents the Dice loss, L BD (G,P) represents the boundary loss, w CE w Dice and w BD Both are weighting factors, G is the label, and P is the segmentation result. c and P c These represent the true label and predicted probability of category c, respectively, where C is the number of categories, and L is the number of categories. mse (·) is the mean squared error loss, and sobel(·) is the Sobel edge operator; The loss between the Mamba-UNet model and the segmentation model L part1 Represented as: THE part1 =L sup +λL con (22) Similarly, the loss L between the UNeXt model and the segmentation model is obtained. part2 The total loss is then expressed as: L total =w1L part1 +w2L part2 (23) In the formula, λ, w1, and w2 are all weighting factors.

2. The semi-supervised pericardial fat segmentation method based on multi-model cross-teaching according to claim 1, characterized in that, In the adapter layer, the input features are processed by average pooling and then enter two processing branches. In one processing branch, the features obtained by average pooling are pre-activated, multiplied by the input features of the adapter layer, and then passed through the Transformer block to obtain the output features of that processing branch. In another processing branch, the features obtained by average pooling are sequentially processed by depthwise convolution, ReLU activation, deconvolution, and Sigmoid activation to obtain the output features of this processing branch. After the output features of the two processing branches are added together, they are sequentially processed by convolution, ReLU activation, deconvolution, and ReLU activation, and then added to the input features of the adapter layer to obtain the output features of the adapter layer.

3. The semi-supervised pericardial fat segmentation method based on multi-model cross-teaching according to claim 2, characterized in that, For the morphological processing of coarse ROIs, the coarse ROIs are first normalized and then binarized. Then, the binarized ROIs are converted into arrays and connected component labels are applied to obtain the maximum connected component, thus obtaining ROIs containing only the maximum connected component. The ROIs containing only the maximum connected component are then subjected to dilation convolution to obtain fine ROIs.

4. The semi-supervised pericardial fat segmentation method based on multi-model cross-teaching according to claim 1, characterized in that, In the frequency domain feature extraction module, the normalized input features are decomposed by wavelet to obtain low-frequency sub-bands, horizontal high-frequency sub-bands, vertical high-frequency sub-bands, and diagonal high-frequency sub-bands. Each sub-band is multiplied pixel-by-pixel with its corresponding learnable weight matrix to obtain weighted low-frequency sub-bands, weighted horizontal high-frequency sub-bands, weighted vertical high-frequency sub-bands, and weighted diagonal high-frequency sub-bands. Each weighted sub-band is then subjected to inverse wavelet transform to obtain reconstructed low-frequency sub-bands, reconstructed horizontal high-frequency sub-bands, reconstructed vertical high-frequency sub-bands, and reconstructed diagonal high-frequency sub-bands. All reconstructed sub-bands are then fused to obtain the output features of the frequency domain feature extraction module.

Images