Self-supervised pre-training method for multi-modal medical image segmentation model

Abstract

The invention relates to a self-supervised pre-training method for a medical image segmentation model, and belongs to the technical field of image recognition. According to the method, for an unlabeled multi-modal three-dimensional medical image, overlapping region sampling is carried out on the multi-modal image, enhancement is carried out based on random region and modal discarding, then a twin deep learning neural network is constructed, and an image reconstruction task is established. On the basis, voxel-level multi-scale feature extraction is further performed by using a twin network, and a self-distillation task is established. And finally, combining an image reconstruction task and a self-distillation task to carry out self-supervised pre-training of the model, wherein the obtained pre-training weight can be used for transfer learning on a downstream image segmentation task. The pre-training model trained by the invention can effectively extract modal invariance features and multi-scale semantic information from a multi-modal three-dimensional image, effectively provide initial parameters with stronger feature expression ability for training of the model on a small amount of labeled data of a downstream task, improve the performance of the model on a downstream data set, and improve the performance of the model on the downstream data set. And the labeling requirement is reduced, and the training efficiency is improved.

Description

Technical Field

[0001] This invention relates to a self-supervised pre-training method for segmentation models based on deep neural networks in multimodal three-dimensional medical images, belonging to the field of image recognition technology. Background Technology

[0002] Multimodal medical image segmentation plays a crucial role in the diagnosis / prognosis, treatment planning, and follow-up of various diseases. Taking multimodal magnetic resonance imaging (MRI) as an example, it provides a primary reference for the clinical diagnosis of brain tumors. Different modalities of images can visualize complementary information from different subregions: T1-weighted (T1) and contrast-enhanced T1-weighted (T1c) images can display the core region of a brain tumor, while T2-weighted (T2) and T2 fluid-attenuated inversion recovery (FLAIR) sequences provide good contrast for peritumoral edema.

[0003] In recent years, deep learning models have achieved excellent performance in automatic segmentation of multimodal images. However, their success relies on large, labeled training sets for fully supervised learning, which has two main drawbacks: First, the annotation process is very time-consuming and expensive for large datasets, especially for multi-class segmentation models of 3D medical images, where pixel-level labels are very difficult to obtain. Second, model performance is limited when the training set is limited by the rarity of the disease or the small size of the specific patient group. For example, the incidence of diffuse midline gliomas and brain metastases in children is much lower than that in adults, resulting in relatively small datasets that cannot meet the needs of training high-performance segmentation models. Furthermore, in some underdeveloped regions, MRI examinations are not widespread, which also leads to relatively small datasets available for training models.

[0004] To overcome the limitations of fully supervised training, pre-training has been proven to effectively improve the performance of deep learning models on small datasets. However, supervised pre-training not only requires additional data labeling costs but also often suffers from limited generalization ability on downstream tasks and datasets due to feature bias. Therefore, self-supervised learning, which does not require additional labeling, has become a more practical pre-training method. By setting up proxy tasks, self-supervised pre-training allows the model to extract more general feature representations from large unlabeled datasets, which can then be used for fine-tuning on small datasets for specific downstream tasks, reducing data labeling requirements and improving model performance.

[0005] Most existing advanced self-supervised learning methods are based on contrastive learning, which learns global feature representations with transformation invariance and semantic discriminativeness by maximizing the feature similarity between a pair of twin images and minimizing the feature similarity between different images. However, this method struggles to achieve effective local feature discrimination, thus limiting its performance in downstream image segmentation tasks. Furthermore, some self-supervised learning methods based on image reconstruction apply various data augmentation strategies to a given image to generate perturbed inputs, and the deep learning model is pre-trained by recovering the original local intensity details. However, these methods mostly utilize a single modality for pre-training, neglecting the potential for image segmentation using complementary information between multimodal images. Other pre-training methods utilizing multimodal images either simply use multimodal stitching as input or employ multiple modality-specific encoders, lacking effective modality-invariant feature extraction capabilities. Moreover, their pre-training processes often ignore the multi-scale semantic features required to handle segmentation targets at different scales. Therefore, there is a need for self-supervised learning methods that possess both global and local feature representation capabilities and can effectively utilize complementary information between multimodal medical images for pre-training. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of existing multimodal image segmentation models. Addressing the performance limitations of deep learning-based medical image segmentation methods due to the scarcity of labeled data, this invention proposes a pixel-level multimodal and multi-scale self-distillation pre-training method. This enables the pre-trained segmentation model to learn the ability to extract modality-invariant features, global semantic information, and local structural information from unlabeled datasets, thereby improving segmentation performance on downstream small sample datasets.

[0007] This invention employs a novel self-supervised pre-training framework based on image reconstruction and pixel-level multi-scale feature self-distillation to address the challenges in training medical image segmentation models. This is achieved through the following technical solution: First, two sub-images with a certain overlap are randomly extracted from a multimodal image. Then, different regions (blocks) of the sub-images are randomly discarded modally and spatially, and an image reconstruction task is established to learn modality-invariant features. Furthermore, random modal sampling of the overlapping region is used to generate voxel pairs with local context and modal-level perturbations. These voxel pairs are then fed into a Siamese network for multi-scale self-distillation feature learning.

[0008] The technical solution of this invention is a self-supervised pre-training method for a multimodal medical image segmentation model, comprising the following steps:

[0009] Step 1: Establish a pre-trained multimodal image dataset and image segmentation model: Collect unlabeled multimodal medical images, preprocess them, and use them as a pre-training dataset; select an image segmentation network with an encoder-decoder structure and use it as a pre-training model;

[0010] Step 2: Sample overlapping regions in the multimodal pre-training dataset and perform augmentation based on random regions and modality dropout;

[0011] Step 3: Construct a twin deep learning neural network to establish an image reconstruction task;

[0012] Step 4: Perform voxel-level multi-scale feature extraction based on Siamese networks to construct a self-distillation task;

[0013] Step 5: Pre-train the model by combining the image reconstruction task and the self-distillation task;

[0014] Step 6: Fine-tune the pre-trained weights for the downstream segmentation task.

[0015] Furthermore, in step 2, the method for sampling overlapping regions of the multimodal pre-trained image and performing enhancement based on random regions and modality discarding is as follows:

[0016] Let C be the number of modalities in the multimodal 3D images in the pre-training dataset. A sub-region is randomly extracted from the image, and its size along the z-axis, y-axis, and x-axis is represented by D, H, and W, respectively. This sub-region is denoted as X1, and its size is C×D×H×W. A low-quality image X′1 is obtained by applying a random region and modality discarding strategy to X1. Specifically, X1 is divided into B distinct blocks, and each modality in each block is discarded with probability p. Let... For the b-th block of X1, randomly generate a binary discard vector M. b ∈{0,1} c If each element is set to 0 with probability p, then the output after random discarding is:

[0017] The output after randomly discarding all blocks in X1 according to the above rules is denoted as X′1; similarly, a sub-region X2 is randomly obtained from the original image, where X2 and X1 have a certain spatial overlap; the above random region and modal discarding strategy is also applied to X2 to obtain the enhanced sub-region X′2.

[0018] Furthermore, the method for constructing the Siamese neural network and reconstructing the image in step 3 is as follows:

[0019] A deep learning network S consisting of an encoder and a decoder contains a neural network T with the same structure; these two networks constitute a Siamese deep learning network, with the encoder having L downsampling layers and the decoder having L upsampling layers, resulting in L feature maps at different scales.

[0020] The original image is reconstructed by inputting X′1 and the deep learning network S, and the reconstructed image is denoted as X′1. Using the same method, X′2 is input into the deep learning network S for reconstruction, and its prediction result is: Using X1 and X2 as the gold standard for reconstruction, the loss of the reconstructed image is calculated as follows:

[0021]

[0022] Furthermore, the implementation method of the multi-scale feature extraction and self-distillation tasks in step 4 is as follows:

[0023] N voxels are selected in the overlapping region of the two sub-images X1 and X2. Each voxel has a different random dropout mode in both sub-images, thus forming N voxel pairs; using Represents the nth voxel pair. From X1, and From X2; each contains different modal information and local structural information; when an image X is input into a neural network T, its feature maps at L different scales in the decoder are denoted as follows: use Represents the l-th feature map The characteristics of the nth voxel, and the multiscale characteristics of this voxel are represented as follows:

[0024]

[0025] in To represent the feature concatenation operation, according to the above formula, use... and These represent the multi-scale features of pixel n in X1 obtained through networks T and S, respectively; using and Let C represent the multi-scale features of pixel n in X2 obtained through networks T and S, respectively; the dimension of these features is labeled C. m Its value is The sum of dimensions;

[0026] In order to and To project from a high dimension to a low dimension, based on the aforementioned Siamese network, a projector is constructed for T and S respectively, and these two projectors are denoted as g. T and g S; where g S Implemented by a multilayer perceptron network, consisting of three fully connected layers; the first two layers have C m The first layer contains K neurons, each with a batch normalization layer and a ReLU activation function; the last layer contains K neurons and a Softmax activation function layer; g T The structure in g S Based on the existing structure, an additional centering layer was added before Softmax;

[0027] Will Enter g T The resulting projection vector is denoted as Will Enter g S The resulting projection vector is denoted as Using the same method, and Enter g respectively T and g S The resulting projection vectors are denoted as follows: and The self-distillation loss defined based on the Siamese networks S and T is as follows:

[0028]

[0029] Among them, L ce This represents the cross-entropy loss.

[0030] Furthermore, the method for pre-training the model in step 5, which combines the image reconstruction task and the self-distillation task, is as follows:

[0031] Define the overall pre-training loss function L. pre :

[0032] L pre =L sd +λL rec

[0033] Where λ is a weighting coefficient used to control the relative magnitude between reconstruction loss and self-distillation loss; using L pre The gradient value of neural network S is calculated and updated using gradient descent; for neural network T, the exponential moving average method is used for updating. The parameters of the Siamese networks S and T are denoted as θ. S and θ T , for θ T The update method is as follows:

[0034] θ T ←αθ T +(1-α)θ S

[0035] Where α represents the momentum coefficient.

[0036] Furthermore, the pre-trained weights in step 6 are fine-tuned in the downstream segmentation task as follows:

[0037] The parameters of the image segmentation model T are initialized using pre-trained weights; the labeled dataset in the downstream task is denoted as... Where X i Y represents a training image. i This represents its label, where M is the number of samples in the training dataset; X... i Inputting the segmentation model T, the predicted result is P. i =T(X) i The loss function for the fine-tuning process is defined as:

[0038]

[0039] Where L ce It is the cross-entropy loss function, defined as:

[0040]

[0041] Where V = DHW represents the number of voxels, and C is the number of categories; This represents the probability that the v-th pixel belongs to the c-th category. L represents the probability corresponding to the label. dice This is the Dice loss function, which is defined as:

[0042]

[0043] Where ε is a constant used to ensure numerical stability; in L tune Based on this, the parameters in T are updated to complete the fine-tuning of the pre-trained model.

[0044] Compared with the prior art, the present invention has the following advantages:

[0045] (1) Most existing image reconstruction-based pre-training methods only focus on a single modality. The image reconstruction based on multimodal image random regions and modality discarding in this invention pays more attention to multimodal information, allowing the model to learn contextual structure features and modality invariance features at the same time, and has better multimodal feature extraction capabilities.

[0046] (2) Existing contrastive learning self-supervised pre-training methods focus on image-level features and lack the ability to distinguish volumetric pixel-level features. This invention establishes a self-distillation learning method by perturbing the same pixel pair with modal perturbation and local context perturbation, which can better learn voxel-level feature representations and has a finer-grained recognition capability.

[0047] (3) Most existing pre-training techniques ignore the use of multi-scale information. This invention greatly enriches the information available for self-distillation by integrating multi-scale features and promotes the combination of global and local features, thereby improving the segmentation effect on downstream tasks. Attached Figure Description

[0048] Figure 1 This is the overall flowchart of the present invention.

[0049] Figure 2 This refers to the random mode discarding of image blocks in a multimodal image in this invention.

[0050] Figure 3 This is a schematic diagram illustrating the principle of the present invention.

[0051] Figure 4 This is a comparison of the results of this invention with other methods in downstream brain tumor image segmentation.

[0052] Figure 5 This is a comparison of the convergence speed of the present invention with other methods during downstream task fine-tuning. Detailed Implementation

[0053] In conjunction with the content of this invention, the following embodiments are provided for multimodal brain tumor image segmentation. These embodiments are implemented on a computer with an Intel(R) Core(TM) i9-9900K 3.60GHz CPU, an Nvidia GTX3060Ti GPU, and 32.0GB of memory. The programming language is Python, and the construction and training of the deep learning model are implemented through PyTorch.

[0054] Step 1: Establish a pre-trained dataset of multimodal images of brain tumor MRI and an image segmentation model;

[0055] This invention utilizes brain tumor data with four modalities obtained from the BraTS2023 public dataset for pre-training. This dataset contains 3D MRI images of 1251 brain tumor patients, with each patient having four modalities of scans: T1-weighted images, T2-weighted images, T1-enhanced images (T1c), and fluid attenuated inversion recovery sequences (FLAIR). The image segmentation model employs the UNet3D model, a common encoder-decoder structure used in 3D medical image segmentation.

[0056] Step 2: Sampling of overlapping regions in multimodal magnetic resonance images and enhancement based on random region and mode discarding;

[0057] Given four modalities of brain tumor images of a patient, denoted by C = 4, a sub-region is randomly extracted from the images. Let D = 96, H = 96, and W = 96 represent the size of this sub-region along the z-axis, y-axis, and x-axis, respectively. This sub-region is denoted as X1, and its size is C × D × H × W. A low-quality image X′1 is obtained by applying a random region and modality discarding strategy to X1. Specifically, X1 is divided into B = 2^16 distinct blocks, each 16 × 16 × 16 in size, and each modality within these blocks is discarded with probability p. Let... For the b-th block of X1, randomly generate a binary discard vector M. b ∈{0,1} C If each element is set to 0 with probability p = 0.85, then the output after random discarding is:

[0058] The output after randomly discarding all blocks in X1 according to the above rules is denoted as X′1. Similarly, a sub-region X2 is randomly obtained from the original image, where X2 and X1 have some spatial overlap and their Intersection over Union (IoU) is at least 10%. The above random region and modality discarding strategy is also applied to X2 to obtain the enhanced sub-region X′2.

[0059] Step 3: Image Reconstruction Task Based on Siamese Deep Learning Neural Network

[0060] Based on the deep learning network described above, two neural networks S and T with identical structures are built. These two networks constitute a Siamese deep learning network. The encoder of both networks has L = 4 downsampling layers, and the decoder has L upsampling layers, which can produce feature maps at L different scales.

[0061] The original image is reconstructed by inputting X′1 and the deep learning network S, and the reconstructed image is denoted as X′1. Similarly, when X′2 is input into the deep learning network S for reconstruction, its prediction result is... Using X1 and X2 as the gold standard for reconstruction, the loss of the reconstructed image is calculated as follows:

[0062]

[0063] Step 4: Self-distillation task based on voxel-level multi-scale feature extraction

[0064] N voxels are selected in the overlapping region of the two sub-images X1 and X2. Each voxel has a different random dropout mode in both sub-images, thus forming N voxel pairs. In this embodiment, N = 1024. Represents the nth voxel pair. From X1, and From X2. Each contains different modal information and local structural information. When an image X is input into T, its feature maps at L different scales in the decoder are denoted as follows: use Represents the l-th feature map The characteristics of the nth voxel, and the multiscale characteristics of this voxel are represented as follows:

[0065]

[0066] in This represents the feature concatenation operation. According to Formula 2, using... and The table represents the multi-scale features of pixel n in X1 obtained through networks T and S. Correspondingly, using... and The sub-table represents the multi-scale features obtained by processing pixel n in X2 through networks T and S. The dimension of these features is labeled C. m Its value is The sum of the dimensions is 992.

[0067] In order to and Projecting from a higher dimension to a lower dimension, based on the aforementioned Siamese network, we construct projectors for T and S respectively, denoted as g. T and g S Among them, g S It is implemented using a multilayer perceptron network, consisting of three fully connected layers. The first two layers have C... m = 992 neurons, each with a batch normalization layer and a ReLU activation function; the last layer contains K = 512 neurons and a Softmax activation function layer. g T The structure in g S Based on the existing structure, an additional centering layer was added before Softmax.

[0068] Will Enter g T The resulting projection vector is denoted as Will Enter g S The resulting projection vector is denoted as Similarly, and Enter g respectively T and g S The resulting projection vectors are denoted as follows: and The self-distillation loss defined based on the Siamese networks S and T is as follows:

[0069]

[0070] Among them, L ce This represents the cross-entropy loss.

[0071] Step 5: Pre-train the model by combining the image reconstruction task and the self-distillation task.

[0072] Based on Equations 1 and 3, the total pre-training loss function L is defined. pre :

[0073] L pre =L sd +λL rec #Formula 4

[0074] Where λ = 1.0 is a weighting coefficient used to control the relative magnitude between reconstruction loss and self-distillation loss. Using L... pre The gradient value of neural network S is calculated and updated using gradient descent. For neural network T, an exponential moving average method is used for updating. Specifically, the parameters of the Siamese networks S and T are denoted as θ. S and θ T , for θ T The update method is as follows:

[0075] θ T ←αθ T +(1-α)θ S #Formula5

[0076] Here, α represents the momentum coefficient, initially set to 0.99, which gradually increases to 1.0 during training. In updating the neural network T, the batch size is 2, the learning rate is set to 0.001, and the number of iterations is 60,000.

[0077] Step 6: Fine-tuning on downstream segmentation tasks

[0078] In the downstream task, a relatively small labeled dataset, BraTS-SSA, was selected for fine-tuning the pre-trained model. This dataset contains MRI images of 60 patients with brain tumors, aiming to segment the necrotic region, edema region, enhanced tumor region, and background of the brain tumor, a total of four categories. 80% of the dataset was randomly selected as the training set, and 20% as the validation set for experiments. The training set is denoted as... Where X i Y represents a training image. i This indicates its labeling, where M=48 is the number of samples in the training dataset. X i Inputting the segmentation model T, the predicted result is P. i =T(X) iThe parameters of the image segmentation model T are initialized using pre-trained weights. The loss function for the fine-tuning process is defined as:

[0079]

[0080] Where L ce It is the cross-entropy loss function, defined as:

[0081]

[0082] Where V = DHW represents the number of voxels, and C = 4 represents the number of categories. This represents the probability that the v-th pixel belongs to the c-th category. This represents the probability corresponding to the label. L dice This is the Dice loss function, which is defined as:

[0083]

[0084] Where ε = 10 -5 It is a relatively small number used to ensure numerical stability. In L tune Based on this, a relatively small learning rate (0.0002) is used, and the Adam optimizer is employed to update the parameters in T. The number of iterations is 10,000, thus completing the fine-tuning of the pre-trained model.

[0085] Figure 4 The image shows the segmentation results obtained by inputting a brain tumor MRI image from the validation set into the model pre-trained and fine-tuned using the above method, and the results obtained by other methods. Each gray area in the image represents the segmentation result of the tumor sub-region. The baseline method refers to training without using the pre-trained model as initialization. The last column shows the segmentation results obtained using the method proposed in this invention, which is closest to the true label. Figure 5 The curves show the performance changes of the method of this invention and other methods on the validation set during the training process of the downstream dataset. It can be seen that the present invention not only achieves better accuracy, but also greatly accelerates the convergence of the model, achieves better results in a shorter time, and reduces the time and computational cost required for fine-tuning of downstream tasks.

Claims

1. A self-supervised pre-training method for a multimodal medical image segmentation model, comprising the following steps: Step 1: Establish a pre-trained multimodal image dataset and image segmentation model: Collect unlabeled multimodal medical images, preprocess them, and use them as a pre-training dataset; select an image segmentation network with an encoder-decoder structure and use it as a pre-training model; Step 2: Sample overlapping regions in the multimodal pre-training dataset and perform augmentation based on random regions and modality dropout; Step 3: Construct a twin deep learning neural network to establish an image reconstruction task; Step 4: Perform voxel-level multi-scale feature extraction based on Siamese networks to construct a self-distillation task; Step 5: Pre-train the model by combining the image reconstruction task and the self-distillation task; Step 6: Fine-tune the pre-trained weights for the downstream segmentation task.

2. The self-supervised pre-training method for a multimodal medical image segmentation model as described in claim 1, characterized in that, In step 2, the method for sampling overlapping regions and enhancing the multimodal pre-trained image based on random regions and modality discarding is as follows: Let C be the number of modalities in the multimodal 3D images in the pre-training dataset. A sub-region is randomly extracted from the image, and its size along the z-axis, y-axis, and x-axis is represented by D, H, and W, respectively. This sub-region is denoted as X1, and its size is C×D×H×W. A low-quality image X′1 is obtained by applying a random region and modality discarding strategy to X1. Specifically, X1 is divided into B distinct blocks, and each modality in each block is discarded with probability p. Let... For the b-th block of X1, randomly generate a binary discard vector M. b ∈{0,1} C If each element is set to 0 with probability p, then the output after random discarding is: The output after randomly discarding all blocks in X1 according to the above rules is denoted as X′1; similarly, a sub-region X2 is randomly obtained from the original image, where X2 and X1 have a certain spatial overlap; the above random region and modal discarding strategy is also applied to X2 to obtain the enhanced sub-region X′2.

3. The self-supervised pre-training method for a multimodal medical image segmentation model as described in claim 1, characterized in that, The method for constructing the twin neural network and reconstructing the image in step 3 is as follows: A deep learning network S consisting of an encoder and a decoder contains a neural network T with the same structure; these two networks constitute a Siamese deep learning network, with the encoder having L downsampling layers and the decoder having L upsampling layers, resulting in L feature maps at different scales. The original image is reconstructed by inputting X′1 and the deep learning network S, and the reconstructed image is denoted as X′1. Using the same method, X′2 is input into the deep learning network S for reconstruction, and its prediction result is: Using X1 and X2 as the gold standard for reconstruction, the loss of the reconstructed image is calculated as follows:

4. The self-supervised pre-training method for a multimodal medical image segmentation model as described in claim 1, characterized in that, The implementation method of multi-scale feature extraction and self-distillation tasks in step 4 is as follows: N voxels are selected in the overlapping region of the two sub-images X1 and X2. Each voxel has a different random dropout mode in both sub-images, thus forming N voxel pairs; using Represents the nth voxel pair. From X1, and From X2; each contains different modal information and local structural information; when an image X is input into a neural network T, its feature maps at L different scales in the decoder are denoted as follows: use Represents the l-th feature map The characteristics of the nth voxel, and the multiscale characteristics of this voxel are represented as follows: Where ⊕ represents the feature concatenation operation, according to the above formula, using and These represent the multi-scale features of pixel n in X1 obtained through networks T and S, respectively; using and Let C represent the multi-scale features of pixel n in X2 obtained through networks T and S, respectively; the dimension of these features is labeled C. m Its value is The sum of dimensions; In order to and To project from a high dimension to a low dimension, based on the aforementioned Siamese network, a projector is constructed for T and S respectively, and these two projectors are denoted as g. T and g S ; where g S Implemented by a multilayer perceptron network, consisting of three fully connected layers; the first two layers have C m The first layer contains K neurons, each with a batch normalization layer and a ReLU activation function; the last layer contains K neurons and a Softmax activation function layer; g T The structure in g S Based on the existing structure, an additional centering layer was added before Softmax; Will Enter g T The resulting projection vector is denoted as Will Enter g S The resulting projection vector is denoted as Using the same method, and Enter g respectively T and g S The resulting projection vectors are denoted as follows: and The self-distillation loss defined based on the Siamese networks S and T is as follows: Among them, L ce This represents the cross-entropy loss.

5. The self-supervised pre-training method for a multimodal medical image segmentation model as described in claim 1, characterized in that, The method for pre-training the model in step 5, which combines the image reconstruction task and the self-distillation task, is as follows: Define the overall pre-training loss function L. pre : THE pre =L sd +λL rec Where λ is a weighting coefficient used to control the relative magnitude between reconstruction loss and self-distillation loss; using L pre The gradient value of neural network S is calculated and updated using gradient descent; for neural network T, the exponential moving average method is used for updating. The parameters of the Siamese networks S and T are denoted as θ. S and θ T , for θ T The update method is as follows: i T ←th T +(1-α)θ S Where α represents the momentum coefficient.

6. The self-supervised pre-training method for a multimodal medical image segmentation model as described in claim 1, characterized in that, The pre-trained weights in step 6 are fine-tuned in the downstream segmentation task as follows: The parameters of the image segmentation model T are initialized using pre-trained weights; the labeled dataset in the downstream task is denoted as... Where X i Y represents a training image. i This represents its label, where M is the number of samples in the training dataset; X... i Inputting the segmentation model T, the predicted result is P. i =T(X) i The loss function for the fine-tuning process is defined as: Where L ce It is the cross-entropy loss function, defined as: Where V = DHW represents the number of voxels, and C is the number of categories; This represents the probability that the v-th pixel belongs to the c-th category. L represents the probability corresponding to the label. dice This is the Dice loss function, which is defined as: Where ε is a constant used to ensure numerical stability; in L tune Based on this, the parameters in T are updated to complete the fine-tuning of the pre-trained model.

Images