A three-dimensional medical image segmentation method based on mask supervision strategy and efficient modeling

Abstract

The present application belongs to the field of medical image processing, and proposes a three-dimensional medical image segmentation method based on mask supervision strategy. In the current three-dimensional medical image segmentation field, there are technical problems such as excessive model parameters, network model modeling redundancy, and poor segmentation accuracy. In order to solve these problems, the present application designs an efficient modeling encoder, uses dynamic depth convolution and global context attention mechanism to extract rich semantic features of the target in the three-dimensional medical image, and achieves the effect of efficient modeling. A mask supervision strategy with only a small amount of calculation is used, and the generated semantic mask information graph is used for model supervision training, so that the model has more prior information about the image to guide segmentation, so as to achieve higher segmentation performance. The experimental results prove that the method proposed in the present application can achieve excellent results with very few parameters, realize efficient modeling, and significantly improve the segmentation performance.

Description

Technical Field

[0001] This invention belongs to the field of medical image processing, and in particular relates to a three-dimensional medical image segmentation method based on a mask-supervised strategy for efficient modeling. Background Technology

[0002] 3D medical image segmentation refers to applying semantic segmentation from computer vision to the modalities of 3D medical images, such as CT and MRI, to complete intensive prediction tasks such as organ segmentation, tumor segmentation, and lesion identification. Acquiring relevant segmentation data can provide effective assistance for medical diagnosis, surgical planning, and treatment course formulation. U-shaped convolutional neural networks are generally an effective method for completing medical image segmentation tasks. In recent years, models represented by "Dosovitskiy A, Beyer L, Kolesnikov A, et al. An image is worth 16x16 words: Transformers for image recognition at scale[J]. arXiv preprint arXiv: 2010.11929, 2020." have become the backbone models of visual networks, achieving remarkable results in downstream tasks such as semantic segmentation. Inspired by this, some methods of Vision Transformer have been transferred to the field of 3D medical image segmentation, resulting in a series of superior 3D medical image segmentation models, such as "Chen J, Lu Y, Yu Q, et al. Transunet: Transformers make strong encoders for medical image segmentation[J]. arXivpreprint arXiv: 2102.04306, 2021." and "Wang W, Chen C, Ding M, et al. Transbts: Multimodal brain tumor segmentation using transformer[C] / / Medical ImageComputing and Computer Assisted Intervention-MICCAI 2021: 24th International Conference, Strasbourg, France, September 27-October 1, 2021, Proceedings, Part I24. Springer International Publishing, 2021: 109-119.", which have achieved good performance in the segmentation field. However, while VisionTransformer has good global dependency modeling capabilities, it loses the biased induction of convolutional neural networks, which means that models based on the VisionTransformer architecture do not achieve ideal results in dense prediction tasks such as 3D medical image segmentation.Therefore, integrating the prior knowledge and biased induction of visual processing from convolutional neural networks into the Vision Transformer method is particularly important. The paper "Zhou HY, Guo J, Zhang Y, et al. nnFormer: Volumetric medical image segmentation via a 3D transformer[J]. IEEE Transactions on Image Processing, 2023" alternates between the Transformer's encoding module and the convolutional neural network's encoding module, employing local attention mechanisms in the low-level encoding stage and global attention mechanisms in the high-level encoding stage, combining the characteristics of both convolutional neural networks and Transformer models. However, models represented by nnFormer do not fully leverage the advantages of both, and introduce a huge number of parameters, leading to model redundancy and low modeling efficiency. Therefore, how to fully combine and better utilize the advantages of convolutional neural networks and Transformers, and adopt a more efficient modeling method to achieve higher medical image segmentation performance, is a problem that urgently needs to be solved. Summary of the Invention

[0003] This invention addresses the aforementioned technical problems by providing a highly efficient 3D medical image segmentation method based on a mask-supervised strategy. It utilizes dynamic deep convolution and global contextual attention mechanisms to extract rich semantic features of targets in 3D medical images, achieving efficient modeling. By employing a mask-supervised strategy that introduces only a minimal amount of computation, the model is trained under supervision using the generated semantic mask information graph. This allows the model to have more prior information about the image to guide segmentation, resulting in higher segmentation performance.

[0004] To achieve the above objectives, the technical solution of the present invention is as follows:

[0005] A highly efficient 3D medical image segmentation method based on a mask-supervised strategy includes the following steps:

[0006] Step 1: Collect a 3D medical image segmentation dataset and perform data preprocessing;

[0007] Step 2: Construct an efficient 3D medical image segmentation network based on a mask-supervised strategy;

[0008] A highly efficient 3D medical image segmentation network based on a mask-supervised strategy is proposed. The overall structure includes a patch embedding layer, a high-efficiency modeling encoder, a mask computation module, a mask supervision branch, a decoder, and a segmentation output head. Furthermore, the loss function of the 3D medical image segmentation network is designed based on the outputs of the mask supervision branch and the segmentation output head. The specific steps are as follows:

[0009] Step 2.1: Build the patch embedding layer

[0010] The patch embedding layer uses convolutional coding to generate feature maps from the preprocessed 3D medical image in step 1, and outputs an image patch encoded feature map. The input 3D medical image is divided into image patches of equal size through a sliding window mechanism, and the image patches are then calculated using linear projection to obtain the embedding feature vectors. The embedding feature vectors are rearranged according to the relative positions of the image patches to form the embedding feature map. The image patch embedding process is shown in the following formula (1):

[0011] PatchEmbedding(img)=[patch1;…;patch n ]·W#(1)

[0012] Among them, the PatchEmbedding(img) sequence is an embedding feature map sequence generated based on the img image; is the weight matrix of the image embedding layer; patch is the image patch, the number of which is N = H × W × D, and the dimension of the embedding vector is D; the patch embedding layer is implemented using a three-dimensional convolutional layer with a kernel size of k, a stride of s, an input channel equal to the number of channels in the patch, and an output channel of E.

[0013] Step 2.2 Constructing an efficient modeling encoder

[0014] The high-efficiency modeling encoder comprises four stages, each consisting of a 3D high-efficiency modeling module with varying repetition counts. This module progressively encodes and downsamples the feature map output from the patch embedding layer. The repetition counts of the 3D high-efficiency modeling module in the four stages are [3, 3, 5, 2]. After each stage, the size of the output feature map is halved. The number of channels in the output feature maps of the four stages is expanded to [32, 64, 160, 320]. The 3D high-efficiency modeling module in the encoder fuses the global contextual information representation of the 3D medical image with the feature map input to the 3D high-efficiency modeling module, performing dynamic depthwise separable convolution. The specific computation process of the 3D high-efficiency modeling module includes two main stages: the first stage involves calculating the global contextual information representation from the input feature map using a global contextual information representation layer; the second stage involves residual concatenation between the global contextual information representation and the input feature map, followed by computation through an image space convolution layer to obtain the output. The specific method is as follows:

[0015] (2.2.1) First, the feature map input to the 3D efficient modeling module is processed by the global context information representation layer to obtain the global context information representation. The global context information representation layer constructs a maximum bottleneck structure using three linear layers to obtain the preliminary global context information representation. Two linear layers are used to compress the dimension to obtain two compressed feature vectors. The maximum value function is used to make these two compressed feature vectors compete to obtain more useful image information. Finally, a linear layer is used to restore the dimension to obtain the preliminary global context information representation. The calculation is shown in formula (2):

[0016] G′=W G max(W g1 X, W g2 X)#(2)

[0017] in, This represents the sequence of feature vectors representing the input feature map; This represents the computational information of the global context obtained from the calculation, where C in This represents the dimension of the feature map input to the global context information representation layer; The weight parameter matrix is ​​used to compress the dimension of the two linear layers, which is used to perform spatial compression on the input features. r is the dimension scaling factor. The max function is a maximum value operation, which makes the compressed feature vectors output by the two linear layers compete with each other, and the largest value is selected for output. The linear layer weight parameter matrix for feature dimension recovery; The initial global context calculation results are obtained; then, feature similarity adaptive operation is performed on G′. First, the input feature map is multiplied with its own channel compression vector to obtain the similarity matrix. Then, the similarity matrix is ​​normalized according to the mean, variance and two learnable parameters of the similarity matrix. Finally, the initial global context information representation is multiplied with the normalized similarity matrix to obtain the final global context information representation. The calculation is shown in formula (3):

[0018]

[0019] Where, μ S With σ S Let represent the mean and variance of the similarity matrix, respectively. The learnable parameter γ represents the scaling factor of the similarity matrix. The deviation of the similarity matrix;

[0020] (2.2.2) After obtaining the global context representation information, it is residually connected with the input feature map, and then passed through the image space convolutional layer; the image space convolutional layer includes three-dimensional depth separable convolution and three-dimensional point convolution; the three-dimensional depth separable convolution uses a convolutional kernel with dynamic parameters to adapt to the input features; k convolutional kernels are pre-constructed, and the attention matrix of the k convolutional kernel parameters is calculated using the SE mechanism; the attention matrix is ​​multiplied and accumulated with the k convolutional kernel parameters to obtain the final convolutional kernel with dynamic parameters; the SE mechanism is a combination of average pooling layer, linear layer, ReLU activation function and Softmax function, and its calculation formula is shown in formula (4):

[0021]

[0022] Where Softmax is the Softmax function; is the weight parameter matrix of the linear layer, where p is the kernel size, m is the kernel scaling factor, and avgPool is the average pooling operation; For convolution kernel parameters, The final generated dynamic convolution kernel parameters are used; the global context information is represented by the depthwise separable convolution using the dynamic parameter convolution kernel, followed by 3D point convolution for dimensional expansion; its calculation is shown in formula (5):

[0023]

[0024] in, PConv is a 3D point convolution that uses depthwise separable convolutions with dynamic kernels. The output feature map of the 3D efficient modeling module, where C outThe dimension of the feature map output by the 3D high-efficiency modeling module;

[0025] Step 2.3 Construct the mask calculation module and mask supervision branch

[0026] The mask calculation module and the mask supervision branch together constitute the mask supervision strategy. The mask calculation module includes four stages, using the calculation results of the three-dimensional efficient modeling module in each of the four stages in step 2.2 as input. The mask attention mechanism is used to calculate the feature map and semantic mask information map of the three-dimensional efficient modeling module that are passed to the next stage for the input feature map. The semantic mask information map has the same size as the feature map of the corresponding three-dimensional efficient modeling module in each stage, and the number of channels is uniformly the number of categories (CLS).

[0027] In the mask calculation module, the input feature map is first normalized, and then the normalized feature matrix is ​​passed through three linear layers to obtain the query matrix, key matrix and value matrix of the corresponding feature matrix; the calculation is as shown in formula (6):

[0028]

[0029] in, These are the query matrix, key matrix, and value matrix, respectively, where CLS represents the number of categories in the 3D medical image segmentation dataset. This is the output feature map of the 3D efficient modeling module; after obtaining the three matrices, the query matrix SM will be used. Query The output is a semantic mask information map; then the query matrix and the key matrix are multiplied together and the semantic score is obtained through the Softmax function. The semantic score is then multiplied with the value matrix and linearly scaled through a linear layer with a learnable parameter α. The feature map is then residually connected to the feature map input from the mask calculation module to obtain the feature map after attention calculation. The calculation is as shown in formula (7):

[0030] Y′=α·W se Softmax(SM) Query SM Key T )SM value )+Y#(7)

[0031] Where α is a learnable parameter for linear scaling of the features. For linear layer parameter matrices, The feature map calculated by the mask calculation module will be passed to the next stage for further calculation;

[0032] In the mask supervision branch, the semantic mask information map of each stage is trilinearly interpolated and added to the semantic mask information map of the next stage to finally obtain a semantic mask map with a size of the image size in the 3D medical image dataset and a channel number of CLS. The semantic mask information map is compared with the test set label of the preprocessed 3D medical image dataset to calculate the loss and participate in supervising the entire model training.

[0033] Step 2.4 Constructing the decoder and splitter output head

[0034] The decoder mainly consists of a residual network computation block and an upsampling module. The feature maps output from each stage of the efficient modeling encoder in step 2.2 are first processed through the residual network computation block and then concatenated with the output feature maps from the previous stage's 3D efficient modeling module via skip connections. Upsampling is performed level by level, and the maps are iteratively passed to the next stage to obtain a feature map with the same output scale as the patch embedding layer. The residual network computation block includes two convolutional layers with residual connections, and the kernel size is set to 3. Finally, the decoder obtains a feature map of size [missing information]. The feature map; the upsampling module is implemented using transposed convolutions with an upsampling rate of 2;

[0035] The segmentation output head reshapes the feature map output by the decoder into the image size of the preprocessed 3D medical image, and compresses the number of image channels into the number of classification categories through convolution to obtain a probability prediction map. Finally, it obtains a classification prediction map through an argmax operation. The segmentation output head consists of a convolutional layer with a kernel size of 3 and a stride of 1. The output feature map of the decoder is input into the segmentation output head to obtain the prediction result. The output of the segmentation output head will be used for the training and inference of the 3D medical image segmentation network model.

[0036] Step 2.5 Design the loss function

[0037] The shapes of the output prediction feature map of the masked supervision branch in step 2.3 and the output feature map of the final segmentation output head in step 2.4 are both... Where CLS represents the number of categories; the two feature maps mentioned above will be compared together with the labels in the 3D medical image dataset to calculate the loss; the weighted sum of Dice loss and modified cross-entropy is used as the loss function, and the calculation is shown in Equation (8):

[0038]

[0039] Where A represents the true label; B represents the predicted label; y ic The sign function is y when the true class of sample i is equal to c. ic Select 1 otherwise select 0; p icβ represents the probability value of the corresponding pixel prediction; N represents the weight of Dice Loss; since the model has two outputs, the segmentation output head and the mask supervision branch, these two results are compared with the label during training to calculate the corresponding DiceCELoss. At the same time, a hyperparameter ρ is introduced to adjust the weight ratio of the two model losses. The overall loss function calculation formula of the efficient modeling three-dimensional medical image segmentation network based on the mask supervision strategy is shown in formula (9):

[0040] Loss=DiceCELoss(P,T)+ρDiceCELoss(S,T)#(9)

[0041] in, Labels in a 3D medical image dataset. This is the output feature map of the mask supervision branch in step 2.3; The feature map output by the segmentation output head in step 2.4;

[0042] Step 3: Design a training strategy. Use the training set data in the 3D medical image segmentation dataset to train the weight parameters of an efficient 3D medical image segmentation network based on a mask supervision strategy, and use the trained network model for data inference in the test set of the 3D image segmentation dataset.

[0043] Before training, the training batch size is determined. In each training iteration, images of the batch size are selected from the dataset processed in step 1 and input into the efficient 3D medical image segmentation network based on the mask supervision strategy constructed in step 2. During training, the Adam optimizer is used to optimize the weight parameters through back gradient propagation, thereby completing one training iteration. When the number of training iterations reaches the maximum number of iterations, training is terminated, and the weights and parameters of the optimal model are obtained.

[0044] When the model performs inference, only the output of the segmentation output head is used for prediction. The test set data is input into the efficient 3D medical image segmentation network based on the mask supervision strategy constructed in step 2, and the weights and parameters of the optimal model trained on the training set are used as the weights and parameters of the model constructed in step 2. Each voxel in the test set data completes classification inference, and the segmentation of the corresponding 3D medical image can be completed.

[0045] The specific method for data preprocessing in step 1 is as follows:

[0046] First, intensity clipping is performed on the soft tissue of the dataset, ranging from [-175, 200]; second, the image is randomly rotated ([-30°, 30°]); finally, intensity normalization is performed on the 99% intensity quantile of the image.

[0047] The beneficial effects of this invention are:

[0048] First, the network model uses a three-dimensional high-efficiency modeling module to integrate global dependency information into the convolution process, thereby achieving high performance in medical image segmentation tasks.

[0049] Second, the 3D efficient modeling module of the network model uses dynamic depth-separable convolution, which enables the model to construct parameter-specific convolution kernels at the sample level. This helps the model understand the differences between samples and improves the model's generalization ability.

[0050] Third, the network model adopts a mask supervision strategy, which supervises the model training by using the semantic mask information graph obtained by the mask calculation module. During the training process, it provides prior semantic information to the feature maps of the corresponding levels, enabling the model to learn potential knowledge better and achieve better segmentation results. Attached Figure Description

[0051] Figure 1(a) is a schematic diagram of the three-dimensional high-efficiency modeling module in this invention, Figure 1(b) is a schematic diagram of the image space convolution layer in the three-dimensional high-efficiency modeling module, and Figure 1(c) is a schematic diagram of the global context information representation layer in the three-dimensional high-efficiency modeling module.

[0052] Figure 2 This is a schematic diagram of the mask calculation module in this invention;

[0053] Figure 3 This is an overall architecture diagram of the efficient 3D medical image segmentation method based on mask supervision strategy of the present invention; wherein For network-related computation modules, → represents feature maps. For splicing operations, For addition operation, This is a trilinear interpolation. Detailed Implementation

[0054] The present invention will be further described in detail below with reference to the accompanying drawings.

[0055] This invention provides a method for efficient modeling of 3D medical images based on a mask-supervised strategy, comprising the following steps:

[0056] Step 1: Collect a medical image segmentation dataset. First, perform intensity cropping on the soft tissue in the dataset, with a range of [-175, 200]. Second, randomly rotate the image ([-30°, 30°]). Finally, normalize the intensity of the image at 99% intensity quantiles.

[0057] Step 2: Construct an efficient 3D medical image segmentation network based on a mask-supervised strategy;

[0058] A highly efficient 3D medical image segmentation network based on a mask-supervised strategy is proposed. The overall structure includes a patch embedding layer, a high-efficiency modeling encoder, a mask computation module, a mask supervision branch, a decoder, and a segmentation output head. Furthermore, the loss function of the 3D medical image segmentation network is designed based on the outputs of the mask supervision branch and the segmentation output head. The specific steps are as follows:

[0059] Step 2.1: Build the patch embedding layer

[0060] The patch embedding layer uses convolutional coding to generate feature maps from the preprocessed 3D medical image in step 1, and outputs an image patch encoded feature map. The input 3D medical image is divided into image patches of equal size through a sliding window mechanism, and the image patches are then calculated using linear projection to obtain the embedding feature vectors. The embedding feature vectors are rearranged according to the relative positions of the image patches to form the embedding feature map. The image patch embedding process is shown in the following formula (1):

[0061] PatchEmbedding(img)=[patch1;…;patch n ]·W#(1)

[0062] Among them, the PatchEmbedding(img) sequence is an embedding feature map sequence generated based on the img image; is the weight matrix of the image embedding layer; patch is the image patch, the number of which is N = H × W × D, and the dimension of the embedding vector is D; the patch embedding layer is implemented using a three-dimensional convolutional layer with a kernel size of k, a stride of s, an input channel equal to the number of channels in the patch, and an output channel of E.

[0063] Step 2.2 Constructing an efficient modeling encoder

[0064] The high-efficiency modeling encoder comprises four stages, each consisting of a 3D high-efficiency modeling module with varying repetition counts. This module progressively encodes and downsamples the feature map output from the patch embedding layer. The repetition counts of the 3D high-efficiency modeling module in the four stages are [3, 3, 5, 2]. After each stage, the size of the output feature map is halved. The number of channels in the output feature maps of the four stages is expanded to [32, 64, 160, 320]. The 3D high-efficiency modeling module in the encoder fuses the global contextual information representation of the 3D medical image with the feature map input to the 3D high-efficiency modeling module, performing dynamic depthwise separable convolution. The specific computation process of the 3D high-efficiency modeling module includes two main stages: the first stage involves calculating the global contextual information representation from the input feature map using a global contextual information representation layer; the second stage involves residual concatenation between the global contextual information representation and the input feature map, followed by computation through an image space convolution layer to obtain the output. The specific method is as follows:

[0065] (2.2.1) First, the feature map input to the 3D efficient modeling module is processed by the global context information representation layer to obtain the global context information representation. The global context information representation layer constructs a maximum bottleneck structure using three linear layers to obtain the preliminary global context information representation. Two linear layers are used to compress the dimension to obtain two compressed feature vectors. The maximum value function is used to make these two compressed feature vectors compete to obtain more useful image information. Finally, a linear layer is used to restore the dimension to obtain the preliminary global context information representation. The calculation is shown in formula (2):

[0066] G′=W G max(W g1 X, W g2 X)#(2)

[0067] in, This represents the sequence of feature vectors representing the input feature map; This represents the computational information of the global context obtained from the calculation, where C in This represents the dimension of the feature map input to the global context information representation layer; The weight parameter matrix is ​​used to compress the dimension of the two linear layers, which is used to perform spatial compression on the input features. r is the dimension scaling factor. The max function is a maximum value operation, which makes the compressed feature vectors output by the two linear layers compete with each other, and the largest value is selected for output. The linear layer weight parameter matrix for feature dimension recovery; The initial global context calculation results are obtained; then, feature similarity adaptive operation is performed on G′. First, the input feature map is multiplied with its own channel compression vector to obtain the similarity matrix. Then, the similarity matrix is ​​normalized according to the mean, variance and two learnable parameters of the similarity matrix. Finally, the initial global context information representation is multiplied with the normalized similarity matrix to obtain the final global context information representation. The calculation is shown in formula (3):

[0068]

[0069] Where, μ S With σ S Let represent the mean and variance of the similarity matrix, respectively. The learnable parameter γ represents the scaling factor of the similarity matrix. The deviation of the similarity matrix;

[0070] (2.2.2) After obtaining the global context representation information, it is residually connected with the input feature map, and then passed through the image space convolutional layer; the image space convolutional layer includes three-dimensional depth separable convolution and three-dimensional point convolution; the three-dimensional depth separable convolution uses a convolutional kernel with dynamic parameters to adapt to the input features; k convolutional kernels are pre-constructed, and the attention matrix of the k convolutional kernel parameters is calculated using the SE mechanism; the attention matrix is ​​multiplied and accumulated with the k convolutional kernel parameters to obtain the final convolutional kernel with dynamic parameters; the SE mechanism is a combination of average pooling layer, linear layer, ReLU activation function and Softmax function, and its calculation formula is shown in formula (4):

[0071]

[0072] Where Softmax is the Softmax function; is the weight parameter matrix of the linear layer, where p is the kernel size, m is the kernel scaling factor, and avgPool is the average pooling operation; For convolution kernel parameters, The final generated dynamic convolution kernel parameters are used; the global context information is represented by the depthwise separable convolution using the dynamic parameter convolution kernel, followed by 3D point convolution for dimensional expansion; its calculation is shown in formula (5):

[0073]

[0074] in, PConv is a 3D point convolution that uses depthwise separable convolutions with dynamic kernels. The output feature map of the 3D efficient modeling module, where C outThe dimension of the feature map output by the 3D high-efficiency modeling module;

[0075] Step 2.3 Construct the mask calculation module and mask supervision branch

[0076] The mask calculation module and the mask supervision branch together constitute the mask supervision strategy. The mask calculation module includes four stages, using the calculation results of the three-dimensional efficient modeling module in each of the four stages in step 2.2 as input. The mask attention mechanism is used to calculate the feature map and semantic mask information map of the three-dimensional efficient modeling module that are passed to the next stage for the input feature map. The semantic mask information map has the same size as the feature map of the corresponding three-dimensional efficient modeling module in each stage, and the number of channels is uniformly the number of categories (CLS).

[0077] In the mask calculation module, the input feature map is first normalized, and then the normalized feature matrix is ​​passed through three linear layers to obtain the query matrix, key matrix and value matrix of the corresponding feature matrix; the calculation is as shown in formula (6):

[0078]

[0079] in, These are the query matrix, key matrix, and value matrix, respectively, where CLS represents the number of categories in the 3D medical image segmentation dataset. This is the output feature map of the 3D efficient modeling module; after obtaining the three matrices, the query matrix SM will be used. Query The output is a semantic mask information map; then the query matrix and the key matrix are multiplied together and the semantic score is obtained through the Softmax function. The semantic score is then multiplied with the value matrix and linearly scaled through a linear layer with a learnable parameter α. The feature map is then residually connected to the feature map input from the mask calculation module to obtain the feature map after attention calculation. The calculation is as shown in formula (7):

[0080] Y′=α·W se Softmax(SM) Query SM Key T )SM Value )+Y#(7)

[0081] Where α is a learnable parameter for linear scaling of the features. For linear layer parameter matrices, The feature map calculated by the mask calculation module will be passed to the next stage for further calculation;

[0082] In the mask supervision branch, the semantic mask information map of each stage is trilinearly interpolated and added to the semantic mask information map of the next stage to finally obtain a semantic mask map with a size of the image size in the 3D medical image dataset and a channel number of CLS. The semantic mask information map is compared with the test set label of the preprocessed 3D medical image dataset to calculate the loss and participate in supervising the entire model training.

[0083] Step 2.4 Constructing the decoder and splitter output head

[0084] The decoder mainly consists of a residual network computation block and an upsampling module. The feature maps output from each stage of the efficient modeling encoder in step 2.2 are first processed through the residual network computation block and then concatenated with the output feature maps from the previous stage's 3D efficient modeling module via skip connections. Upsampling is performed level by level, and the maps are iteratively passed to the next stage to obtain a feature map with the same output scale as the patch embedding layer. The residual network computation block includes two convolutional layers with residual connections, and the kernel size is set to 3. Finally, the decoder obtains a feature map of size [missing information]. The feature map; the upsampling module is implemented using transposed convolutions with an upsampling rate of 2;

[0085] The segmentation output head reshapes the feature map output by the decoder into the image size of the preprocessed 3D medical image, and compresses the number of image channels into the number of classification categories through convolution to obtain a probability prediction map. Finally, it obtains a classification prediction map through an argmax operation. The segmentation output head consists of a convolutional layer with a kernel size of 3 and a stride of 1. The output feature map of the decoder is input into the segmentation output head to obtain the prediction result. The output of the segmentation output head will be used for the training and inference of the 3D medical image segmentation network model.

[0086] Step 2.5 Design the loss function

[0087] The shapes of the output prediction feature map of the masked supervision branch in step 2.3 and the output feature map of the final segmentation output head in step 2.4 are both... Where CLS represents the number of categories; the two feature maps mentioned above will be compared together with the labels in the 3D medical image dataset to calculate the loss; the weighted sum of Dice loss and modified cross-entropy is used as the loss function, and the calculation is shown in Equation (8):

[0088]

[0089] Where A represents the true label; B represents the predicted label; y ic The sign function is y when the true class of sample i is equal to c. ic Select 1 otherwise select 0; p icβ represents the probability value of the corresponding pixel prediction; N represents the weight of Dice Loss; since the model has two outputs, the segmentation output head and the mask supervision branch, these two results are compared with the label during training to calculate the corresponding DiceCELoss. At the same time, a hyperparameter ρ is introduced to adjust the weight ratio of the two model losses. The overall loss function calculation formula of the efficient modeling three-dimensional medical image segmentation network based on the mask supervision strategy is shown in formula (9):

[0090] Loss=DiceCELoss(P,T)+pDiceCELoss(S,T)#(9)

[0091] in, Labels in a 3D medical image dataset. This is the output feature map of the mask supervision branch in step 2.3; The feature map output by the segmentation output head in step 2.4;

[0092] Step 3: Design a training strategy. Use the training set data in the 3D medical image segmentation dataset to train the weight parameters of an efficient 3D medical image segmentation network based on a mask supervision strategy, and use the trained network model for data inference in the test set of the 3D image segmentation dataset.

[0093] Before training, the training batch size is determined. In each training iteration, images of the batch size are selected from the dataset processed in step 1 and input into the efficient 3D medical image segmentation network based on the mask supervision strategy constructed in step 2. During training, the Adam optimizer is used to optimize the weight parameters through back gradient propagation, thereby completing one training iteration. When the number of training iterations reaches the maximum number of iterations, training is terminated, and the weights and parameters of the optimal model are obtained.

[0094] When the model performs inference, only the output of the segmentation output head is used for prediction. The test set data is input into the efficient 3D medical image segmentation network based on the mask supervision strategy constructed in step 2, and the weights and parameters of the optimal model trained on the training set are used as the weights and parameters of the model constructed in step 2. Each voxel in the test set data completes classification inference, and the segmentation of the corresponding 3D medical image can be completed.

[0095] The examples were trained and tested on the AMOS2022 and WORD datasets, respectively. The hyperparameter settings for training the network are shown in Table 1.

[0096] Table 1 Hyperparameter settings for training the network

[0097]

[0098] The examples were tested on the test sets of two datasets and compared with existing high-performance medical image segmentation models, as shown in Table 2; where Params represents the number of model parameters. AMOS2022 represents the Mean Dice results tested on the AMOS2022 dataset, and WORD represents the Mean Dice results tested on the WORD dataset.

[0099] Table 2 Comparison with existing excellent models

[0100]

[0101] Experimental results show that the method proposed in this invention can achieve excellent results with very few parameters, realize efficient modeling, and significantly improve segmentation performance.

Claims

1. A highly efficient modeling method for 3D medical image segmentation based on a mask-supervised strategy, characterized in that, Includes the following steps: Step 1: Collect a 3D medical image segmentation dataset and perform data preprocessing; Step 2: Construct an efficient 3D medical image segmentation network based on a mask-supervised strategy; A highly efficient 3D medical image segmentation network based on a mask-supervised strategy is proposed. The overall structure includes a patch embedding layer, a high-efficiency modeling encoder, a mask computation module, a mask supervision branch, a decoder, and a segmentation output head. Furthermore, the loss function of the 3D medical image segmentation network is designed based on the outputs of the mask supervision branch and the segmentation output head. The specific steps are as follows: Step 2.1: Build the patch embedding layer The patch embedding layer uses convolutional coding to generate feature maps from the 3D medical images after the preprocessing in step 1, and the output is an image patch-coded feature map; The input 3D medical image is divided into equal-sized image patches using a sliding window mechanism. The image patches are then used to calculate the embedding feature vectors through linear projection. The embedding feature vectors are rearranged according to the relative positions of the image patches to form an embedding feature map. The image patch embedding process is shown in the following formula (1): PatchEmbedding(img)=[patch1;…;patch n ]·W#(1) Among them, the PatchEmbedding(img) sequence is an embedding feature map sequence generated based on the img image; is the weight matrix of the image embedding layer; patch is the image patch, the number of which is N = H × W × D, and the dimension of the embedding vector is D; the patch embedding layer is implemented using a three-dimensional convolutional layer with a kernel size of k, a stride of s, an input channel equal to the number of channels in the patch, and an output channel of E. Step 2.2 Constructing an efficient modeling encoder The high-efficiency modeling encoder comprises four stages, each consisting of a 3D high-efficiency modeling module with varying repetition counts. This module progressively encodes and downsamples the feature map output from the patch embedding layer. The repetition counts of the 3D high-efficiency modeling module in each of the four stages are [3, 3, 5, 2]. After each stage, the size of the output feature map is halved. The number of channels in the output feature maps of the four stages is expanded to [32, 64, 160, 320]. The 3D high-efficiency modeling module in the encoder fuses the global contextual information representation of the 3D medical image with the feature map input to the 3D high-efficiency modeling module, performing dynamic depthwise separable convolution. The specific computation process of the 3D high-efficiency modeling module includes two stages: the first stage involves calculating the global contextual information representation from the input feature map using a global contextual information representation layer; the second stage involves residual concatenation between the global contextual information representation and the input feature map, followed by computation through an image space convolution layer to obtain the output. The specific method is as follows: (2.2.1) First, the feature map input to the 3D efficient modeling module is processed by the global context information representation layer to obtain the global context information representation. The global context information representation layer constructs a maximum bottleneck structure using three linear layers to obtain the preliminary global context information representation. Two linear layers are used to compress the dimension to obtain two compressed feature vectors. The maximum value function is used to make these two compressed feature vectors compete to obtain more useful image information. Finally, a linear layer is used to restore the dimension to obtain the preliminary global context information representation. The calculation is shown in formula (2): G′=W G max(W g1 X,W g2 X)#(2) in, This represents the sequence of feature vectors representing the input feature map; This represents the computational information of the global context obtained from the calculation, where C in W represents the dimension of the feature map input to the global context information representation layer. g1 , The weight parameter matrix is ​​used to compress the dimension of the two linear layers, which is used to perform spatial compression on the input features. r is the dimension scaling factor. The max function is a maximum value operation, which makes the compressed feature vectors output by the two linear layers compete with each other, and the largest value is selected for output. The linear layer weight parameter matrix for feature dimension recovery; The initial global context calculation results are obtained; then, feature similarity adaptive operation is performed on G′. First, the input feature map is multiplied with its own channel compression vector to obtain the similarity matrix. Then, the similarity matrix is ​​normalized according to the mean, variance and two learnable parameters of the similarity matrix. Finally, the initial global context information representation is multiplied with the normalized similarity matrix to obtain the final global context information representation. The calculation is shown in formula (3): Where, μ S With σ s Let represent the mean and variance of the similarity matrix, respectively. The learnable parameter γ represents the scaling factor of the similarity matrix. The deviation of the similarity matrix; (2.2.2) After obtaining the global context representation information, it is residually connected with the input feature map, and then passed through the image space convolutional layer; the image space convolutional layer includes three-dimensional depth separable convolution and three-dimensional point convolution; the three-dimensional depth separable convolution uses a convolutional kernel with dynamic parameters to adapt to the input features; k convolutional kernels are pre-constructed, and the attention matrix of the k convolutional kernel parameters is calculated using the SE mechanism; the attention matrix is ​​multiplied and accumulated with the k convolutional kernel parameters to obtain the final convolutional kernel with dynamic parameters; the SE mechanism is a combination of average pooling layer, linear layer, ReLU activation function and Softmax function, and its calculation formula is shown in formula (4): Where Softmax is the Softmax function; is the weight parameter matrix of the linear layer, where p is the kernel size, m is the kernel scaling factor, and avgPool is the average pooling operation; For convolution kernel parameters, The final generated dynamic convolution kernel parameters are used; the global context information is represented by the depthwise separable convolution using the dynamic parameter convolution kernel, followed by 3D point convolution for dimensional expansion; its calculation is shown in formula (5): in, PConv is a 3D point convolution that uses depthwise separable convolutions with dynamic kernels. The output feature map of the 3D efficient modeling module, where C out The dimension of the feature map output by the 3D high-efficiency modeling module; Step 2.3 Construct the mask calculation module and mask supervision branch The mask calculation module and the mask supervision branch together constitute the mask supervision strategy. The mask calculation module includes four stages, using the calculation results of the three-dimensional efficient modeling module in each of the four stages in step 2.2 as input. The mask attention mechanism is used to calculate the feature map and semantic mask information map of the three-dimensional efficient modeling module that are passed to the next stage for the input feature map. The semantic mask information map has the same size as the feature map of the corresponding three-dimensional efficient modeling module in each stage, and the number of channels is uniformly the number of categories (CLS). In the mask calculation module, the input feature map is first normalized, and then the normalized feature matrix is ​​passed through three linear layers to obtain the query matrix, key matrix and value matrix of the corresponding feature matrix; the calculation is as shown in formula (6): in, These are the query matrix, key matrix, and value matrix, respectively, where CLS represents the number of categories in the 3D medical image segmentation dataset. This is the output feature map of the 3D efficient modeling module; after obtaining the three matrices, the query matrix SM will be used. Query The output is a semantic mask information map; then the query matrix and the key matrix are multiplied together and the semantic score is obtained through the Softmax function. The semantic score is then multiplied with the value matrix and linearly scaled through a linear layer with a learnable parameter α. The feature map is then residually connected to the feature map input from the mask calculation module to obtain the feature map after attention calculation. The calculation is as shown in formula (7): Y′=α·W se (Softmax(SM Query SM Key T )SM Value )+Y#(7) Where α is a learnable parameter for linear scaling of the features. For linear layer parameter matrices, The feature map calculated by the mask calculation module will be passed to the next stage for further calculation; In the mask supervision branch, the semantic mask information map of each stage is trilinearly interpolated and added to the semantic mask information map of the next stage to finally obtain a semantic mask map with a size of the image size in the 3D medical image dataset and a channel number of CLS. The semantic mask information map is compared with the test set label of the preprocessed 3D medical image dataset to calculate the loss and participate in supervising the entire model training. Step 2.4 Constructing the decoder and splitter output head The decoder consists of a residual network computation block and an upsampling module. The feature maps output from each stage of the efficient modeling encoder in step 2.2 are first processed through the residual network computation block and then concatenated with the output feature maps from the previous stage's 3D efficient modeling module via skip connections. Upsampling is performed level by level, and the maps are iteratively passed to the next stage to obtain a feature map with the same output scale as the patch embedding layer. The residual network computation block includes two convolutional layers with residual connections, and the kernel size is set to 3. Finally, the decoder obtains a feature map of size [missing information]. The feature map; the upsampling module is implemented using transposed convolutions with an upsampling rate of 2; The segmentation output head reshapes the feature map output by the decoder into the image size of the preprocessed 3D medical image, and compresses the number of image channels into the number of classification categories through convolution to obtain a probability prediction map. Finally, it obtains a classification prediction map through an argmax operation. The segmentation output head consists of a convolutional layer with a kernel size of 3 and a stride of 1. The output feature map of the decoder is input into the segmentation output head to obtain the prediction result. The output of the segmentation output head will be used for the training and inference of the 3D medical image segmentation network model. Step 2.5 Design the loss function The shapes of the output prediction feature map of the masked supervision branch in step 2.3 and the output feature map of the final segmentation output head in step 2.4 are both... Where CLS represents the number of categories; the two feature maps mentioned above will be compared together with the labels in the 3D medical image dataset to calculate the loss; the weighted sum of Dice loss and modified cross-entropy is used as the loss function, and the calculation is shown in Equation (8): Where A represents the true label; B represents the predicted label; y ic The sign function is y when the true class of sample i is equal to c. ic Select 1 otherwise select 0; p ic β represents the probability value of the corresponding pixel prediction; N represents the weight of Dice Loss; since the model has two outputs, the segmentation output head and the mask supervision branch, these two results are compared with the label during training to calculate the corresponding DiceCELoss. At the same time, a hyperparameter ρ is introduced to adjust the weight ratio of the two model losses. The overall loss function calculation formula of the efficient modeling three-dimensional medical image segmentation network based on the mask supervision strategy is shown in formula (9): Loss=DiceCELoss(P,T)+ρDiceCELoss(S,T)#(9) in, Labels in a 3D medical image dataset. This is the output feature map of the mask supervision branch in step 2.3; The feature map output by the segmentation output head in step 2.4; Step 3: Design a training strategy. Use the training set data in the 3D medical image segmentation dataset to train the weight parameters of an efficient 3D medical image segmentation network based on a mask supervision strategy, and use the trained network model for data inference in the test set of the 3D image segmentation dataset. Before training, the training batch size is determined. In each training iteration, images of the batch size are selected from the dataset processed in step 1 and input into the efficient 3D medical image segmentation network based on the mask supervision strategy constructed in step 2. During training, the Adam optimizer is used to optimize the weight parameters through back gradient propagation, thereby completing one training iteration. When the number of training iterations reaches the maximum number of iterations, training is terminated, and the weights and parameters of the optimal model are obtained. When the model performs inference, only the output of the segmentation output head is used for prediction. The test set data is input into the efficient 3D medical image segmentation network based on the mask supervision strategy constructed in step 2, and the weights and parameters of the optimal model trained on the training set are used as the weights and parameters of the model constructed in step 2. Each voxel in the test set data completes classification inference, and the segmentation of the corresponding 3D medical image can be completed.

2. The three-dimensional medical image segmentation method based on efficient modeling using a mask-supervised strategy according to claim 1, characterized in that, The specific methods for data preprocessing in step 1 are as follows: First, intensity clipping is performed on soft tissue in the 3D medical image segmentation dataset, with a range of [-175, 200]. Second, the image is randomly rotated [-30°, 30°]. Finally, intensity normalization is performed on the 99% intensity quantile of the image.

Images