Brain tumor image segmentation method based on improved NestedUNet network and Transform attention mechanism
By improving the NestedUNet network and Transformer attention mechanism, and combining multi-level residual blocks and channel attention mechanism, the problems of feature extraction and multi-modal information fusion in brain tumor segmentation are solved, achieving higher accuracy and robustness in segmentation.
Patent Information
- Application Number
- CN202410956776.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-17
- Publication Date
- 2024-11-08
AI Technical Summary
Existing medical image segmentation methods, especially brain tumor segmentation, suffer from insufficient feature extraction capabilities and inadequate multimodal information fusion, resulting in low segmentation accuracy and robustness.
An improved NestedUNet network is adopted, which combines Transformer attention mechanism, multi-level residual block and channel attention mechanism to capture multi-scale features through upsampling and downsampling operations, and utilizes multimodal image information and a hybrid loss function to optimize the segmentation results.
It significantly improves the accuracy and robustness of brain tumor segmentation, and enhances the quality of segmentation results and the reliability of clinical diagnosis.
Smart Images

Figure CN118918123A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of medical image processing, and in particular relates to a brain tumor segmentation method based on an improved NestedUNet network and a Transformer attention mechanism. Background Art
[0002] Medical image segmentation is an important topic in the field of modern medical image processing and is widely used in disease diagnosis, treatment planning, and postoperative evaluation. Accurate medical image segmentation can help doctors better understand the location, size, and morphology of lesions, thereby improving the accuracy of diagnosis and treatment. With the rapid development of deep learning technology, medical image segmentation methods based on convolutional neural networks (CNNs) have gradually become a research hotspot.
[0003] Traditional medical image segmentation methods mainly rely on manual feature extraction and classic image processing techniques such as threshold segmentation, region growing, and edge detection. These methods have certain effects when processing simple images, but it is often difficult to achieve satisfactory results when facing complex medical images. Especially in the task of brain tumor segmentation, the shapes and sizes of tumors are varied, and the boundaries are fuzzy, so the limitations of traditional methods are more obvious.
[0004] To solve these problems, researchers began to introduce deep learning technology, especially convolutional neural networks. U-Net is a classic medical image segmentation network that has attracted widespread attention for its excellent performance in multiple medical image segmentation tasks. U-Net can effectively capture detailed information in images through an encoder-decoder structure. However, U-Net has certain limitations when processing multi-scale features and is not able to capture some detailed information.
[0005] In order to further improve the segmentation performance, the researchers proposed NestedUNet (also known as UNet++), which enhances the fusion capability of multi-scale features by introducing multiple dense jump connections between the encoder and decoder. NestedUNet performs well in capturing detailed features and processing complex shapes, but there is still some room for improvement, such as insufficient ability to distinguish features of different importance.
[0006] In addition, the fusion of multimodal medical images also plays a vital role in brain tumor segmentation. Multimodal images such as T1, T1ce, T2 and FLAIR provide different information about brain tissue. How to effectively fuse these multimodal information to improve the accuracy of segmentation is an important research direction.
[0007] Based on the above background, the present invention proposes a brain tumor segmentation method based on an improved NestedUNet network and multi-scale feature fusion. Based on NestedUNet, this method introduces a multi-level residual block and channel attention mechanism, and effectively captures multi-scale features through up-sampling and down-sampling operations. By making full use of the information of multimodal images, the accuracy and robustness of brain tumor segmentation are improved, which has high clinical application value.
[0008] In addition, the present invention innovatively introduces the Transformer attention mechanism, which enhances the feature representation capability through a multi-head self-attention mechanism. The Transformer attention mechanism can capture global information and improve the correlation between features, thereby improving the segmentation performance of the model. At the same time, the use of the hybrid loss function BCEDiceLoss combines the binary cross entropy loss and the Dice loss to optimize the prediction accuracy and the overlap of the segmented regions. These innovations enable the method of the present invention to perform well in processing complex medical image segmentation tasks, and has significant application prospects. Summary of the invention
[0009] The purpose of the present invention is to address the deficiencies of the prior art in medical image segmentation and propose a brain tumor segmentation method based on an improved NestedUNet network and multi-scale feature fusion. By introducing the Transformer attention mechanism, multi-level residual blocks and channel attention mechanism into the network, the limitations of the traditional U-Net model in feature extraction and segmentation accuracy are solved. This method effectively captures multi-scale features through up and down sampling operations, and uses Transformer and channel attention mechanisms to enhance the expression of important features, thereby improving the accuracy and robustness of brain tumor segmentation and significantly improving the quality of segmentation results. Technical solution of the present invention:
[0010] A brain tumor image segmentation method based on an improved NestedUNet network and a Transformer attention mechanism, wherein the brain tumor image segmentation method is implemented by using a model based on an improved NestedUNet network and a Transformer attention mechanism, wherein the model based on the improved NestedUNet network and the Transformer attention mechanism is hereinafter referred to as “IMTN-UNet”, and comprises the following steps:
[0011] Step S1: Preprocessing of cranial medical imaging data:
[0012] The step S1 includes: ① loading multi-modal image data of cranial magnetic resonance images; ② standardizing the image data of each modality uniformly using the z-score method; ③ slicing the three-dimensional medical image data and cutting it into multiple two-dimensional slices along the depth axis; ④ removing the two-dimensional slices that do not contain the lesion part to alleviate the category imbalance problem; ⑤ merging the slices of each modality and saving them in array form.
[0013] In step S1, the multimodal imaging data of the brain magnetic resonance image includes T1-weighted imaging, T2-weighted imaging, contrast-enhanced T1-weighted imaging and liquid attenuated inversion recovery imaging data; in step S1, z-score standardization is defined as Wherein, X′ represents the original data value, μ represents the mean value of the value, σ represents the standard deviation of the data, and Z represents the standardized data value; the specific operation of step S1 ③ is: for a given three-dimensional medical image data I i and label data L a , let the original image data I a The shape of is (D,H,W,M), where D is the depth, H and W are the height and width respectively, and M is the number of modalities. The corresponding label data L a The shape of the three-dimensional image data I i Cut into a 2D slice set I a,d , where a represents the ath sample and d represents the depth index of the slice: I a,d =I a [d, d, d, d], where d = 0, 1, ..., D-1. Similarly, the label data L a Cut into a two-dimensional slice set L a,d :L a,d =I i [d, d, d]; In step S1, the specific operation of removing the slices without lesions is: for each label slice L a,d If there is no lesion in the slice, that is, all pixel values are 0, the slice is discarded. The conditions for judging whether it is a lesion-free slice are: If the condition is met, the slice L is discarded a,d and the corresponding image slice I a,d In step S1, the specific operation of merging slices of each modality is as follows: combining image slices of multiple modalities into a multi-channel image, assuming that slices of different modalities are I a,d,m , where m = 0, 1, ..., M-1. Combine them into a multi-channel image I ′ a,d , whose shape is (H,W,M). ′ a,d =Concat(I a,d,0 ,Ia,d,1 ,...,I a,d,M-1 ), where Concat represents the concatenation operation in the channel dimension.
[0014] Step S2: Initialize model parameters and set the number of input channels C in , batch size B, training rounds E, early stopping rounds P, learning rate η1 of Adam optimizer, learning rate η2 of SGD optimizer, momentum μ′ and weight decay parameter λ.
[0015] Step S3: Model training and iterative update:
[0016] The step S3 includes: ① input data in the shape of (B, C in ,H,W); ② Gradually extract features through five residual blocks ResBlock1 to ResBlock5, and then downsample through each residual block to reduce the spatial dimension of the feature map; ③ Add TransformerBlock after each ResBlock to further process the features to capture global dependencies; ④ The feature map then gradually restores the spatial dimension through four upsampling layers, and further extracts and fuses features through VGG blocks after each upsampling; ⑤ The output shape is (B,C out ,H,W), where C out is the number of output channels; ⑥ During the training process, each batch of data is predicted through forward propagation, and the error between the predicted value and the true value is calculated through the loss function. Subsequently, the back propagation algorithm calculates the gradient of the loss function relative to each weight. The optimizer uses these gradients to update the weights and biases through gradient descent, so that the model is gradually optimized in each iteration, ultimately improving the model performance.
[0017] Specific steps of operation ②: In the network layers ResBlock1 to ResBlock5, the input data passes through two 3×3 convolutional layers, batch normalization and ReLU activation function, and then the weights are adjusted through the channel attention mechanism, and then the adjusted feature map is added to the feature map after input data processing through residual connection, and finally downsampled through the maximum pooling operation. The number of output channels of each ResBlock increases layer by layer, namely 32, 64, 128, 256 and 512, so as to extract richer features and reduce the spatial dimension of the feature map layer by layer.
[0018] ResBlock1: Let the input data be X, whose shape is (B,C in ,H,W). First, the input data X passes through the first convolution layer. This convolution layer contains 32 convolution kernels, each of which is 3×3 in size and is applied to 4 input channels. The convolution operation is: Where i is the batch index, j is the output channel index, k is the input channel index, m and n are the height and width indices, W1 is the weight matrix of the first convolutional layer, and b1 is the bias vector of the first convolutional layer. The shape of the output X1 of the convolution operation is (B, 32, H, W). Then, the output X1 of the convolutional layer passes through batch normalization and ReLU activation function. The batch normalization operation is: Among them, μ j and are the mean and variance of the jth channel, respectively, and ∈ is a decimal to prevent division by zero. Next, the ReLU activation function is used: The shape of the output X2 after ReLU activation is (B, 32, H, W). Next, the output X2 passes through the second convolution layer. This convolution layer contains 32 convolution kernels, each of which is 3×3 in size and is applied to 32 input channels. The convolution operation is: Where W2 is the weight matrix of the second convolutional layer, and b2 is the bias vector of the second convolutional layer. The shape of the output X3 of the convolution operation is (B, 32, H, W). Then, the output X3 of the convolutional layer is subjected to a batch normalization operation: Among them, μ j is the mean of the jth channel, and the calculation formula is: N is the batch size, is the variance of the jth channel, and is calculated as: The shape of the batch normalized output X4 is (B, 32, H, W). Next, the output X4 passes through the channel attention mechanism. First, the output is average pooled and max pooled: The output shape of the pooling operation is (B, 32, 1, 1). Then, it is processed by two fully connected layers, where the first fully connected layer: Among them, W fc1 is the weight matrix of the first fully connected layer, b fc1 is the bias vector of the first fully connected layer. Next, merge the pooling results and pass them through the second fully connected layer: Among them, W fc2 is the weight matrix of the second fully connected layer, b fc2 is the bias vector of the second fully connected layer. Finally, adjust the attention weight: X ca [i,j,m,n]=X4[i,j,m,n]·X ca [i,j,1,1]. Output X after attention adjustment ca The shape of is (B, 32, H, W). Then, through residual connection and ReLU activation: Among them, Widentity is the weight matrix of the residual connection layer, b identity is the bias vector of the residual connection layer. Then, through the ReLU activation function: X out [i,j,m,n]=ReLU(X ca [i,j,m,n]+X identity [i,j,m,n]). The output X after ReLU activation out The shape of is (B, 32, H, W). Finally, downsampling is performed through maximum pooling: X down [i,j,m,n]=max 1≤p≤2,1≤q≤2 X out [i,j,2m+p-1,2n+q-1]. Output X after downsampling down The shape is (B, 32, H / 2, W / 2).
[0019] ResBlock2 to ResBlock5: The operation steps of each residual block are the same as ResBlock1, but the number of input channels and output channels are different. The input data shape changes layer by layer starting from (B, 32, H / 2, W / 2) until the output shape of ResBlock5 is (B, 512, H / 32, W / 32). The number of convolution kernels, input channels, and output channels of each layer are as follows: ResBlock2 has 32 input channels and 64 output channels; ResBlock3 has 64 input channels and 128 output channels; ResBlock4 has 128 input channels and 256 output channels; ResBlock5 has 256 input channels and 512 output channels.
[0020] Specific steps of operation ③: After the output of each ResBlock, add TransformerBlock to enhance feature expression.
[0021] TransformerBlock: ⑴ Reshape the input data from a 4D tensor to a 3D tensor to adapt it to the input requirements of the multi-head self-attention mechanism. Let the input data be X * , whose shape is (B,C,H,W), where B is the batch size, C is the number of channels, H and W are the height and width respectively. First, the input data is reshaped into (B,C,H×W), and then reshaped into)H×W,B,C). ⑵ Apply the multi-head self-attention mechanism to the reshaped data to capture global dependencies. ⑶ Residual connection and layer normalization: Add the attention output to the original input and perform layer normalization. Among them, γ and β are trainable parameter vectors with the same dimensions as the input h. (4) Input the data processed by the attention mechanism into the feedforward neural network to further enhance the feature expression. Among them, the feedforward neural network (FFN) includes two linear transformation layers and an activation function: W1 ′ is the weight matrix of the first linear transformation, is the bias vector of the first linear transformation, W2 ′ is the weight matrix of the second linear transformation. Is the bias vector of the second linear transformation. ⑸ Reshape the processed data from the 3D tensor back to the 4D tensor to adapt to the subsequent convolution operation.
[0022] For the specific steps of operation ④ in step S3: In UpConv1 to UpConv5, the input data is upsampled in turn to double the spatial size of the feature map, and then passes through two 3×3 convolutional layers, batch normalization and ReLU activation function, and then adjusts the weights through the channel attention mechanism, and then adds the adjusted feature map to the feature map processed by the upsampled input data through residual connection, and finally generates an output that integrates more detailed features layer by layer. The number of output channels of each UpConv layer decreases layer by layer, gradually decreasing from 512 to 16, and the original spatial dimension of the feature map is restored. The detailed network layer description is:
[0023] UpConv1: Let the input data be the output of the fifth network layer Its shape is (B, 512, H / 32, W / 32). First, input data X down5 Through the upsampling layer of the sixth network, the spatial size of the feature map is doubled: Among them, Upsample is the function of the upsampling operation, and scale_factor is the scale factor of the upsampling, which indicates the size magnification of the feature map in the spatial dimension. The output after upsampling The shape of is (B,512,H / 16,W / 16). Next, output Through a convolutional layer, containing 256 3×3 convolution kernels, applied to 512 input channels: Among them, W1 (6) is the convolution kernel weight of the first convolution layer, is the bias of the first convolutional layer. Then, the output of the convolutional layer Through batch normalization and ReLU activation function: in, is the mean of the jth channel, is the variance of the jth channel. Next, output Through the second convolutional layer, containing 256 3×3 convolution kernels, applied to the 256 input channels: Among them, W2 (6) is the convolution kernel weight of the second convolution layer, is the bias of the second convolutional layer. Then, the output of the convolutional layer With batch normalization: Next, output Through the channel attention mechanism, average pooling and maximum pooling are performed, and the attention weight is adjusted after processing through the fully connected layer. Through the channel attention mechanism, the processing of average pooling and maximum pooling is the same as the corresponding steps in the network layer ResBlock1. The process of adjusting the attention weight after processing through the fully connected layer is: in, is the weight matrix of the fully connected layer used to further adjust the feature map, is the weight matrix used by the fully connected layer to process the pooled features. Finally, through residual connection and ReLU activation: in, is the weight matrix of the residual connection convolutional layer, is the bias of the residual connected convolutional layer.
[0024] UpConv2: Set the input data to the output of the sixth layer network Its shape is (B, 256, H / 16, W / 16). Input data Through the upsampling layer, the spatial size of the feature map is doubled, and the output passes through a series of convolutional layers, batch normalization, ReLU activation and channel attention mechanism, and finally through residual connection and ReLU activation.
[0025] UpConv3: Set the input data to the output of the seventh layer network Its shape is (B, 128, H / 8, W / 8). Input data Through the upsampling layer, the spatial size of the feature map is doubled, and the output passes through a series of convolutional layers, batch normalization, ReLU activation and channel attention mechanism, and finally through residual connection and ReLU activation.
[0026] UpConv4: Set the input data to the output of the eighth layer network Its shape is (B, 64, H / 4, W / 4). Input data Through the upsampling layer, the spatial size of the feature map is doubled, and the output passes through a series of convolutional layers, batch normalization, ReLU activation and channel attention mechanism, and finally through residual connection and ReLU activation.
[0027] UpConv5: Let the input data be the output of the ninth layer network Its shape is 9B, 32, H / 2, W / 2). Input data Through the upsampling layer, the spatial size of the feature map is doubled, and the output passes through a series of convolutional layers, batch normalization, ReLU activation and channel attention mechanism, and finally through residual connection and ReLU activation.
[0028] For the specific steps of operation ⑤ in step S3: After passing through the last network layer Output Layer. Assume that the input data is the output of the tenth layer network Its shape is (B,16,H,W). Input data Through a convolution layer, the number of channels is reduced to the target number of channels. Assuming the target number of channels is 3, the output is the final segmentation result.
[0029] For the specific steps of operation 6 in step S3: the input data X is passed to the model through the forward propagation process, and the prediction result is calculated For each batch of data, (1) Forward propagation: Input data X is calculated through the model to obtain the predicted result. f(X; θ). θ represents all the weights and bias parameters of the model, and f represents the forward propagation process of the model. ⑵ Loss function calculation: Through the loss function Calculate predicted values The error between the true value Y and Where N is the batch size and l is the single sample loss function. A combination of binary cross entropy loss and Dice loss is used, namely: ⑶ Back propagation: Calculate the gradient of the loss function with respect to each weight During the back-propagation process, the calculated gradient is used by the optimizer to update the weights. (4) Weight update: The optimizer uses the calculated gradient to update the weights and biases through the gradient descent algorithm: Where η is the learning rate. The Adam optimizer or SGD optimizer is used for weight update.
[0030] Step S4: Load the model and implement image segmentation:
[0031] The specific process is as follows: ⑴ Loading model: Loading the trained model parameters θ * ⑵ Preprocessing of the input data to be segmented: Convert the input data X into the format and shape required by the model according to the processing method of step S1. ⑶ Output of prediction results: Output the final prediction result Y pred .
[0032] Beneficial effects of the present invention:
[0033] 1. This invention adopts a brain tumor segmentation method based on an improved NestedUNet network, Transformer mechanism and multi-scale feature fusion. By introducing multi-level residual blocks, channel attention mechanisms and Transformer blocks into the network, the feature extraction and segmentation accuracy are greatly improved compared to the traditional U-Net model. The multi-level residual blocks effectively capture the detailed features of the image, the channel attention mechanism enhances the expression of important features, and the Transformer blocks capture global dependencies through the self-attention mechanism, further improving the interpretability and robustness of the model.
[0034] 2. The method of the present invention effectively captures multi-scale features through up-sampling and down-sampling operations in each layer of the network, and combines global and local information to improve the accuracy and consistency of the segmentation results. The upsampling layer restores the spatial resolution of the feature map layer by layer, the convolution layer extracts features, batch normalization and ReLU activation function enhance nonlinear expression capabilities, and the channel attention mechanism and Transformer block further optimize the importance and global correlation of the feature map. This method can significantly improve the quality of brain tumor segmentation and improve the reliability and accuracy of clinical diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] Figure 1 It is a schematic diagram of the model structure of the present invention.
[0036] Figure 2 This is a graph showing the results of brain tumor image segmentation. DETAILED DESCRIPTION
[0037] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings.
[0038] A brain tumor image segmentation method based on an improved NestedUNet network and a Transformer attention mechanism, wherein the brain tumor image segmentation method is implemented by using a model based on an improved NestedUNet network and a Transformer attention mechanism, wherein the model based on the improved NestedUNet network and the Transformer attention mechanism is hereinafter referred to as “IMTN-UNet”, and comprises the following steps:
[0039] Step S1: Preprocessing of cranial medical imaging data:
[0040] The step S1 includes: ① loading multi-modal image data of cranial magnetic resonance images; ② standardizing the image data of each modality uniformly using the z-score method; ③ slicing the three-dimensional medical image data and cutting it into multiple two-dimensional slices along the depth axis; ④ removing the two-dimensional slices that do not contain the lesion part to alleviate the category imbalance problem; ⑤ merging the slices of each modality and saving them in array form.
[0041] In step S1, the multimodal imaging data of the brain magnetic resonance image includes T1-weighted imaging, T2-weighted imaging, contrast-enhanced T1-weighted imaging and liquid attenuated inversion recovery imaging data; in step S1, z-score standardization is defined as Wherein, X′ represents the original data value, μ represents the mean value of the value, σ represents the standard deviation of the data, and Z represents the standardized data value; the specific operation of step S1 ③ is: for a given three-dimensional medical image data I i and label data L a , let the original image data I a The shape of is (D,H,W,M), where D is the depth, H and W are the height and width respectively, and M is the number of modalities. The corresponding label data L a The shape of the three-dimensional image data I i Cut into a 2D slice set I a,d , where a represents the ath sample and d represents the depth index of the slice: I a,d =I a [d, d, d, d], where d = 0, 1, ..., D-1. Similarly, the label data L a Cut into a two-dimensional slice set L a,d :L a,d =I i [d, d, d]; In step S1, the specific operation of removing the slices without lesions is: for each label slice L a,d If there is no lesion in the slice, that is, all pixel values are 0, the slice is discarded. The conditions for judging whether it is a lesion-free slice are: If the condition is met, the slice L is discarded a,d and the corresponding image slice I a,d In step S1, the specific operation of merging slices of each modality is as follows: combining image slices of multiple modalities into a multi-channel image, assuming that slices of different modalities are I a,d,m , where m = 0, 1, ..., M-1. Combine them into a multi-channel image I ′ a,d , whose shape is (H,W,M). ′ a,d =Concat(I a,d,0 ,Ia,d,1 ,...,I a,d,M-1 ), where Concat represents the concatenation operation in the channel dimension.
[0042] Step S2: Initialize model parameters and input the number of channels C in is 4, the batch size B is 18, the training round E is 100, and the early stopping round P is 20. The learning rate of the Adam optimizer is set to η1 = 3 × 10 -4 , the learning rate of the SGD optimizer is set to η2 = 3 × 10 -4 , momentum μ′=0.9 and weight decay parameter λ=1×10 -4 .
[0043] Step S3: Model training and iterative update:
[0044] The step S3 includes: ① input data in the shape of (B, C in ,H,W); ② Gradually extract features through five residual blocks ResBlock1 to ResBlock5, and then downsample through each residual block to reduce the spatial dimension of the feature map; ③ Add TransformerBlock after each ResBlock to further process the features to capture global dependencies; ④ The feature map then gradually restores the spatial dimension through four upsampling layers, and further extracts and fuses features through VGG blocks after each upsampling; ⑤ The output shape is (B,C out ,H,W), where C out is the number of output channels; ⑥ During the training process, each batch of data is predicted through forward propagation, and the error between the predicted value and the true value is calculated through the loss function. Subsequently, the back propagation algorithm calculates the gradient of the loss function relative to each weight. The optimizer uses these gradients to update the weights and biases through gradient descent, so that the model is gradually optimized in each iteration, ultimately improving the model performance.
[0045] Specific steps of operation ②: In the network layers ResBlock1 to ResBlock5, the input data passes through two 3×3 convolutional layers, batch normalization and ReLU activation function, and then the weights are adjusted through the channel attention mechanism, and then the adjusted feature map is added to the feature map after input data processing through residual connection, and finally downsampled through the maximum pooling operation. The number of output channels of each ResBlock increases layer by layer, namely 32, 64, 128, 256 and 512, so as to extract richer features and reduce the spatial dimension of the feature map layer by layer.
[0046] ResBlock1: Let the input data be X, whose shape is (B,C in,H,W). First, the input data X passes through the first convolution layer. This convolution layer contains 32 convolution kernels, each of which is 3×3 in size and is applied to 4 input channels. The convolution operation is: Where i is the batch index, j is the output channel index, k is the input channel index, m and n are the height and width indices, W1 is the weight matrix of the first convolutional layer, and b1 is the bias vector of the first convolutional layer. The shape of the output X1 of the convolution operation is (B, 32, H, W). Then, the output X1 of the convolutional layer passes through batch normalization and ReLU activation function. The batch normalization operation is: Among them, μ j and are the mean and variance of the jth channel, respectively, and ∈ is a decimal to prevent division by zero. Next, the ReLU activation function is used: The shape of the output X2 after ReLU activation is (B, 32, H, W). Next, the output X2 passes through the second convolution layer. This convolution layer contains 32 convolution kernels, each of which is 3×3 in size and is applied to 32 input channels. The convolution operation is: Where W2 is the weight matrix of the second convolutional layer, and b2 is the bias vector of the second convolutional layer. The shape of the output X3 of the convolution operation is (B, 32, H, W). Then, the output X3 of the convolutional layer is subjected to a batch normalization operation: Among them, μ j is the mean of the jth channel, and the calculation formula is: N is the batch size, is the variance of the jth channel, and is calculated as: The shape of the batch normalized output X4 is (B, 32, H, W). Next, the output X4 passes through the channel attention mechanism. First, the output is average pooled and max pooled:
[0047] X max [i,j]=max 1≤m≤H,1≤n≤W X4[i,j,m,n], pooling
[0048] The output shape of the operation is (B, 32, 1, 1). Then, it is processed by two fully connected layers, where the first fully connected layer: Among them, W fc1 is the weight matrix of the first fully connected layer, b fc1 is the bias vector of the first fully connected layer. Next, merge the pooling results and pass them through the second fully connected layer: Among them, W fc2 is the weight matrix of the second fully connected layer, b fc2is the bias vector of the second fully connected layer. Finally, adjust the attention weight: X ca [i,j,m,n]=X4[i,j,m,n]·X ca [i,j,1,1]. Output X after attention adjustment ca The shape of is (B, 32, H, W). Then, through residual connection and ReLU activation: Among them, W identity is the weight matrix of the residual connection layer, b identity is the bias vector of the residual connection layer. Then, through the ReLU activation function: X out [i,j,m,n]=ReLU(X ca [i,j,m,n]+X identity [i,j,m,n]). The output X after ReLU activation out The shape of is (B, 32, H, W). Finally, downsampling is performed through maximum pooling: X down [i,j,m,n]=max 1≤p≤2,1≤q≤2 X out [i,j,2m+p-1,2n+q-1]. Output X after downsampling down The shape is (B, 32, H / 2, W / 2).
[0049] ResBlock2 to ResBlock5: The operation steps of each residual block are the same as ResBlock1, but the number of input channels and output channels are different. The input data shape changes layer by layer starting from (B, 32, H / 2, W / 2) until the output shape of ResBlock5 is (B, 512, H / 32, W / 32). The number of convolution kernels, input channels, and output channels of each layer are as follows: ResBlock2 has 32 input channels and 64 output channels; ResBlock3 has 64 input channels and 128 output channels; ResBlock4 has 128 input channels and 256 output channels; ResBlock5 has 256 input channels and 512 output channels.
[0050] Specific steps of operation ③: After the output of each ResBlock, add TransformerBlock to enhance feature expression.
[0051] TransformerBlock: ⑴ Reshape the input data from a 4D tensor to a 3D tensor to adapt it to the input requirements of the multi-head self-attention mechanism. Let the input data be X *, whose shape is (B,C,H,W), where B is the batch size, C is the number of channels, H and W are the height and width respectively. First, the input data is reshaped into (B,C,H×W), and then reshaped into)H×W,B,C). ⑵ Apply the multi-head self-attention mechanism to the reshaped data to capture global dependencies. ⑶ Residual connection and layer normalization: Add the attention output to the original input and perform layer normalization. Among them, and γ and β are trainable parameter vectors with the same dimensions as the input h. (4) Input the data processed by the attention mechanism into the feedforward neural network to further enhance the feature expression. Among them, the feedforward neural network (FFN) includes two linear transformation layers and an activation function: W1 ′ is the weight matrix of the first linear transformation, is the bias vector of the first linear transformation, W2 ′ is the weight matrix of the second linear transformation. Is the bias vector of the second linear transformation. ⑸ Reshape the processed data from the 3D tensor back to the 4D tensor to adapt to the subsequent convolution operation.
[0052] For the specific steps of operation ④ in step S3: In UpConv1 to UpConv5, the input data is upsampled in turn to double the spatial size of the feature map, and then passes through two 3×3 convolutional layers, batch normalization and ReLU activation function, and then adjusts the weights through the channel attention mechanism, and then adds the adjusted feature map to the feature map processed by the upsampled input data through residual connection, and finally generates an output that integrates more detailed features layer by layer. The number of output channels of each UpConv layer decreases layer by layer, gradually decreasing from 512 to 16, and the original spatial dimension of the feature map is restored. The detailed network layer description is:
[0053] UpConv1: Let the input data be the output of the fifth network layer Its shape is (B, 512, H / 32, W / 32). First, input data X down5 Through the upsampling layer of the sixth network, the spatial size of the feature map is doubled: Among them, Upsample is the function of the upsampling operation, and scale_factor is the scale factor of the upsampling, which indicates the size magnification of the feature map in the spatial dimension. The output after upsampling The shape of is (B,512,H / 16,W / 16). Next, output Through a convolutional layer, containing 256 3×3 convolution kernels, applied to 512 input channels: Among them, W1(6) is the convolution kernel weight of the first convolution layer, is the bias of the first convolutional layer. Then, the output of the convolutional layer Through batch normalization and ReLU activation function: in, is the mean of the jth channel, σ j (6)2 is the variance of the jth channel. Next, output Through the second convolutional layer, containing 256 3×3 convolution kernels, applied to the 256 input channels: Among them, W2 (6) is the convolution kernel weight of the second convolution layer, is the bias of the second convolutional layer. Then, the output of the convolutional layer With batch normalization: Next, output Through the channel attention mechanism, average pooling and maximum pooling are performed, and the attention weight is adjusted after processing through the fully connected layer. Through the channel attention mechanism, the processing of average pooling and maximum pooling is the same as the corresponding steps in the network layer ResBlock1. The process of adjusting the attention weight after processing through the fully connected layer is: in, is the weight matrix of the fully connected layer used to further adjust the feature map, is the weight matrix used by the fully connected layer to process the pooled features. Finally, through residual connection and ReLU activation: in, is the weight matrix of the residual connection convolutional layer, is the bias of the residual connected convolutional layer.
[0054] UpConv2: Set the input data to the output of the sixth layer network Its shape is (B, 256, H / 16, W / 16). Input data Through the upsampling layer, the spatial size of the feature map is doubled, and the output passes through a series of convolutional layers, batch normalization, ReLU activation and channel attention mechanism, and finally through residual connection and ReLU activation.
[0055] UpConv3: Set the input data to the output of the seventh layer network Its shape is (B, 128, H / 8, W / 8). Input data Through the upsampling layer, the spatial size of the feature map is doubled, and the output passes through a series of convolutional layers, batch normalization, ReLU activation and channel attention mechanism, and finally through residual connection and ReLU activation.
[0056] UpConv4: Set the input data to the output of the eighth layer network Its shape is (B, 64, H / 4, W / 4). Input data Through the upsampling layer, the spatial size of the feature map is doubled, and the output passes through a series of convolutional layers, batch normalization, ReLU activation and channel attention mechanism, and finally through residual connection and ReLU activation.
[0057] UpConv5: Let the input data be the output of the ninth layer network Its shape is 9B, 32, H / 2, W / 2). Input data Through the upsampling layer, the spatial size of the feature map is doubled, and the output passes through a series of convolutional layers, batch normalization, ReLU activation and channel attention mechanism, and finally through residual connection and ReLU activation.
[0058] For the specific steps of operation ⑤ in step S3: After passing through the last network layer Output Layer. Assume that the input data is the output of the tenth layer network Its shape is (B,16,H,W). Input data Through a convolution layer, the number of channels is reduced to the target number of channels. Assuming the target number of channels is 3, the output is the final segmentation result.
[0059] For the specific steps of operation 6 in step S3: the input data X is passed to the model through the forward propagation process, and the prediction result is calculated For each batch of data, (1) Forward propagation: Input data X is calculated through the model to obtain the predicted result. Among them, θ represents all the weights and bias parameters of the model, and f represents the forward propagation process of the model. ⑵ Loss function calculation: Through the loss function Calculate predicted values The error between the true value Y and Where N is the batch size and l is the single sample loss function. A combination of binary cross entropy loss and Dice loss is used, namely: ⑶ Back propagation: Calculate the gradient of the loss function with respect to each weight During the back-propagation process, the calculated gradient is used by the optimizer to update the weights. (4) Weight update: The optimizer uses the calculated gradient to update the weights and biases through the gradient descent algorithm: Where η is the learning rate. The Adam optimizer or SGD optimizer is used for weight update.
[0060] Step S4: Load the model and implement image segmentation:
[0061] The specific process is as follows: ⑴ Loading model: Loading the trained model parameters θ * ⑵ Preprocessing of the input data to be segmented: Convert the input data X into the format and shape required by the model according to the processing method of step S1. ⑶ Output of prediction results: Output the final prediction result Y pred .
[0062] The specific implementation steps are as follows:
[0063] Performance evaluation: Divide the dataset and labels into training set, validation set, and test set in a ratio of 8:1:1. Train the model on the training set and adjust the hyperparameters on the validation set using the grid parameter search method. Evaluate the segmentation performance of the model on the test set. Record the results and take the average as the final segmentation performance evaluation index.
[0064] The brain tumor segmentation method based on the improved NestedUNet network and multi-scale feature fusion described in the present invention can be illustrated by the real BraTS dataset:
[0065] (1) BraTS dataset experimental process
[0066] In order to demonstrate the effect of the present invention, the BraTS dataset (Brain Tumor Segmentation dataset) is used in the embodiment.
[0067] The BraTS dataset provides multimodal MRI images, including four different sequences: T1-weighted images (T1), T1-weighted enhanced images (T1ce), T2-weighted images (T2), and Fluid-attenuated inversion recovery (Fluid-attenuated inversion recovery). There are three parts that need to be segmented, namely WT, ET, and TC. The dataset stipulates that Figure 2The green color is the edema area (ED), the yellow color is the enhanced tumor area (ET), and the red color is the gangrene (NET), which are merged into three nested sub-areas WT, TC, and ET. The whole tumor (WT) class includes all visible markers, the tumor core (TC) class is a combination of red and yellow, and the enhanced tumor core (ET) class is displayed in yellow. The BraTs dataset is further divided into HGG and LGG, which respectively represent: HGG: high-grade glioma (WHO grade 3-4) is a poorly differentiated glioma; this type of tumor is a malignant tumor and the patient has a poor prognosis. LGG: low-grade glioma (WHO grade 1-2) is a well-differentiated glioma; although this type of tumor is not biologically benign, the patient's prognosis is relatively good.
[0068] (2) Experimental results
[0069] The classification performance of five different algorithms on the BraTS dataset is shown in Table 2, and the best results are shown in bold in the figure. Among them, the present invention uses DICE similarity coefficient, positive prediction rate (PPV) and sensitivity to measure the classification performance of each algorithm:
[0070]
[0071] Among them, TP, FN, FP, and TN represent true positive examples, false negative examples, false positive examples, and true negative examples, respectively. As shown in Table 2, the method IMN-Unet achieved the best comprehensive classification effect in WT, ET, and TC imaging. For the deepresunet algorithm, DenseUnet algorithm, Hybridresunet algorithm, and Unet2D algorithm, which are also based on the Unet structure, the IMN-Unet algorithm is significantly better than the other four algorithms, which shows that extracting multi-scale features through multi-level residual blocks and channel attention mechanisms is beneficial to improving the quality of brain tumor segmentation.
[0072] Table 2 Classification performance of various algorithms
[0073]
[0074]
[0075] Table 2 is a comparison chart of the classification performance of the classification method of the embodiment of the present invention and other existing classification algorithms in different imaging parts, in which the best result is displayed in bold.
Claims
1. A brain tumor image segmentation method based on an improved NestedUNet network and Transformer attention mechanism, characterized in that: The brain tumor image segmentation method is implemented by using an improved NestedUNet network and a Transformer attention mechanism model, which is referred to as "IMTN-UNet" below, and includes the following steps: Step S1: preprocessing the cranial medical imaging data; Step S2: model parameter initialization; Step S3: model training and iterative update; Step S4: Load the model and implement image segmentation.
2. According to claim 1, a brain tumor image segmentation method based on an improved NestedUNet network and a Transformer attention mechanism is characterized in that: The step S1 includes: ① loading multi-modal image data of cranial magnetic resonance images; ② standardizing the image data of each modality uniformly using the z-score method; ③ slicing the three-dimensional medical image data, cutting it into multiple two-dimensional slices along the depth axis; ④ removing the two-dimensional slices without the lesion part to alleviate the problem of category imbalance; ⑤ merging the slices of each modality and saving them in array form; In step S1, the multimodal imaging data of the brain magnetic resonance image includes T1-weighted imaging, T2-weighted imaging, contrast-enhanced T1-weighted imaging and liquid attenuated inversion recovery imaging data; in step S1, z-score standardization is defined as: Wherein, X′ represents the original data value, μ represents the mean value of the value, σ represents the standard deviation of the data, and Z represents the standardized data value; the specific operation of step S1 ③ is: for a given three-dimensional medical image data I i and label data L a , let the original image data I a The shape is (D, H, W, M), where D is the depth, H and W are the height and width respectively, and M is the number of modalities; the corresponding label data L a shape (D, H, W); the three-dimensional image data I i Cut into a 2D slice set I a,d , where a represents the ath sample and d represents the depth index of the slice: I a,d =I a [d, d, d, d], where d = 0, 1, ..., D-1; similarly, the label data L a Cut into a two-dimensional slice set L a,d :L a,d =I i [d, d, d]; In step S1, the specific operation of removing the slices without lesions is: for each label slice L a,d If there is no lesion in the slice, that is, all pixel values are 0, the slice is discarded; the condition for judging whether it is a lesion-free slice is: If the condition is met, the slice L is discarded a,d and the corresponding image slice I a,d ; In step S1 ⑤, the specific operation of merging slices of each modality is: combining image slices of multiple modalities into a multi-channel image, assuming that slices of different modalities are I a,d,m , where m = 0, 1, ..., M-1; combine them into a multi-channel image I ′ a,d , whose shape is (H,W,M); I ′ a,d =Concat(I a,d,0 ,I a,d,1 ,...,I a,d,M-1 ), where Concat represents the concatenation operation in the channel dimension.
3. According to claim 1, a brain tumor image segmentation method based on an improved NestedUNet network and a Transformer attention mechanism is characterized in that: Step S2: Initialize model parameters and set the number of input channels C in , batch size B, training rounds E, early stopping rounds P, learning rate η1 of Adam optimizer, learning rate η2 of SGD optimizer, momentum μ′ and weight decay parameter λ.
4. According to claim 1, a brain tumor image segmentation method based on an improved NestedUNet network and a Transformer attention mechanism is characterized in that: Step S3: Model training and iterative update: The step S3 includes: ① input data in the shape of (B, C in ,H,W); ② Gradually extract features through five residual blocks ResBlock1 to ResBlock5, and then downsample through each residual block to reduce the spatial dimension of the feature map; ③ Add TransformerBlock after each ResBlock to further process the features to capture global dependencies; ④ The feature map then gradually restores the spatial dimension through four upsampling layers, and further extracts and fuses features through VGG blocks after each upsampling; ⑤ The output shape is (B,C out ,H,W), where C out is the number of output channels; ⑥ During the training process, each batch of data is predicted through forward propagation, and the error between the predicted value and the true value is calculated through the loss function; then, the backpropagation algorithm calculates the gradient of the loss function relative to each weight; the optimizer uses these gradients to update the weights and biases through gradient descent, so that the model is gradually optimized in each iteration, ultimately improving the model performance.
5. According to claim 4, a brain tumor image segmentation method based on an improved NestedUNet network and a Transformer attention mechanism is characterized in that: Specific steps of step S3 operation ②: In the network layers ResBlock1 to ResBlock5, the input data passes through two 3×3 convolutional layers, batch normalization and ReLU activation function, and then the weights are adjusted through the channel attention mechanism, and then the adjusted feature map is added to the feature map after the input data is processed through the residual connection, and finally down-sampled through the maximum pooling operation; the number of output channels of each ResBlock increases layer by layer, which are 32, 64, 128, 256 and 512 respectively, so as to extract richer features and reduce the spatial dimension of the feature map layer by layer; ResBlock1: Let the input data be X, whose shape is (B,C in ,H,W); First, the input data X passes through the first convolution layer; this convolution layer contains 32 convolution kernels, each of which is 3×3 in size and is applied to 4 input channels; the convolution operation is: Where i is the batch index, j is the output channel index, k is the input channel index, m and n are the height and width indices, W1 is the weight matrix of the first convolutional layer, and b1 is the bias vector of the first convolutional layer; the shape of the output X1 of the convolution operation is (B, 32, H, W); then, the output X1 of the convolutional layer passes through batch normalization and ReLU activation function; the batch normalization operation is: Among them, μ j and are the mean and variance of the jth channel, respectively, and ∈ is a decimal to prevent division by zero; then, the ReLU activation function is used: The shape of the output X2 after ReLU activation is (B, 32, H, W); next, the output X2 passes through the second convolutional layer; this convolutional layer contains 32 convolution kernels, each of which is 3×3 in size and is applied to 32 input channels; the convolution operation is: Where W2 is the weight matrix of the second convolutional layer, b2 is the bias vector of the second convolutional layer, and the shape of the output X3 of the convolution operation is (B, 32, H, W). Then, the output X3 of the convolutional layer is batch normalized: Among them, μ j is the mean value of the jth channel, and the calculation formula is: N is the batch size, is the variance of the jth channel, calculated as: The shape of the batch normalized output X4 is (B, 32, H, W). Next, the output X4 passes through the channel attention mechanism. First, the output is average pooled and max pooled: X max [i,j]=max 1≤m≤H,1≤n≤W X4[i,j,m,n], the output shape of the pooling operation is (B,32,1,1); then, it is processed through two fully connected layers, where the first fully connected layer: Among them, W fc1 is the weight matrix of the first fully connected layer, b fc1 is the bias vector of the first fully connected layer; then, the pooling results are merged and passed through the second fully connected layer: Among them, W fc2 is the weight matrix of the second fully connected layer, b fc2 is the bias vector of the second fully connected layer; finally adjust the attention weight: X ca [i,j,m,n]=X4[i,j,m,n]·X ca [i,j,1,1]; output X after attention adjustment ca The shape is (B, 32, H, W); then, through the residual connection and ReLU activation: Among them, W identity is the weight matrix of the residual connection layer, b identity is the bias vector of the residual connection layer; then, through the ReLU activation function: X out [i,j,m,n]=ReLU(X ca [i,j,m,n]+X identity [i,j,m,n]); output X after ReLU activation out The shape of is (B, 32, H, W); finally, down-sample by max pooling: X down [i,j,m,n]=max 1≤p≤2,1≤q≤2 X out [i,j,2m+p-1,2n+q-1]; output X after downsampling down The shape is (B, 32, H / 2, W / 2); ResBlock2 to ResBlock5: The operation steps of each residual block are the same as ResBlock1, but the number of input channels and output channels are different; the input data shape changes layer by layer from (B, 32, H / 2, W / 2) until the output shape of ResBlock5 is (B, 512, H / 32, W / 32); the number of convolution kernels, input channels, and output channels of each layer are as follows: ResBlock2 has 32 input channels and 64 output channels; ResBlock3 has 64 input channels and 128 output channels; ResBlock4 has 128 input channels and 256 output channels; ResBlock5 has 256 input channels and 512 output channels.
6. The brain tumor image segmentation method based on the improved NestedUNet network and Transformer attention mechanism according to claim 4, characterized in that: Specific steps of step S3 operation ③: After the output of each ResBlock, add TransformerBlock to enhance feature expression; TransformerBlock: ⑴Reshape the input data from a 4D tensor to a 3D tensor to adapt it to the input requirements of the multi-head self-attention mechanism; let the input data be X * , whose shape is (B,C,H,W), where B is the batch size, C is the number of channels, H and W are the height and width respectively; first, the input data is reshaped into (B,C,H×W), and then reshaped into (H×W,B,C); ⑵ Apply a multi-head self-attention mechanism to the reshaped data to capture global dependencies; ⑶ Residual connection and layer normalization: Add the attention output to the original input and perform layer normalization; where, and γ and β are trainable parameter vectors with the same dimensions as the input h;⑷The data processed by the attention mechanism is input into the feedforward neural network to further enhance the feature expression; Among them, the feedforward neural network (FFN) includes two linear transformation layers and an activation function: W1 ′ is the weight matrix of the first linear transformation, is the bias vector of the first linear transformation, W2 ′ is the weight matrix of the second linear transformation; It is the bias vector of the second linear transformation; ⑸ Reshape the processed data from the 3D tensor back to the 4D tensor to adapt to subsequent convolution operations.
7. The brain tumor image segmentation method based on the improved NestedUNet network and Transformer attention mechanism according to claim 4, characterized in that: For the specific steps of operation ④ in step S3: in UpConv1 to UpConv5, the input data is sequentially upsampled to double the spatial size of the feature map, and then passes through two 3×3 convolutional layers, batch normalization and ReLU activation function, and then adjusts the weights through the channel attention mechanism, and then adds the adjusted feature map to the feature map processed by the upsampled input data through residual connection, and finally generates an output that integrates more detailed features layer by layer; the number of output channels of each UpConv layer is reduced layer by layer, gradually reduced from 512 to 16, and the original spatial dimension of the feature map is restored; the detailed network layer description is: UpConv1: Let the input data be the output of the fifth network layer Its shape is (B, 512, H / 32, W / 32); First, input data X down5 Through the upsampling layer of the sixth network, the spatial size of the feature map is doubled: Among them, Upsample is the function of the upsampling operation, scale_factor is the scale factor of the upsampling, which indicates the size magnification of the feature map in the spatial dimension; the output after upsampling is The shape is (B, 512, H / 16, W / 16); Next, output Through a convolutional layer, containing 256 3×3 convolution kernels, applied to 512 input channels: Among them, W1 (6) is the convolution kernel weight of the first convolution layer, is the bias of the first convolutional layer; then, the output of the convolutional layer Through batch normalization and ReLU activation function: in, is the mean of the jth channel, is the variance of the jth channel; next, the output Through the second convolutional layer, containing 256 3×3 convolution kernels, applied to the 256 input channels: Among them, W2 (6) is the convolution kernel weight of the second convolution layer, is the bias of the second convolutional layer; then, the output of the convolutional layer With batch normalization: Next, output Through the channel attention mechanism, average pooling and maximum pooling are performed, and the attention weight is adjusted after processing through the fully connected layer. Through the channel attention mechanism, the processing of average pooling and maximum pooling is the same as the corresponding steps in the network layer ResBlock1. The process of adjusting the attention weight after processing through the fully connected layer is: in, is the weight matrix of the fully connected layer used to further adjust the feature map, It is the weight matrix of the fully connected layer used to process the pooled features; finally, through the residual connection and ReLU activation: in, is the weight matrix of the residual connection convolutional layer, is the bias of the residual connection convolutional layer; UpConv2: Set the input data to the output of the sixth layer network Its shape is (B, 256, H / 16, W / 16); input data Through the upsampling layer, the spatial size of the feature map is doubled, and the output passes through a series of convolutional layers, batch normalization, ReLU activation and channel attention mechanism, and finally through residual connection and ReLU activation; UpConv3: Set the input data to the output of the seventh layer network Its shape is (B, 128, H / 8, W / 8); input data Through the upsampling layer, the spatial size of the feature map is doubled, and the output passes through a series of convolutional layers, batch normalization, ReLU activation and channel attention mechanism, and finally through residual connection and ReLU activation; UpConv4: Set the input data to the output of the eighth layer network Its shape is (B, 64, H / 4, W / 4); input data Through the upsampling layer, the spatial size of the feature map is doubled, and the output passes through a series of convolutional layers, batch normalization, ReLU activation and channel attention mechanism, and finally through residual connection and ReLU activation; UpConv5: Let the input data be the output of the ninth layer network Its shape is (B, 32, H / 2, W / 2); input data Through the upsampling layer, the spatial size of the feature map is doubled, and the output passes through a series of convolutional layers, batch normalization, ReLU activation and channel attention mechanism, and finally through residual connection and ReLU activation.
8. The brain tumor image segmentation method based on the improved NestedUNet network and Transformer attention mechanism according to claim 4, characterized in that: For the specific steps of operation ⑤ in step S3: after passing through the last network layer Output Layer; let the input data be the output of the tenth layer network Its shape is (B, 16, H, W); input data Through a convolution layer, the number of channels is reduced to the target number of channels. Assuming the target number of channels is 3, the output is the final segmentation result.
9. The brain tumor image segmentation method based on the improved NestedUNet network and Transformer attention mechanism according to claim 4, characterized in that: The specific steps of operation ⑥ in step S3 are as follows: the input data X is passed to the model through the forward propagation process, and the prediction result is calculated. For each batch of data, (1) Forward propagation: Input data X is calculated through the model to obtain the predicted result. Among them, θ represents all weights and bias parameters of the model, and f represents the forward propagation process of the model; ⑵ Loss function calculation: Through the loss function Calculate predicted values The error between the true value Y and Where N is the batch size and l is the single sample loss function; a combination of binary cross entropy loss and Dice loss is used, namely: ⑶ Back propagation: Calculate the gradient of the loss function with respect to each weight During the back-propagation process, the calculated gradient is used by the optimizer to update the weights; (4) Weight update: The optimizer uses the calculated gradient to update the weights and biases through the gradient descent algorithm: Where η is the learning rate; the Adam optimizer or SGD optimizer is used to update the weights.
10. The brain tumor image segmentation method based on the improved NestedUNet network and Transformer attention mechanism according to claim 1, characterized in that: Step S4: Load the model and implement image segmentation. The specific process is as follows: (1) Load the model: Load the trained model parameters θ * ⑵ Preprocessing of the input data to be segmented: convert the input data X into the format and shape required by the model according to the processing method of step S1; ⑶ Output of prediction results: output the final prediction result Y pred .
Citation Information
Cited By
Thangka image segmentation method based on edge feature guidance and detail feature denoising
CN119169296A
Magnetic resonance brain tumor segmentation method based on reversible feature generation
CN119693393A
Decommissioned nuclear graphite potential prediction method and system based on multi-modal vision
CN120451651A
