A three-dimensional image segmentation method applying a double-scale cross-attention mechanism

Abstract

The application discloses a three-dimensional image segmentation method applying a double-scale cross attention mechanism, which comprises the following steps: collecting a three-dimensional image; and building a three-dimensional image segmentation network model DS-UNETR++ applying the double-scale cross attention mechanism, wherein the three-dimensional image segmentation network model DS-UNETR++ is improved based on a UNETR++ model and comprises four parts of three-dimensional patch partition, an encoder, a bottleneck and a decoder; the collected three-dimensional image is input into the network model after being preprocessed, image features are extracted through the three-dimensional patch partition, the encoder and the bottleneck part, then the image is restored by using the decoder, and a binary image segmentation region is output; and the trained image segmentation network model is used for three-dimensional image segmentation. The application has the characteristics of improving segmentation precision through different scale feature fusion.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and specifically to a three-dimensional image segmentation method that applies a dual-scale cross-attention mechanism. Background Technology

[0002] In the field of image analysis, image segmentation is a very important and challenging research direction. At the same time, this direction also has great practical application value, such as tumor image segmentation, heart image segmentation, abdominal multi-organ image segmentation, etc. With the rapid development of computer technology today, using image segmentation methods to assist doctors in diagnosis can not only help doctors improve the speed of diagnosis, but also further improve the accuracy of diagnostic results.

[0003] In existing technologies, with the successful application of deep learning in the image domain, self-attention mechanisms have replaced convolutional operations in CNNs, enabling models to establish long-range dependencies between pixels globally and thus better capture global information in images. More and more researchers are attempting segmentation models based on the Transformer architecture. For example, RT-UNet designed a Skip-Transformer that uses a multi-head self-attention mechanism to reduce the impact of shallow features on the overall network model performance; SWTRU designed a star-shaped window self-attention mechanism that, while maintaining computational complexity, further expands the model's attention area, achieving global attention; CFNet proposed the CFF method, extracting multi-view features of the image to balance the effectiveness of features at different angles in space, then calculating feature weights to optimize the semantic gap between shallow and deep features; NETR transforms 3D medical images into a series of sequences, transforming the segmentation task into a sequence prediction problem. It uses the Transformer architecture to learn multi-scale global information and finally designs an encoder and decoder based on a U-shaped network for pixel-level prediction. However, for 3D images, it cannot establish long-range dependencies at the pixel level globally, resulting in low segmentation accuracy. Summary of the Invention

[0004] The purpose of this invention is to overcome the above-mentioned shortcomings and propose a three-dimensional image segmentation method that improves segmentation accuracy by fusing features at different scales and applying a dual-scale cross-attention mechanism.

[0005] The present invention provides a three-dimensional image segmentation method applying a dual-scale cross-attention mechanism, comprising the following steps:

[0006] Step 1: Collect 3D images and divide them into training and test sets;

[0007] Step 2: Image segmentation network model construction: Build a 3D image segmentation network model DS-UNETR++ that applies a dual-scale cross-attention mechanism. The 3D image segmentation network model DS-UNETR++ is an improvement on the UNETR++ model and includes four parts: 3D patch partitioning, encoder, bottleneck, and decoder. The collected 3D images are preprocessed and then input into the network model. Image features are extracted through the 3D patch partitioning, encoder, and bottleneck parts. Then, the decoder is used to reconstruct the image and output the binarized image segmentation region.

[0008] The three-dimensional patch partition includes a convolutional layer and a normalization layer, which project the input three-dimensional image at both coarse and fine scales, converting it into two high-dimensional tensors X1 and X2 at different scales.

[0009] The encoder comprises three stages, each with two encoding blocks and a feature fusion block. The two encoding blocks receive high-dimensional tensors X1 and X2 from different scales of the 3D patch partition, respectively. The downsampling layer of the encoding block is used to capture the hierarchical object characteristics of the feature information of the high-dimensional tensors at different scales. The two gated shared weight pairwise attention submodules of the encoding block are used to capture the long-range dependencies between the feature information of the high-dimensional tensors at different scales. The convolutional fusion module receives the high-dimensional tensors output from the two encoding blocks and fuses the feature information of the two scales of high-dimensional tensors through convolution operations, with its outputs represented as F1 and F. The gated shared weight pairwise attention submodule of the encoding block adds a spatial attention and a channel attention gate to the output stages of its spatial attention and channel attention layers, respectively. The gated shared weight pairwise attention submodule receives the high-dimensional tensors and then inputs them into the spatial attention and channel attention layers to obtain spatial and channel attention maps. The calculation process is represented as follows:

[0010]

[0011] in, and These represent spatial and channel attention maps, respectively. SAM and CAM represent spatial attention and channel attention, respectively. SA G represents the spatial attention gating parameter. CA Q represents the channel attention gating parameter. SW and K SW V represents matrices for shared queries and shared keys, respectively. Spatial and V Channel These represent matrices for the spatial value layer and the channel value layer, respectively.

[0012] The bottleneck section includes a stage and a feature fusion block. The bottleneck section first receives the outputs F1 and F2 from the encoder section, and after attention calculation, they are sent to the feature fusion block, namely the gated dual-scale cross-attention module. The gated dual-scale cross-attention module adopts a standard Transformer block to allow the two volumetric high-dimensional tensors to interact fully, and further obtain more effective long-range dependencies. In order to control the influence of the two high-dimensional tensors after self-attention calculation on the final output, two gating valves are designed to automatically adjust the proportion of the two high-dimensional tensors, so that the generalization performance of the model is stronger.

[0013] The decoder consists of three stages, each with a decoding block. Each decoding block comprises an upsampling layer and a G-SWPA submodule. The upsampling layer uses deconvolution to progressively restore low-resolution features to high resolution. Meanwhile, skip connections are used to facilitate information transfer across layers in order to preserve more detailed information in the segmentation task.

[0014] Step 3: Image segmentation network model training: Train the 3D image segmentation network model DS-UNETR++ using a dual-scale cross-attention mechanism on the input training set, and evaluate and optimize the 3D image segmentation network model DS-UNETR++ using the test set;

[0015] Step 4: Image segmentation: Perform 3D image segmentation using the trained image segmentation network model.

[0016] The above-mentioned three-dimensional image segmentation method using a dual-scale cross-attention mechanism, wherein: in step 1, the three-dimensional image is a CT scan image.

[0017] The above-mentioned three-dimensional image segmentation method using a dual-scale cross-attention mechanism includes: in step 1, the collected CT scan images are divided into a training set and a test set in an 8:2 ratio.

[0018] The above-mentioned three-dimensional image segmentation method using a dual-scale cross-attention mechanism, wherein: in step 2, the spatial attention map Calculation: Taking the input high-dimensional tensor X1 as an example, Q is first calculated using three linear layers based on dimension merging. SW K SW and V Spatial The calculation formula is:

[0019] Q SW =W Q ×X1

[0020] K SW =W K ×X1

[0021] V Spatial =W V-Spatial ×X1

[0022] Among them, W Q W K W V-Spatial They are Q SW K SW V Spatial The projection weight matrix;

[0023] In the specific calculation, the shared key matrix K SW and spatial value layer matrix V Spatial Projecting to m dimensions yields the projected shared-bond matrix K. Proj and the projection space value layer matrix V Proj Then K Proj Transpose and multiply by Q SW After passing through Softmax, multiply by V Proj A preliminary spatial attention map was obtained. Finally multiply by G SA Obtain the final spatial attention map The calculation formula is as follows:

[0024]

[0025] Where d is the scale of the high-dimensional tensor.

[0026] The above-mentioned three-dimensional image segmentation method using a dual-scale cross-attention mechanism, wherein: in step 2, the channel attention map Computation: To further capture the interdependencies between feature channels, a dot product is used to compute the channel attention map. Taking the input high-dimensional tensor X1 as an example, the same Q-mapping algorithm as the spatial attention module is used. SW and K SW The channel attention value is calculated separately using a linear layer, as shown in the following formula:

[0027] V Channel =W V-Channel ×X1

[0028] Among them, W V-Channel It is V Channel The projection weight matrix;

[0029] In the specific calculations, the shared query matrix Q is used. SW transpose multiplied by the shared key matrix K SW After passing through Softmax, it is multiplied by the channel value layer matrix V. Channel Obtain the channel attention map Finally, multiply by the channel attention gating parameter G.CA Obtain the final spatial attention map The calculation formula is as follows:

[0030]

[0031] The above-mentioned 3D image segmentation method using a dual-scale cross-attention mechanism includes the following step: In step 2, the gated dual-scale cross-attention module is calculated as follows: Taking the interaction operation of the coarse-grained high-dimensional tensor F2 as an example, F1 is first subjected to average pooling, and then flattened to obtain a vector representing the entire fine-grained feature information. Next, F2 will also be flattened, reducing it from three dimensions to one dimension. Then spliced ​​together The result is fed into a Transformer for self-attention calculation, then multiplied by the gating parameter G2, and finally removed after the calculation is complete. Then, it is restored from one dimension to three dimensions to obtain the interactive feature information. Finally The final result is obtained by adding it to the original high-dimensional tensor F2. The specific calculation formula is as follows:

[0032]

[0033] The above-mentioned three-dimensional image segmentation method using a dual-scale cross-attention mechanism includes the following: In step 2, the skip connection refers to a high-dimensional tensor information transmission channel built between the encoder and the decoder. Through this channel, the output of each stage of the encoder can be directly transmitted to the corresponding stage in the decoder, so that more detailed information of shallow features can be preserved, the gradient vanishing problem can be alleviated, and the problem of sample imbalance can be overcome, thereby improving the performance and generalization ability of the model.

[0034] The above-mentioned three-dimensional image segmentation method using a dual-scale cross-attention mechanism includes the following steps: In step 3, when training the three-dimensional image segmentation network model DS-UNETR++ using the dual-scale cross-attention mechanism, the training hyperparameters are set as follows: initial learning rate of 0.01, weight decay of 3e-5, training period of 1000 epochs, and loss function.

[0035] The above-described 3D image segmentation method using a dual-scale cross-attention mechanism, wherein the loss function uses the Dice+CE Loss function, denoted as L(G,P):

[0036]

[0037] Where G refers to the set of actual results, and P refers to the set of predicted results.i,j and G i,j Let I and J represent the probability output and one-hot encoded true value of class j at voxel i, respectively. I is the number of voxels and J is the number of classes.

[0038] Compared with existing technologies, this invention has significant advantages. As shown in the above scheme, the 3D image segmentation method DS-UNETR++ of this invention designs a dual-scale feature encoding sub-network, which can simultaneously extract feature information at different scales, effectively capturing target features at different scales. It also improves upon existing technologies by designing a gated efficient pairwise attention block (G-EPA), using parallel spatial and channel attention mechanisms to extract features. A gate valve is designed for the outputs of the two attention mechanisms to limit their negative impact on the final model. Furthermore, a gated dual-scale cross-attention module (G-DSCAM) is proposed for the fusion of the two feature representations. By performing dimensionality reduction and cross-interaction on the feature information at the two scales, self-attention calculation is completed, allowing for more thorough entanglement and fusion of information at the two scales. In summary, this invention improves segmentation accuracy through the fusion of features at different scales.

[0039] The following specific embodiments further illustrate the beneficial effects of the present invention. Attached Figure Description

[0040] Figure 1 This is a flowchart of the present invention;

[0041] Figure 2 This is a schematic diagram of the three-dimensional image segmentation network model structure in an embodiment of the present invention;

[0042] Figure 3 This is a schematic diagram of the gated shared weight pairwise attention block network structure of the coding block in an embodiment of the present invention;

[0043] Figure 4 This is a schematic diagram of the gated dual-scale cross-attention module network structure for the bottleneck part in an embodiment of the present invention. Detailed Implementation

[0044] The following detailed description, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, features, and effects of a three-dimensional image segmentation method based on a dual-scale cross-attention mechanism proposed in accordance with the present invention.

[0045] See Figure 1 The present invention provides a three-dimensional image segmentation method using a dual-scale cross-attention mechanism, comprising the following steps:

[0046] Step 1: Collect three-dimensional images; the collected CT scan images are divided into training and test sets in an 8:2 ratio; the three-dimensional images are CT scan images.

[0047] Step 2: Image segmentation network model construction: Build a 3D image segmentation network model DS-UNETR++ that applies a dual-scale cross-attention mechanism. The 3D image segmentation network model DS-UNETR++ is an improvement on the UNETR++ model and includes four parts: 3D patch partitioning, encoder, bottleneck, and decoder. The collected 3D images are preprocessed and then input into the network model. Image features are extracted through the 3D patch partitioning, encoder, and bottleneck parts. Then, the decoder is used to reconstruct the image and output the binarized image segmentation region.

[0048] The three-dimensional patch partition includes a convolutional layer and a normalization layer, which project the input three-dimensional image at both coarse and fine scales, converting it into two high-dimensional tensors X1 and X2 at different scales.

[0049] The encoder comprises three stages, each with two coding blocks and a feature fusion block. The two coding blocks receive high-dimensional tensors X1 and X2 of different scales from the 3D patch partition, respectively. The downsampling layer of the coding block is used to capture the hierarchical object characteristics of the feature information of the high-dimensional tensors at different scales. The two gated shared weight pairwise attention submodules (G-SWPA) of the coding block are used to capture the long-range dependencies between the feature information of the high-dimensional tensors at different scales. The convolutional fusion module (Conv-Fusion) receives the high-dimensional tensors output from the two coding blocks, fuses the feature information of the two high-dimensional tensors at different scales through convolution operations, and then outputs it to the decoder.

[0050] The gated shared weight pairwise attention submodule (G-SWPA) of the encoded block adds a spatial attention gate and a channel attention gate to the output stages of its spatial attention and channel attention layers, respectively. The G-SWPA receives a high-dimensional tensor and then inputs it into the spatial attention and channel attention layers to obtain spatial and channel attention graphs. The calculation process is as follows:

[0051]

[0052] in, and These represent spatial and channel attention maps, respectively. SAM and CAM represent spatial attention and channel attention, respectively. G SA G represents the spatial attention gating parameter. CA Q represents the channel attention gating parameter. SW and K SW V represents matrices for shared queries and shared keys, respectively. Spatial and V ChannelThese represent matrices for the spatial value layer and the channel value layer, respectively.

[0053] The spatial attention map Calculation: Taking the input high-dimensional tensor X1 as an example, Q is first calculated using three linear layers based on dimension merging. SW K SW and V Spatial The calculation formula is:

[0054] Q SW =W Q ×X1

[0055] K SW =W K ×X1

[0056] V Spatial =W V-Spatial ×X1

[0057] Among them, W Q W K W V-Spatial They are Q SW K SW V Spatial The projection weight matrix;

[0058] In the specific calculation, the shared key matrix K SW and spatial value layer matrix V Spatial Projecting to m dimensions yields the projected shared-bond matrix K. Proj and the projection space value layer matrix V Proj Then K Proj Transpose and multiply by Q SW After passing through Softmax, multiply by V Proj A preliminary spatial attention map was obtained. Finally multiply by G SA Obtain the final spatial attention map The calculation formula is as follows:

[0059]

[0060] Where d is the scale of the high-dimensional tensor.

[0061] The channel attention map Computation: To further capture the interdependencies between feature channels, a dot product is used to compute the channel attention map. Taking the input high-dimensional tensor X1 as an example, the same Q-mapping algorithm as the spatial attention module is used. SW and K SW The channel attention value is calculated separately using a linear layer, as shown in the following formula:

[0062] VChannel =W V-Channel ×X1

[0063] Among them, W V-Channel It is V Channel The projection weight matrix;

[0064] In the specific calculations, the shared query matrix Q is used. SW transpose multiplied by the shared key matrix K SW After passing through Softmax, it is multiplied by the channel value layer matrix V. Channel Obtain the channel attention map Finally, multiply by the channel attention gating parameter G. CA Obtain the final spatial attention map The calculation formula is as follows:

[0065]

[0066] The bottleneck section includes a stage with two encoding blocks and a feature fusion block. The bottleneck section first receives the output from the encoder section, which is the result of feature extraction of high-dimensional tensors X1 and X2 by the encoder. This result is named high-dimensional tensors F1 and F2. After attention calculation, it is transmitted to the feature fusion block, namely the gated dual-scale cross-attention module (G-DSCAM).

[0067] The gated dual-scale cross-attention module (G-DSCAM) employs a standard Transformer block, allowing high-dimensional tensors of two volumetric scales to interact fully, thereby obtaining more effective long-range dependencies. To control the influence of the two high-dimensional tensors after self-attention computation on the final output, two gating valves are designed to automatically adjust the weight of the two high-dimensional tensors, resulting in stronger generalization performance of the model.

[0068] The gated dual-scale cross-attention module (G-DSCAM) is calculated as follows: A detailed explanation is given using the interaction operation of the coarse-grained high-dimensional tensor F2 as an example. This operation is also applicable to the fine-grained feature information branch. In practice, the operations of these two branches are executed simultaneously. First, F1 is subjected to average pooling, and then it is flattened to obtain a vector representing the entire fine-grained feature information. Next, F2 will also be flattened, reducing it from three dimensions to one dimension. Then spliced ​​together The result is fed into a Transformer for self-attention calculation, then multiplied by the gating parameter G2, and finally removed after the calculation is complete. Then, it is restored from one dimension to three dimensions to obtain the interactive feature information. Finally The final result is obtained by adding it to the original high-dimensional tensor F2. The specific calculation formula is as follows:

[0069]

[0070] The decoder consists of three stages, each with a decoding block. Each decoding block comprises an upsampling layer and a G-SWPA submodule. The upsampling layer uses deconvolution to gradually restore low-resolution features to high resolution. Meanwhile, in order to preserve more detailed information in the segmentation task, skip connections are used to facilitate information transfer across layers.

[0071] The skip connection refers to a high-dimensional tensor information transmission channel built between the encoder and the decoder. Through this channel, the output of each stage of the encoder can be directly transmitted to the corresponding stage in the decoder, so that more detailed information of shallow features can be preserved, the gradient vanishing problem can be mitigated, and the problem of sample imbalance can be overcome, thereby improving the performance and generalization ability of the model.

[0072] Step 3: Image Segmentation Network Model Training: Train the 3D image segmentation network model DS-UNETR++ using a dual-scale cross-attention mechanism on the input training set. Evaluate and optimize the DS-UNETR++ model using the test set. During training, the hyperparameters are set as follows: initial learning rate of 0.01, weight decay of 3e-5, training duration of 1000 epochs, and the loss function is the Dice+CE Loss function, denoted as L(G,P).

[0073]

[0074] Where G refers to the set of actual results, and P refers to the set of predicted results. i,j and G i,j Let I and J represent the probability output and one-hot encoded true value of class j at voxel i, respectively; I is the number of voxels; and J is the number of classes.

[0075] Step 4: Image segmentation: Perform 3D image segmentation using the trained image segmentation network model.

[0076] Specifically, taking the multi-organ CT segmentation dataset (Synapse) as an example, the workflow of a 3D image segmentation method applying a dual-scale cross-attention mechanism is described below:

[0077] Step 1: Preprocess the Synapse dataset, which mainly consists of 30 abdominal CT scan images in 8 categories. Divide the dataset into training and test sets in an 8:2 ratio.

[0078] Step 2: Create the DS-UNETR++ network model, which mainly includes four parts: 3D patch partitioning, encoder, bottleneck, and decoder. Input the preprocessed image into the network, extract sufficient image features through four layers of feature extraction, and then use the decoder to restore the image, finally obtaining the binarized image segmentation region. Figure 2 The structure of the DS-UNETR++ network model is shown.

[0079] Step 3: For the creation of the DS-UNETR++ network model, firstly, a three-dimensional patch partition is constructed and projected at both coarse-grained and fine-grained scales, transforming it into two different high-dimensional tensors. This is mainly divided into... and

[0080] Step 4: The encoder part of the DS-UNETR++ network model is mainly divided into three stages. Each stage has two encoding blocks and one feature fusion block. The two encoding blocks receive high-dimensional tensors X1 and X2 from different scales of the 3D patch partitions, respectively. The downsampling layer inside the encoding block is used to capture the hierarchical object characteristics of feature information at different scales. Then, the long-range dependencies between patches are captured through two gated shared weight pairwise attention (G-SWPA) submodules inside the encoding block. Then, the high-dimensional tensors X1 and X2 are output to the convolutional fusion block (Conv-Fusion), where the feature information of the two scales is fused through two convolution operations, and then output to the decoder stage.

[0081] Step 4.1: Figure 3 This paper demonstrates the network structure of a gated shared weight pairwise attention block (G-SWPA) for a single coding block, with a gating valve added at the output stages of spatial attention and channel attention. First, the G-SWPA module receives the feature map X, which is then input into the channel and spatial attention modules of the G-SWPA block. The weights Q and K of the two attention modules are shared, but different V values ​​are used. The specific expressions are as follows:

[0082]

[0083] This module mainly involves the calculation of spatial attention and channel attention, which will be explained in detail below:

[0084] Spatial attention: For the input feature map X, the width W, height H, and depth D are first merged to form a shape of HWD×C. Then, Q is calculated using three linear layers. SW K SW and V Spatial , will K SW and V Spatial Projecting onto m dimensions yields KProj and V Proj Then K Proj Transpose and multiply by Q SW After passing through Softmax, multiply by V Proj Obtain spatial attention map Finally multiply by G SA Obtain the final spatial attention map The calculation formulas are as follows:

[0085]

[0086] Channel attention: For the input feature map X, the same Q-mapping mechanism as the spatial attention module is used. SW and K SW The V value of channel attention is calculated separately using a linear layer, and Q is used. SW transpose multiplied by K SW After passing through Softmax, multiply by V Channel Obtain the channel attention map Finally multiply by G CA Obtain the final spatial attention map The specific calculation formula is as follows:

[0087]

[0088] Step 4.2: Figure 4 This paper demonstrates the network structure of the gated dual-scale cross-attention module (G-DSCAM) in the bottleneck section of this invention. It employs a standard Transformer block, allowing sufficient interaction between feature information from two volumetric scales to obtain more effective long-range dependencies. To control the influence of the two feature information types after self-attention computation on the final output, two gating valves are designed to automatically adjust the weight of the two feature information types, resulting in stronger generalization performance of the model.

[0089] In this invention, G-DSCAM enables further interaction between features from two branches at different scales. The following detailed explanation uses the interaction operation of the coarse-grained feature information branch as an example; this operation also applies to the fine-grained feature information branch. In practice, the operations of these two branches are executed simultaneously. This invention denotes the input fine-grained feature information as... Coarse-grained feature information is denoted as First, perform average pooling on F1, then flatten it to obtain a vector representing the entire fine-grained feature information. Next, F2 is also flattened, which flattens it from three dimensions to one dimension. Then spliced ​​together The result is fed into a Transformer for self-attention calculation, and then multiplied by the gating parameter G2 designed in this invention to obtain a new value. Remove after calculation is complete. Get the final Then Restored to 3D, obtaining the interactive feature information. Then, it is added to the original input features. The specific calculation formula is as follows:

[0090]

[0091] Step 5: In this embodiment of the invention, the decoder part of the DS-UNETR++ network architecture is mainly divided into three stages, each stage having a decoding block. Each decoding block consists of an upsampling layer and a G-SWPA submodule. In contrast to the downsampling layer, this embodiment uses deconvolution in the upsampling layer to progressively restore low-resolution features to high resolution. Simultaneously, to preserve more detailed information in the segmentation task, skip connections are used to promote cross-layer information transfer, allowing more detailed information from lower-level features to be preserved, mitigating gradient vanishing, overcoming sample imbalance, and thus improving the model's performance and generalization ability.

[0092] Step 6: Train the network model and obtain the weights for training the DS-UNETR++ network model.

[0093] Step 6.1: Steps 1 to 5 above mainly describe the construction of the DS-UNETR++ network model. In this embodiment of the invention, the specific training process is as follows:

[0094] The system environment is Ubuntu 20.04, using PyCharm software and the PyTorch framework (torch 1.11.0 + cu113). The DS-UNETR++ model is created in PyTorch and run on a single NVIDIA 4090 24G GPU. Training hyperparameters are set as follows: initial learning rate of 0.01, weight decay of 3e-5, training duration of 1000 epochs, and the Dice+CE Loss function, whose mathematical expression is as follows:

[0095]

[0096] Where G refers to the set of actual results, and P refers to the set of predicted results. i,j and G i,j Let I and J represent the probability output and one-hot encoded true value of class j at voxel i, respectively; I is the number of voxels; and J is the number of classes.

[0097] Step 6.2: Run the program, train the model, and save the final model training weights.

[0098] Step 7: On the test set, run the DS-UNETR++ network model using the trained weights and obtain the Synapse image segmentation results. Use DSC and HD95 as evaluation metrics. The specific test results are as follows:

[0099] Table 1 Test Results

[0100]

[0101] Table 1 presents the DSC and HD95 scores of U-Net, TransUNet, Swin-UNet, UNETR, MISSFormer, Swin-UNETR, UNETR++, and our model on the Synapse dataset, and calculates the final average scores for eight categories. UNETR++ performs well across all categories, achieving the best performance in average DSC and HD95 scores of 87.22% and 7.53, respectively. In comparison, DS-UNETR++ achieves even better results, with an average DSC of 87.75%, 0.53% higher than the baseline model, and an average HD95 of 6.67, 0.86 lower than the baseline model. Furthermore, in the specific category index analysis, DS-UNETR++ achieves the highest scores in six categories: right kidney, left kidney, gallbladder, liver, aorta, and pancreas. The segmentation performance of the right kidney and pancreas shows particularly significant improvements, increasing by 4.87% and 1.05% respectively compared to the baseline model.

[0102] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Any simple modifications, equivalent changes, and alterations made to the above embodiments without departing from the technical essence of the present invention shall still fall within the scope of the present invention.

Claims

1. A three-dimensional image segmentation method applying a dual-scale cross-attention mechanism, characterized in that: This method Includes the following steps: Includes the following steps: Step 1: Collect 3D images and divide them into training and test sets; Step 2: Image segmentation network model construction: Build a 3D image segmentation network model DS-UNETR++ that applies a dual-scale cross-attention mechanism. The 3D image segmentation network model DS-UNETR++ is an improvement on the UNETR++ model and includes four parts: 3D patch partitioning, encoder, bottleneck, and decoder. The collected 3D images are preprocessed and then input into the network model. Image features are extracted through the 3D patch partitioning, encoder, and bottleneck parts. Then, the decoder is used to reconstruct the image and output the binarized image segmentation region. The three-dimensional patch partition includes a convolutional layer and a normalization layer, which project the input three-dimensional image at both coarse and fine scales, converting it into two high-dimensional tensors X1 and X2 at different scales. The encoder comprises three stages, each with two encoding blocks and a feature fusion block. The two encoding blocks receive high-dimensional tensors X1 and X2 from different scales of the 3D patch partition, respectively. The downsampling layer of the encoding block is used to capture the hierarchical object characteristics of the feature information of the high-dimensional tensors at different scales. The two gated shared weight pairwise attention submodules of the encoding block are used to capture the long-range dependencies between the feature information of the high-dimensional tensors at different scales. The convolutional fusion module receives the high-dimensional tensors output from the two encoding blocks and fuses the feature information of the two scales of high-dimensional tensors through convolution operations, with its outputs represented as F1 and F2. The gated shared weight pairwise attention submodule of the encoding block adds a spatial attention and a channel attention gate to the output stages of its spatial attention and channel attention layers, respectively. The gated shared weight pairwise attention submodule receives the high-dimensional tensors and then inputs them into the spatial attention and channel attention layers to obtain spatial and channel attention maps. The calculation process is represented as follows: in, and These represent spatial and channel attention maps, respectively. SAM and CAM represent spatial attention and channel attention, respectively. G SA G represents the spatial attention gating parameter. CA Q represents the channel attention gating parameter. SW and K SW V represents matrices for shared queries and shared keys, respectively. Spatial and V Channel These represent matrices for the spatial value layer and the channel value layer, respectively. The bottleneck section includes a feature fusion block. The bottleneck section first receives the outputs F1 and F2 from the encoder section, and after attention calculation, transmits them to the feature fusion block, i.e., the gated dual-scale cross-attention module. This gated dual-scale cross-attention module uses a standard Transformer block to allow sufficient interaction between two high-dimensional tensors of different volumetric scales, further obtaining more effective long-range dependencies. To control the influence of the two high-dimensional tensors after self-attention calculation on the final output, two gate valves are designed to automatically adjust the proportion of the two high-dimensional tensors, making the model's generalization performance stronger. The decoder consists of three stages, each with a decoding block. Each decoding block comprises an upsampling layer and a G-SWPA submodule. The upsampling layer uses deconvolution to progressively restore low-resolution features to high resolution. Meanwhile, skip connections are used to facilitate information transfer across layers in order to preserve more detailed information in the segmentation task. Step 3: Image segmentation network model training: Train the 3D image segmentation network model DS-UNETR++ using a dual-scale cross-attention mechanism on the input training set, and evaluate and optimize the 3D image segmentation network model DS-UNETR++ using the test set; Step 4: Image segmentation: Perform 3D image segmentation using the trained image segmentation network model.

2. The three-dimensional image segmentation method applying a dual-scale cross-attention mechanism as described in claim 1, characterized in that: In step 1, the three-dimensional image is a CT scan image.

3. The three-dimensional image segmentation method using a dual-scale cross-attention mechanism as described in claim 1, characterized in that: In step 1, the collected CT scan images are divided into a training set and a test set in an 8:2 ratio.

4. The three-dimensional image segmentation method using a dual-scale cross-attention mechanism as described in claim 1, characterized in that: In step 2, the spatial attention map Calculation: Taking the input high-dimensional tensor X1 as an example, Q is first calculated using three linear layers based on dimension merging. SW K SW and V Spatial The calculation formula is: Among them, W Q W K W V-Spatial They are Q SW K SW V Spatial The projection weight matrix; In the specific calculation, the shared key matrix K SW and spatial value layer matrix V Spatial Projecting to m dimensions yields the projected shared-bond matrix K. Proj and the projection spatial value layer matrix V Proj Then K Proj Transpose and multiply by Q SW After passing through Softmax, multiply by V Proj Obtain a preliminary spatial attention map Finally, multiply by G SA Obtain the final spatial attention map The calculation formula is as follows: Where d is the scale of the high-dimensional tensor.

5. A three-dimensional image segmentation method using a dual-scale cross-attention mechanism as described in claim 1, characterized in that: In step 2, the channel attention map Computation: To further capture the interdependencies between feature channels, a dot product is used to compute the channel attention map. Taking the input high-dimensional tensor X1 as an example, the same Q-mapping algorithm as the spatial attention module is used. SW and K SW The channel attention value is calculated separately using a linear layer, as shown in the following formula: Among them, W V-Channel It is V Channel The projection weight matrix; In the specific calculations, the shared query matrix Q is used. SW transpose multiplied by the shared key matrix K SW After passing through Softmax, it is multiplied by the channel value layer matrix V. Channel Obtain the channel attention map Finally, multiply by the channel attention gating parameter G. CA Obtain the final spatial attention map The calculation formula is as follows: 。 6. The three-dimensional image segmentation method applying a dual-scale cross-attention mechanism as described in claim 1, characterized in that: In step 2, the gated dual-scale cross-attention module is calculated as follows: Taking the interaction operation of the coarse-grained high-dimensional tensor F2 as an example, F1 is first subjected to average pooling, and then flattened to obtain a vector representing the entire fine-grained feature information. Next, F2 will also be flattened, reducing it from three dimensions to one dimension. Then spliced ​​together The result is fed into a Transformer for self-attention calculation, then multiplied by the gating parameter G2, and finally removed after the calculation is complete. And it is restored from one dimension to three dimensions to obtain the feature information after interaction. Finally The final result is obtained by adding it to the original high-dimensional tensor F2. The specific calculation formula is as follows: 。 7. A three-dimensional image segmentation method using a dual-scale cross-attention mechanism as described in claim 1, characterized in that: In step 2, the skip connection refers to the high-dimensional tensor information transmission channel built between the encoder and the decoder. Through this channel, the output of each stage of the encoder can be directly transmitted to the corresponding stage in the decoder, so that more detailed information of shallow features can be preserved and gradient vanishing can be mitigated.

8. The three-dimensional image segmentation method using a dual-scale cross-attention mechanism as described in claim 1, characterized in that: In step 3, when training the DS-UNETR++ 3D image segmentation network model using the dual-scale cross-attention mechanism, the training hyperparameters are set as follows: initial learning rate of 0.01, weight decay of 3e-5, training period of 1000 epochs, and loss function.

9. A three-dimensional image segmentation method applying a dual-scale cross-attention mechanism as described in claim 8, characterized in that: The loss function used is the Dice+CE Loss function, denoted as L(G,P): Where G refers to the set of actual results, and P refers to the set of predicted results. i,j and G i,j Let I and J represent the probability output and one-hot encoded true value of class j at voxel i, respectively. I is the number of voxels and J is the number of classes.

Images