An image extraction method based on three-dimensional attention mechanism

Abstract

The present invention relates to the field of medical artificial intelligence technology, specifically a method for image extraction based on a three-dimensional attention mechanism, the steps of which are as follows: creating a data set, then dividing the data set into a training set and a test set; cropping the training set and the test set, then converting the cropped images into a unified format to obtain pre-processed training set and test set; building a network structure, configuring it, inputting the pre-processed training set into the network structure for training, iterating the training process multiple times, continuously updating the network structure parameters, saving the optimal weights and biases obtained during training, and completing the training of the network structure; inputting the saved optimal weights and biases and the image data in the pre-processed test set into the trained network structure, and outputting the extraction results of the final pre-processed test set. The present invention is mainly used for processing and extracting images to improve image quality and enhance image information, and better understand the long-distance dependencies in images.

Description

Technical Field

[0001] The present invention relates to the field of medical artificial intelligence technology, and in particular to an image extraction method based on a three-dimensional attention mechanism. Background Art

[0002] Technological development in the field of medical artificial intelligence has made tremendous progress in the past few years and is expected to continue rapidly in the future. The application of deep learning technology in medical image analysis has achieved significant breakthroughs. This includes automated analysis and diagnosis of images such as X-rays, CT scans, and MRIs. Artificial intelligence algorithms can help doctors more accurately detect and diagnose various diseases, such as tumors and lesions. Leveraging machine learning and big data technologies, personalized diagnosis, treatment, and prevention plans can be provided to patients based on information such as their genome, biomarkers, and clinical data. Personalized medicine can improve treatment outcomes, reduce adverse reactions, and lower healthcare costs. Artificial intelligence technology can assist doctors in making diagnostic and treatment decisions. By analyzing large amounts of clinical data and medical literature, AI systems can provide case-specific advice and guidance, helping doctors make more accurate decisions.

[0003] In recent years, the rapid development of computer-assisted diagnosis (CAD) has gradually replaced manual analysis of patient pathology images. CAD assists doctors in diagnosing diseases by providing additional information and analysis. Automation and computer-assisted technologies can accelerate the diagnostic process, saving both doctors and patients time. The advent of UNet has made a significant stride forward in the field of artificial intelligence (AI) medical image extraction. Its high-precision extraction results, compatibility with images of varying sizes, minimal need for annotated data, and rapid training and inference speeds demonstrate its widespread value in clinical applications.

[0004] In the field of three-dimensional medical data, many technologies and methods are currently being used to improve the accuracy and efficiency of extraction. CNNs have been widely used in medical image extraction. By utilizing convolution and pooling, CNNs can automatically learn features in images. For three-dimensional images, 3DCNNs are designed to process three-dimensional data directly or convert it into 2D slices for processing. U-Net is a very common medical image extraction architecture with an encoder and decoder structure, and the decoder uses skip connections to retain more detailed information. The self-attention mechanism can help the network better understand long-range dependencies in images when performing extraction tasks and improve extraction accuracy.

[0005] Therefore, the present invention proposes an image extraction method based on three-dimensional attention mechanism to solve the above problems. Summary of the Invention

[0006] In response to the shortcomings of the existing technology, the present invention develops an image extraction method based on a three-dimensional attention mechanism. The present invention can help the network better understand the long-distance dependencies in the image when performing extraction tasks and improve the accuracy of extraction.

[0007] The technical solution to the technical problem solved by the present invention is an image extraction method based on a three-dimensional attention mechanism, comprising the following steps:

[0008] S1. Select existing 3D medical image data as a dataset, and then divide the dataset into a training set and a test set in proportion;

[0009] S2. Perform centroid cropping on the training set and test set, and then convert the cropped image data into a unified format to obtain the preprocessed training set and test set;

[0010] S3. Build and configure the network structure, input the preprocessed training set into the network structure for training, iterate the training process multiple times, continuously update the network structure parameters, save the optimal weights and biases obtained during training, and complete the training of the network structure;

[0011] S4. Input the saved optimal weights and biases, as well as the image data in the preprocessed test set, into the trained network structure, and output the final extraction results of the preprocessed test set.

[0012] S1 is as follows:

[0013] The data set is divided into training set in a ratio of 8:2 and test set , training set , test set ,in, represents the number of images in the training set, represents the number of images in the test set, , Indicates the first images, Indicates the first images.

[0014] S2 is as follows:

[0015] Perform centroid cropping on the image data in the training set and test set, and then convert the cropped images into a unified format to obtain the preprocessed training set and test set , where the training set after preprocessing , the preprocessed test set , represents the first images, Represents the first images.

[0016] S3 is as follows:

[0017] The network structure consists of four parts: encoder, decoder, three MFF multi-feature fusion modules and three 3DAEM three-dimensional attention enhancement mechanism modules. The encoder and decoder are composed of four convolution blocks. The four convolution blocks of the encoder are composed of two convolution layers with a convolution kernel size of 3, a step size of 1 and a padding of 1, an InstanceNorm layer, a ReLU layer and a maximum pooling layer with a downsampling coefficient of 2. The four convolution blocks of the decoder are composed of an upsampling block with a kernel size of 2 and two convolution layers with a convolution kernel size of 3, a step size of 2 and a padding of 1, an InstanceNorm layer and a ReLU activation function. After the network structure is constructed, the network structure is configured and the CosineAnnealLingLR learning rate adjustment strategy, Adam optimizer and back propagation algorithm are used to optimize the network structure. Then the preprocessed training set is trained. Input into the network structure for training until the final feature map result is output, and the weights of the network structure training process are saved and offset , and then iterate the training process multiple times to continuously update the weights of the network structure and offset , save the optimal weight and offset ;

[0018] Among them, the preprocessed training set The image is input into the network structure for training. The training process is mainly divided into two parts: the encoder processing stage and the decoder processing stage.

[0019] Encoder processing stage:

[0020] (1) The preprocessed training set Any image in Input to the first convolution block of the encoder and output the feature map , and then the feature map Input into the first MFF module to obtain low-level feature maps ;

[0021] (2) Low-level feature maps Input to the second convolution block of the encoder to obtain the feature map , and then the feature map and feature maps Input to the second MFF module, first the feature map Perform the maximum pooling operation with a kernel of 2 and then add it to the feature map Perform a convolution operation with a convolution kernel of 1 to obtain a low-level feature map , The calculation formula is:

[0022] ,

[0023] in, represents the maximum pooling operation with a kernel of 2, Represents a convolution operation with a convolution kernel of 1, Represents a splicing operation;

[0024] (3) Low-level feature maps Input to the third convolution block of the encoder to get the feature map , and then the feature map , feature map and feature maps Input to the third MFF module, first the feature map Perform the maximum pooling operation with a kernel of 4, Perform the maximum pooling operation with a kernel of 2, and then combine the two with the feature map Perform a convolution operation with a convolution kernel of 1 to obtain a low-level feature map , The calculation formula is:

[0025] ,

[0026] in, represents the maximum pooling operation with a kernel of 2, Indicates the maximum pooling operation with a kernel of 4, Represents a convolution operation with a convolution kernel of 1, Represents a splicing operation;

[0027] (4) Low-level feature maps Input to the fourth convolution block of the encoder to get the feature map .

[0028] Decoder processing stage:

[0029] (1) Feature map Input to the first convolutional block of the decoder to obtain high-level feature maps , and then the high-level feature map and low-level feature maps Input to the first 3DAEM module, the high-level feature map Perform bilinear interpolation upsampling and then upsample the high-level feature map With low-level feature maps Splicing to get feature map ;

[0030] Feature Map The three dimensions X, Y, and Z are averaged and pooled respectively. The average values ​​of the pooling kernels in the three dimensions are (X, 1, 1), (1, Y, 1), and (1, 1, Z), respectively. X, Y, and Z represent the values ​​of the height, width, and depth, respectively. The one-dimensional features obtained after the pooling operation are then subjected to the Concat splicing operation, the convolution operation with a kernel of 1, the BatchNorm normalization operation, the ReLU activation function, and the Split extraction operation. The one-dimensional features obtained after the Concat splicing operation, the convolution operation with a kernel of 1, the BatchNorm normalization operation, and the ReLU activation function are processed through the Split operation. Extract according to the three dimensions of height h, width w, and depth d. The shape of the extracted one-dimensional feature is the same as the shape of the one-dimensional feature obtained after the pooling operation. Then, the extracted one-dimensional feature is uniformly subjected to the convolution operation with a kernel of 1 and the Sigmoid activation function operation to obtain the attention coefficient of the three dimensions. 、 and , the attention coefficients of the three dimensions 、 、 and feature maps Multiply to get the attention feature map , the specific calculation process is:

[0031] ,

[0032] ,

[0033] ,

[0034] ,

[0035] ,

[0036] ,

[0037] in, represents upsampling, represents matrix concatenation, Indicates the average pooling operation on the X dimension. Indicates the average pooling operation on the Y dimension. Indicates the average pooling operation on the Z dimension. Indicates the convolution calculation with the convolution kernel of 1, represents the normalization operation, Represents the activation function Operation, Indicates the extraction operation on the X dimension. Indicates the extraction operation on the Y dimension. Indicates the extraction operation on the Z dimension. Represents the Sigmoid activation function;

[0038] (2) Attention feature map Input to the second convolutional block of the decoder to obtain high-level feature maps , and then the high-level feature map and low-level feature maps Input to the second 3DAEM module, the high-level feature map Perform bilinear interpolation upsampling and then upsample the high-level feature map With low-level feature maps Splicing to get feature map ;

[0039] Feature Map The three dimensions X, Y, and Z are averaged and pooled respectively. The average values ​​of the pooling kernels in the three dimensions are (X, 1, 1), (1, Y, 1), and (1, 1, Z), respectively. X, Y, and Z represent the values ​​of the height, width, and depth dimensions, respectively. The one-dimensional features obtained after the pooling operation are then subjected to the Concat splicing operation, the convolution operation with a kernel of 1, BatchNorm normalization, the ReLU activation function, and the Split operation. The one-dimensional features obtained after the Concat splicing operation, the convolution operation with a kernel of 1, the BatchNorm normalization, and the ReLU activation function are processed through the Split operation. According to the three dimensions of height h, width w, and depth d, the shape of the extracted one-dimensional feature is the same as the shape of the one-dimensional feature obtained after the pooling operation. Then the Sigmoid activation function operation is uniformly performed on the extracted one-dimensional features to obtain the attention coefficient of the three dimensions 、 and , the attention coefficients of the three dimensions 、 、 and feature maps Multiply to get the attention feature map , the specific calculation process is:

[0040] ,

[0041] ,

[0042] ,

[0043] ,

[0044] ,

[0045] ,

[0046] in, represents upsampling, represents matrix concatenation, Indicates the average pooling operation on the X dimension. Indicates the average pooling operation on the Y dimension. Indicates the average pooling operation on the Z dimension. Indicates the convolution calculation with the convolution kernel of 1, represents the normalization operation, Represents the activation function Operation, Indicates the extraction operation on the X dimension. Indicates the extraction operation on the Y dimension. Indicates the extraction operation on the Z dimension. Represents the Sigmoid activation function;

[0047] (3) Attention feature map Input to the third convolutional block of the decoder to obtain high-level feature maps , and then the high-level feature map and low-level feature maps Input to the third 3DAEM module, the high-level feature map Perform bilinear interpolation upsampling and then upsample the high-level feature map With low-level feature maps Splicing to get feature map ;

[0048] Feature Map The three dimensions X, Y, and Z are averaged and pooled respectively. The average values ​​of the pooling kernels in the three dimensions are (X, 1, 1), (1, Y, 1), and (1, 1, Z), respectively. X, Y, and Z represent the values ​​of the height, width, and depth, respectively. The one-dimensional vector obtained after the pooling operation is then subjected to the Concat splicing operation, the convolution operation with a kernel of 1, BatchNorm normalization, the ReLU activation function, and the Split operation. The one-dimensional features obtained after the Concat splicing operation, the convolution operation with a kernel of 1, the BatchNorm normalization, and the ReLU activation function are extracted according to the three dimensions of height h, width w, and depth d through the Split operation. The shape of the extracted one-dimensional features is the same as the shape of the one-dimensional features obtained after the pooling operation. The Sigmoid activation function is then uniformly performed on the extracted one-dimensional features to obtain the attention coefficients of the three dimensions. 、 and , the attention coefficients of the three dimensions 、 、 and feature maps Multiply to get the attention feature map , the specific calculation process is:

[0049] ,

[0050] ,

[0051] ,

[0052] ,

[0053] ,

[0054] ,

[0055] in, represents upsampling, represents matrix concatenation, Indicates the average pooling operation on the X dimension. Indicates the average pooling operation on the Y dimension. Indicates the average pooling operation on the Z dimension. Indicates the convolution calculation with the convolution kernel of 1, represents the normalization operation, Represents the activation function Operation, Indicates the extraction operation on the X dimension. Indicates the extraction operation on the Y dimension. Indicates the extraction operation on the Z dimension. Represents the Sigmoid activation function;

[0056] (4) Attention feature map The fourth convolution block input to the decoder obtains the final image extraction result .

[0057] The effects provided in the summary of the invention are only the effects of the embodiments, rather than all the effects of the invention. The above technical solution has the following advantages or beneficial effects:

[0058] The present invention applies the attention mechanism to the dimensionality of the image, and achieves the effect of enhancing the image's area of ​​interest by calculating the attention coefficient of each dimension. Secondly, the present invention fully integrates the feature map generated by the encoder end, solving the problem of poor extraction effect caused by image size differences. At the same time, the dimensional attention mechanism also captures information on the three-dimensional spatial structure, improving the performance of the model in processing three-dimensional data. In addition, at the jump connection between the encoder and the decoder, the present invention uses the feature map generated by each encoder as a low-level feature branch to input into the 3DAEM module, and at the same time, the high-level features generated by the lower-level decoder are also input into the 3DAEM module as another branch, thereby enhancing the model's attention to the relevant parts of the features generated by the encoder end, thereby improving the application of encoder features in the decoder stage. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0060] Figure 1 Schematic diagram of the network structure in the present invention.

[0061] Figure 2 Schematic diagram of the 3DAEM structure in the present invention.

[0062] Figure 3 Comparison of image extraction results between the method in the present invention and the existing method. DETAILED DESCRIPTION

[0063] To clearly illustrate the technical features of this solution, the present invention is described in detail below through specific embodiments and in conjunction with the accompanying drawings. The following disclosure provides many different embodiments or examples for implementing different structures of the present invention. To simplify the disclosure of the present invention, the components and configurations of specific examples are described below.

[0064] Example 1

[0065] An image extraction method based on a three-dimensional attention mechanism includes the following steps:

[0066] S1. Select existing 3D medical image data as a dataset, and then divide the dataset into a training set and a test set in proportion;

[0067] S2. Perform centroid cropping on the training set and test set, and then convert the cropped image data into a unified format to obtain the preprocessed training set and test set;

[0068] S3. Build and configure the network structure, input the preprocessed training set into the network structure for training, iterate the training process multiple times, continuously update the network structure parameters, save the optimal weights and biases obtained during training, and complete the training of the network structure;

[0069] S4. Input the saved optimal weights and biases, as well as the image data in the preprocessed test set, into the trained network structure, and output the final extraction results of the preprocessed test set.

[0070] S1 is as follows:

[0071] The data set is divided into training set in a ratio of 8:2 and test set , training set , test set ,in, represents the number of images in the training set, represents the number of images in the test set, , Indicates the first images, Indicates the first images.

[0072] S2 is as follows:

[0073] Perform centroid cropping on the image data in the training set and test set, and then convert the cropped images into a unified format to obtain the preprocessed training set and test set , where the training set after preprocessing , the preprocessed test set , represents the first images, Represents the first images.

[0074] S3 is as follows:

[0075] The network structure consists of four parts: encoder, decoder, three MFF multi-feature fusion modules and three 3DAEM three-dimensional attention enhancement mechanism modules. The encoder and decoder are composed of four convolution blocks. The four convolution blocks of the encoder are composed of two convolution layers with a convolution kernel size of 3, a step size of 1 and a padding of 1, an InstanceNorm layer, a ReLU layer and a maximum pooling layer with a downsampling coefficient of 2. The four convolution blocks of the decoder are composed of an upsampling block with a kernel size of 2 and two convolution layers with a convolution kernel size of 3, a step size of 2 and a padding of 1, an InstanceNorm layer and a ReLU activation function. After the network structure is constructed, the network structure is configured and the CosineAnnealLingLR learning rate adjustment strategy, Adam optimizer and back propagation algorithm are used to optimize the network structure. Then the preprocessed training set is trained. Input into the network structure for training until the final feature map result is output, and the weights of the network structure training process are saved and offset , and then iterate the training process multiple times to continuously update the weights of the network structure and offset , save the optimal weight and offset ;

[0076] Among them, the preprocessed training set The image is input into the network structure for training. The training process is mainly divided into two parts: the encoder processing stage and the decoder processing stage.

[0077] Encoder processing stage:

[0078] (1) The preprocessed training set Any image in Input to the first convolution block of the encoder and output the feature map , and then the feature map Input into the first MFF module to obtain low-level feature maps ;

[0079] (2) Low-level feature maps Input to the second convolution block of the encoder to obtain the feature map , and then the feature map and feature maps Input to the second MFF module, first the feature map Perform the maximum pooling operation with a kernel of 2 and then add it to the feature map Perform a convolution operation with a convolution kernel of 1 to obtain a low-level feature map , The calculation formula is:

[0080] ,

[0081] in, represents the maximum pooling operation with a kernel of 2, Represents a convolution operation with a convolution kernel of 1, Represents a splicing operation;

[0082] (3) Low-level feature maps Input to the third convolution block of the encoder to get the feature map , and then the feature map , feature map and feature maps Input to the third MFF module, first the feature map Perform the maximum pooling operation with a kernel of 4, Perform the maximum pooling operation with a kernel of 2, and then combine the two with the feature map Perform a convolution operation with a convolution kernel of 1 to obtain a low-level feature map , The calculation formula is:

[0083] ,

[0084] in, represents the maximum pooling operation with a kernel of 2, Indicates the maximum pooling operation with a kernel of 4, Represents a convolution operation with a convolution kernel of 1, Represents a splicing operation;

[0085] (4) Low-level feature maps Input to the fourth convolution block of the encoder to get the feature map .

[0086] Decoder processing stage:

[0087] (1) Feature map Input to the first convolutional block of the decoder to obtain high-level feature maps , and then the high-level feature map and low-level feature maps Input to the first 3DAEM module, the high-level feature map Perform bilinear interpolation upsampling and then upsample the high-level feature map With low-level feature maps Splicing to get feature map ;

[0088] Feature Map The three dimensions X, Y, and Z are averaged and pooled respectively. The average values ​​of the pooling kernels in the three dimensions are (X, 1, 1), (1, Y, 1), and (1, 1, Z), respectively. X, Y, and Z represent the values ​​of the height, width, and depth, respectively. The one-dimensional features obtained after the pooling operation are then subjected to the Concat splicing operation, the convolution operation with a kernel of 1, the BatchNorm normalization operation, the ReLU activation function, and the Split extraction operation. The one-dimensional features obtained after the Concat splicing operation, the convolution operation with a kernel of 1, the BatchNorm normalization operation, and the ReLU activation function are processed through the Split operation. Extract according to the three dimensions of height h, width w, and depth d. The shape of the extracted one-dimensional feature is the same as the shape of the one-dimensional feature obtained after the pooling operation. Then, the extracted one-dimensional feature is uniformly subjected to the convolution operation with a kernel of 1 and the Sigmoid activation function operation to obtain the attention coefficient of the three dimensions. 、 and , the attention coefficients of the three dimensions 、 、 and feature maps Multiply to get the attention feature map , the specific calculation process is:

[0089] ,

[0090] ,

[0091] ,

[0092] ,

[0093] ,

[0094] ,

[0095] in, represents upsampling, represents matrix concatenation, Indicates the average pooling operation on the X dimension. Indicates the average pooling operation on the Y dimension. Indicates the average pooling operation on the Z dimension. Indicates the convolution calculation with the convolution kernel of 1, represents the normalization operation, Represents the activation function Operation, Indicates the extraction operation on the X dimension. Indicates the extraction operation on the Y dimension. Indicates the extraction operation on the Z dimension. Represents the Sigmoid activation function;

[0096] (2) Attention feature map Input to the second convolutional block of the decoder to obtain high-level feature maps , and then the high-level feature map and low-level feature maps Input to the second 3DAEM module, the high-level feature map Perform bilinear interpolation upsampling and then upsample the high-level feature map With low-level feature maps Splicing to get feature map ;

[0097] Feature Map The three dimensions X, Y, and Z are averaged and pooled respectively. The average values ​​of the pooling kernels in the three dimensions are (X, 1, 1), (1, Y, 1), and (1, 1, Z), respectively. X, Y, and Z represent the values ​​of the height, width, and depth dimensions, respectively. The one-dimensional features obtained after the pooling operation are then subjected to the Concat splicing operation, the convolution operation with a kernel of 1, BatchNorm normalization, the ReLU activation function, and the Split operation. The one-dimensional features obtained after the Concat splicing operation, the convolution operation with a kernel of 1, the BatchNorm normalization, and the ReLU activation function are processed through the Split operation. According to the three dimensions of height h, width w, and depth d, the shape of the extracted one-dimensional feature is the same as the shape of the one-dimensional feature obtained after the pooling operation. Then the Sigmoid activation function operation is uniformly performed on the extracted one-dimensional features to obtain the attention coefficient of the three dimensions 、 and , the attention coefficients of the three dimensions 、 、 and feature maps Multiply to get the attention feature map , the specific calculation process is:

[0098] ,

[0099] ,

[0100] ,

[0101] ,

[0102] ,

[0103] ,

[0104] in, represents upsampling, represents matrix concatenation, Indicates the average pooling operation on the X dimension. Indicates the average pooling operation on the Y dimension. Indicates the average pooling operation on the Z dimension. Indicates the convolution calculation with the convolution kernel of 1, represents the normalization operation, Represents the activation function Operation, Indicates the extraction operation on the X dimension. Indicates the extraction operation on the Y dimension. Indicates the extraction operation on the Z dimension. Represents the Sigmoid activation function;

[0105] (3) Attention feature map Input to the third convolutional block of the decoder to obtain high-level feature maps , and then the high-level feature map and low-level feature maps Input to the third 3DAEM module, the high-level feature map Perform bilinear interpolation upsampling and then upsample the high-level feature map With low-level feature maps Splicing to get feature map ;

[0106] Feature Map The three dimensions X, Y, and Z are averaged and pooled respectively. The average values ​​of the pooling kernels in the three dimensions are (X, 1, 1), (1, Y, 1), and (1, 1, Z), respectively. X, Y, and Z represent the values ​​of the height, width, and depth, respectively. The one-dimensional vector obtained after the pooling operation is then subjected to the Concat splicing operation, the convolution operation with a kernel of 1, BatchNorm normalization, the ReLU activation function, and the Split operation. The one-dimensional features obtained after the Concat splicing operation, the convolution operation with a kernel of 1, the BatchNorm normalization, and the ReLU activation function are extracted according to the three dimensions of height h, width w, and depth d through the Split operation. The shape of the extracted one-dimensional features is the same as the shape of the one-dimensional features obtained after the pooling operation. The Sigmoid activation function is then uniformly performed on the extracted one-dimensional features to obtain the attention coefficients of the three dimensions. 、 and , the attention coefficients of the three dimensions 、 、 and feature maps Multiply to get the attention feature map , the specific calculation process is:

[0107] ,

[0108] ,

[0109] ,

[0110] ,

[0111] ,

[0112] ,

[0113] in, represents upsampling, represents matrix concatenation, Indicates the average pooling operation on the X dimension. Indicates the average pooling operation on the Y dimension. Indicates the average pooling operation on the Z dimension. Indicates the convolution calculation with the convolution kernel of 1, represents the normalization operation, Represents the activation function Operation, Indicates the extraction operation on the X dimension. Indicates the extraction operation on the Y dimension. Indicates the extraction operation on the Z dimension. Represents the Sigmoid activation function;

[0114] (4) Attention feature map The fourth convolution block input to the decoder obtains the final image extraction result .

[0115] Example 2

[0116] As shown in Table 1, the last row shows the experimental results obtained using the method of the present invention. The other three rows show the experimental results of existing methods, namely Unet (a convolutional neural network for image extraction), AttUNet (a deep learning method structure that combines the attention mechanism and Unet), and ResUNet (a deep learning structure that combines Unet and ResNet). The experimental results of these three methods are compared with the experimental results of the method of the present invention in terms of Dice (the value of the Dice indicator is usually between 0 and 1, and the higher the value, the better the extraction performance), IoU (used to measure the difference between the predicted value and the actual value), and Recall. According to the experimental data, the results obtained by the method of the present invention are superior to those of the other three methods. Therefore, the method of the present invention can achieve the effect of enhancing the image region of interest, and can also solve the problem of poor extraction effect caused by image size differences, thereby improving the performance of the model in processing three-dimensional data.

[0117] Table 1 Comparison results between the method of the present invention and the other three methods

[0118]

[0119] like Figure 3 As shown in FIG, the extraction results of the method of the present invention and the other three methods are compared. Figure 3 In the figure, two sets of images are compared. The red line represents the actual label of the manually annotated part, and the green line represents the extraction results using Unet, AttUNet, ResUNet and the method of the present invention. Figure 3 It can be seen that the extraction results obtained by using the method of the present invention for the image are more consistent with the actual annotations. Therefore, the method of the present invention can help the network better understand the long-distance dependencies in the image when performing extraction tasks and improve the accuracy of extraction.

[0120] Although the above describes the specific implementation methods of the invention in conjunction with the accompanying drawings, it does not limit the scope of protection of the invention. Based on the technical solution of the present invention, various modifications or variations that can be made by those skilled in the art without creative work are still within the scope of protection of the present invention.

Claims

1. An image extraction method based on a three-dimensional attention mechanism, characterized in that: The following steps are involved: S1. Select existing 3D medical image data as a dataset, and then divide the dataset into a training set and a test set in proportion; S2. Perform centroid cropping on the training set and test set, and then convert the cropped image data into a unified format to obtain the preprocessed training set and test set ; S3. Build and configure the network structure, input the preprocessed training set into the network structure for training, iterate the training process multiple times, continuously update the network structure parameters, save the optimal weights and biases obtained during training, and complete the training of the network structure; The network structure consists of four parts: encoder, decoder, three MFF multi-feature fusion modules and three 3DAEM three-dimensional attention enhancement mechanism modules. The encoder and decoder are composed of four convolution blocks. The four convolution blocks of the encoder are composed of two convolution layers with a convolution kernel size of 3, a step size of 1 and a padding of 1, an InstanceNorm layer, a ReLU layer and a maximum pooling layer with a downsampling coefficient of 2. The four convolution blocks of the decoder are composed of an upsampling block with a kernel size of 2 and two convolution layers with a convolution kernel size of 3, a step size of 2 and a padding of 1, an InstanceNorm layer and a ReLU activation function. After the network structure is constructed, the network structure is configured and the CosineAnnealLingLR learning rate adjustment strategy, the Adam optimizer and the back propagation algorithm are used to optimize the network structure. Then the preprocessed training set is trained. Input into the network structure for training until the final feature map result is output, and the weights of the network structure training process are saved and offset , and then iterate the training process multiple times to continuously update the weights of the network structure and offset , save the optimal weight and offset ; Among them, the preprocessed training set The image is input into the network structure for training. The training process is mainly divided into two parts: the encoder processing stage and the decoder processing stage; Encoder processing stage: (1) The preprocessed training set Any image in Input to the first convolution block of the encoder and output the feature map , and then the feature map Input into the first MFF module to obtain low-level feature maps ; (2) Low-level feature maps Input to the second convolution block of the encoder to obtain the feature map , and then the feature map and feature maps Input to the second MFF module, first the feature map Perform the maximum pooling operation with a kernel of 2 and then add it to the feature map Perform a convolution operation with a convolution kernel of 1 to obtain a low-level feature map , The calculation formula is: ， in, represents the maximum pooling operation with a kernel of 2, Represents a convolution operation with a convolution kernel of 1, Represents a splicing operation; (3) Low-level feature maps Input to the third convolution block of the encoder to get the feature map , and then the feature map , feature map and feature maps Input to the third MFF module, first the feature map Perform the maximum pooling operation with a kernel of 4, Perform the maximum pooling operation with a kernel of 2, and then combine the two with the feature map Perform a convolution operation with a convolution kernel of 1 to obtain a low-level feature map , The calculation formula is: ， in, represents the maximum pooling operation with a kernel of 2, Indicates the maximum pooling operation with a kernel of 4, Represents a convolution operation with a convolution kernel of 1, Represents a splicing operation; (4) Low-level feature maps Input to the fourth convolution block of the encoder to get the feature map ; S4. Input the saved optimal weights and biases, as well as the image data in the preprocessed test set, into the trained network structure, and output the final extraction results of the preprocessed test set.

2. The image extraction method based on the three-dimensional attention mechanism according to claim 1 is characterized in that: S1 is as follows: The data set is divided into training set in a ratio of 8:2 and test set , training set , test set ,in, represents the number of images in the training set, represents the number of images in the test set, , Indicates the first images, Indicates the first images.

3. The image extraction method based on the three-dimensional attention mechanism according to claim 2, characterized in that: S2 is as follows: Perform centroid cropping on the image data in the training set and test set, and then convert the cropped images into a unified format to obtain the preprocessed training set and test set , where the training set after preprocessing , the preprocessed test set , represents the first images, Represents the first images.

4. The image extraction method based on the three-dimensional attention mechanism according to claim 3 is characterized in that: Decoder processing stage: (1) Feature map Input to the first convolutional block of the decoder to obtain high-level feature maps , and then the high-level feature map and low-level feature maps Input to the first 3DAEM module, the high-level feature map Perform bilinear interpolation upsampling and then upsample the high-level feature map With low-level feature maps Splicing to get feature map ; Feature Map The three dimensions X, Y, and Z are averaged and pooled respectively. The average values ​​of the pooling kernels in the three dimensions are (X, 1, 1), (1, Y, 1), and (1, 1, Z), respectively. X, Y, and Z represent the values ​​of the height, width, and depth, respectively. The one-dimensional features obtained after the pooling operation are then subjected to the Concat splicing operation, the convolution operation with a kernel of 1, the BatchNorm normalization operation, the ReLU activation function, and the Split extraction operation. The one-dimensional features obtained after the Concat splicing operation, the convolution operation with a kernel of 1, the BatchNorm normalization operation, and the ReLU activation function are processed through the Split operation. Extract according to the three dimensions of height h, width w, and depth d. The shape of the extracted one-dimensional feature is the same as the shape of the one-dimensional feature obtained after the pooling operation. Then, the extracted one-dimensional feature is uniformly subjected to the convolution operation with a kernel of 1 and the Sigmoid activation function operation to obtain the attention coefficient of the three dimensions. 、 and , the attention coefficients of the three dimensions 、 、 and feature maps Multiply to get the attention feature map , the specific calculation process is: ， ， ， ， ， ， in, represents upsampling, represents matrix concatenation, Indicates the average pooling operation on the X dimension. Indicates the average pooling operation on the Y dimension. Indicates the average pooling operation on the Z dimension. Indicates the convolution calculation with the convolution kernel of 1, represents the normalization operation, Represents the activation function Operation, Indicates the extraction operation on the X dimension. Indicates the extraction operation on the Y dimension. Indicates the extraction operation on the Z dimension. Represents the Sigmoid activation function; (2) Attention feature map Input to the second convolutional block of the decoder to obtain high-level feature maps , and then the high-level feature map and low-level feature maps Input to the second 3DAEM module, the high-level feature map Perform bilinear interpolation upsampling and then upsample the high-level feature map With low-level feature maps Splicing to get feature map ; Feature Map The three dimensions X, Y, and Z are averaged and pooled respectively. The average values ​​of the pooling kernels in the three dimensions are (X, 1, 1), (1, Y, 1), and (1, 1, Z), respectively. X, Y, and Z represent the values ​​of the height, width, and depth dimensions, respectively. The one-dimensional features obtained after the pooling operation are then subjected to the Concat splicing operation, the convolution operation with a kernel of 1, BatchNorm normalization, the ReLU activation function, and the Split operation. The one-dimensional features obtained after the Concat splicing operation, the convolution operation with a kernel of 1, the BatchNorm normalization, and the ReLU activation function are processed through the Split operation. According to the three dimensions of height h, width w, and depth d, the shape of the extracted one-dimensional feature is the same as the shape of the one-dimensional feature obtained after the pooling operation. Then the Sigmoid activation function operation is uniformly performed on the extracted one-dimensional features to obtain the attention coefficient of the three dimensions 、 and , the attention coefficients of the three dimensions 、 、 and feature maps Multiply to get the attention feature map , the specific calculation process is: ， ， ， ， ， ， in, represents upsampling, represents matrix concatenation, Indicates the average pooling operation on the X dimension. Indicates the average pooling operation on the Y dimension. Indicates the average pooling operation on the Z dimension. Indicates the convolution calculation with the convolution kernel of 1, represents the normalization operation, Represents the activation function Operation, Indicates the extraction operation on the X dimension. Indicates the extraction operation on the Y dimension. Indicates the extraction operation on the Z dimension. Represents the Sigmoid activation function; (3) Attention feature map Input to the third convolution block of the decoder to obtain high-level feature maps , and then the high-level feature map and low-level feature maps Input to the third 3DAEM module, the high-level feature map Perform bilinear interpolation upsampling and then upsample the high-level feature map With low-level feature maps Splicing to get feature map ; Feature Map The three dimensions X, Y, and Z are averaged and pooled respectively. The average values ​​of the pooling kernels in the three dimensions are (X, 1, 1), (1, Y, 1), and (1, 1, Z), respectively. X, Y, and Z represent the values ​​of the height, width, and depth, respectively. The one-dimensional vector obtained after the pooling operation is then subjected to the Concat splicing operation, the convolution operation with a kernel of 1, BatchNorm normalization, the ReLU activation function, and the Split operation. The one-dimensional features obtained after the Concat splicing operation, the convolution operation with a kernel of 1, the BatchNorm normalization, and the ReLU activation function are extracted according to the three dimensions of height h, width w, and depth d through the Split operation. The shape of the extracted one-dimensional features is the same as the shape of the one-dimensional features obtained after the pooling operation. The Sigmoid activation function is then uniformly performed on the extracted one-dimensional features to obtain the attention coefficients of the three dimensions. 、 and , the attention coefficients of the three dimensions 、 、 and feature maps Multiply to get the attention feature map , the specific calculation process is: ， ， ， ， ， ， in, represents upsampling, represents matrix concatenation, Indicates the average pooling operation on the X dimension. Indicates the average pooling operation on the Y dimension. Indicates the average pooling operation on the Z dimension. Indicates the convolution calculation with the convolution kernel of 1, represents the normalization operation, Represents the activation function Operation, Indicates the extraction operation on the X dimension. Indicates the extraction operation on the Y dimension. Indicates the extraction operation on the Z dimension. Represents the Sigmoid activation function; (4) Attention feature map The fourth convolution block input to the decoder obtains the final image extraction result .

Images