Image extraction method and system based on three-dimensional attention mechanism

Abstract

The invention relates to the technical field of medical artificial intelligence, in particular to an image extraction method and system based on a three-dimensional attention mechanism, and the method comprises the following steps: creating a data set, and dividing the data set into a training set and a test set; cutting the training set and the test set, and converting the cut images into a unified format to obtain the preprocessed training set and test set; constructing a network structure, configuring the network structure, inputting the preprocessed training set into the network structure for training, iterating the training process for multiple times, continuously updating parameters of the network structure, storing the optimal weight and offset obtained in the training, and completing the training of the network structure; and inputting the stored optimal weight and offset and the image data in the preprocessed test set into the trained network structure, and outputting an extraction result of the final preprocessed test set. The method is mainly used for processing and extracting the image so as to improve the image quality and enhance the image information, and the long-distance dependency relationship in the image can be better understood.

Description

Technical Field

[0001] The present invention relates to the field of medical artificial intelligence technology, and in particular to an image extraction method and system based on a three-dimensional attention mechanism. Background Art

[0002] Technological development in the field of medical artificial intelligence has made great progress in the past few years and is expected to continue to develop rapidly in the future. The application of deep learning technology in the field of medical image analysis has achieved major breakthroughs. This includes automatic analysis and diagnosis of images such as X-rays, CT scans, MRIs, etc. Artificial intelligence algorithms can help doctors detect and diagnose various diseases such as tumors and lesions more accurately. Using machine learning and big data technology, patients can be provided with personalized diagnosis, treatment and prevention plans based on information such as the patient's genome, biomarkers and clinical data. Personalized medicine can improve treatment effectiveness, reduce adverse reactions and reduce medical costs. Artificial intelligence technology can assist doctors in making diagnostic and treatment decisions. By analyzing large amounts of clinical data and medical literature, artificial intelligence systems can provide advice and guidance for specific cases, helping doctors make more accurate decisions.

[0003] In recent years, the rapid development of computer-aided diagnosis has gradually replaced manual analysis of patient pathology images. Computer-aided diagnosis assists doctors in diagnosing diseases and provides additional information and analysis. Automation and computer-aided technology can speed up the diagnosis process and save time for doctors and patients. The advent of UNet has made a big step forward in artificial intelligence in the field of medical image extraction. Its high-precision extraction effect, applicability to images of different sizes, small amount of labeled data requirements, and fast training and reasoning speed reflect its wide value in clinical applications. In the field of three-dimensional medical data, many technologies and methods have been used to improve the accuracy and efficiency of extraction. CNN has been widely used in medical image extraction. By using convolution and pooling, CNN can automatically learn the features in the image. For three-dimensional images, 3DCNN is designed to process three-dimensional data directly, or convert three-dimensional data into 2D slices for processing. U-Net is a very common medical image extraction architecture, which has an encoder and decoder structure, and uses jump connections in the decoder to retain more detailed information. The self-attention mechanism can help the network better understand the long-distance dependencies in the image when performing extraction tasks and improve the accuracy of extraction.

[0004] Therefore, the present invention proposes an image extraction method and system based on a three-dimensional attention mechanism to solve the above problems. Summary of the invention

[0005] In view of the deficiencies in the prior art, the present invention develops an image extraction method and system based on a three-dimensional attention mechanism. The present invention can help the network better understand the long-distance dependencies in the image when performing extraction tasks and improve the accuracy of extraction.

[0006] The technical solution to the technical problem of the present invention is an image extraction method based on a three-dimensional attention mechanism, comprising the following steps: S1. Select existing three-dimensional medical image data as a dataset, and then divide the dataset into a training set and a test set in proportion; S2. Perform centroid cropping on the training set and the test set, and then convert the cropped image data into a unified format to obtain the preprocessed training set and the test set; S3. Build and configure the network structure, input the preprocessed training set into the network structure for training, iterate the training process multiple times, continuously update the network structure parameters, save the optimal weights and biases obtained in the training, and complete the training of the network structure; S4. Input the saved optimal weights and biases, as well as the image data in the preprocessed test set into the trained network structure, and output the final extraction result of the preprocessed test set.

[0007] S1 is as follows: The data set is divided into training sets in a ratio of 8:2. and test set , training set , test set ,in, represents the number of images in the training set, represents the number of images in the test set, , Indicates the training set images, Indicates the test set images.

[0008] S2 is as follows: Perform centroid cropping on the image data in the training set and test set, and then convert the cropped images into a unified format to obtain the preprocessed training set and test set , where the training set after preprocessing , the preprocessed test set , represents the first images, It represents the first images.

[0009] S3 is as follows: The network structure consists of four parts: an encoder, a decoder, three MFF multi-feature fusion modules and three 3DAEM three-dimensional attention enhancement mechanism modules. The encoder and decoder are composed of four convolution blocks. The four convolution blocks of the encoder are composed of two convolution layers with a convolution kernel size of 3, a step size of 1, and a padding of 1, an InstanceNorm layer, a ReLU layer and a maximum pooling layer with a downsampling coefficient of 2. The four convolution blocks of the decoder are composed of an upsampling block with a kernel size of 2 and two convolution layers with a convolution kernel size of 3, a step size of 2, and a padding of 1, an InstanceNorm layer and a ReLU activation function. After the network structure is constructed, the network structure is configured, and the CosineAnnealLingLR learning rate adjustment strategy, Adam optimizer and back propagation algorithm are used to optimize the network structure. Then the preprocessed training set is used. Input into the network structure for training until the final feature map result is output, and the weights of the network structure training process are saved and offset , and then iterate the training process multiple times to continuously update the weights of the network structure and offset , save the optimal weight and offset ; Among them, the preprocessed training set The image is input into the network structure for training. The training process is mainly divided into two parts: the encoder processing stage and the decoder processing stage.

[0010] Encoder processing stage: (1) The preprocessed training set Any image in Input to the first convolution block of the encoder and output the feature map , and then the feature map Input into the first MFF module to get the low-level feature map ; (2) Low-level feature maps Input to the second convolution block of the encoder to get the feature map , and then the feature map and feature map Input to the second MFF module, first the feature map Perform a maximum pooling operation with a kernel of 2 and then add it to the feature map Perform a convolution operation with a convolution kernel of 1 to obtain a low-level feature map , The calculation formula is: , in, represents the maximum pooling operation with a kernel of 2, represents a convolution operation with a convolution kernel of 1. Represents a splicing operation; (3) Low-level feature maps Input to the third convolution block of the encoder to get the feature map , and then the feature map , feature map and feature map Input to the third MFF module, first the feature map Perform the maximum pooling operation with a kernel of 4. Perform the maximum pooling operation with a kernel of 2, and then combine the two with the feature map Perform a convolution operation with a convolution kernel of 1 to obtain a low-level feature map , The calculation formula is: , in, represents the maximum pooling operation with a kernel of 2, represents the maximum value pooling operation with a kernel of 4. represents a convolution operation with a convolution kernel of 1. Represents a splicing operation; (4) Low-level feature maps Input to the fourth convolution block of the encoder to get the feature map .

[0011] Decoder processing stage: (1) Feature map Input to the first convolutional block of the decoder to obtain a high-level feature map , and then the high-level feature map and low-level feature maps Input to the first 3DAEM module, the high-level feature map Upsample by bilinear interpolation, and then upsample the high-level feature map With low-level feature maps Concatenate to get the feature map ; For feature maps The three dimensions X, Y, and Z are averaged and pooled respectively. The average values ​​of the pooling kernels in the three dimensions are (X, 1, 1), (1, Y, 1), and (1, 1, Z), respectively. X, Y, and Z represent the values ​​of the height, width, and depth, respectively. The one-dimensional features obtained after the pooling operation are then subjected to the Concat splicing operation, the convolution operation with a kernel of 1, the BatchNorm normalization operation, the ReLU activation function, and the Split extraction operation. The one-dimensional features obtained after the Concat splicing operation, the convolution operation with a kernel of 1, the BatchNorm normalization operation, and the ReLU activation function are processed through the Split operation. The extraction is performed according to the three dimensions of height h, width w, and depth d. The shape of the extracted one-dimensional feature is the same as the shape of the one-dimensional feature obtained after the pooling operation. Then the extracted one-dimensional feature is uniformly subjected to the convolution operation with a kernel of 1 and the Sigmoid activation function operation to obtain the attention coefficient of the three dimensions. , and , the attention coefficients of the three dimensions , , and feature map Multiply to get the attention feature map , the specific calculation process is: , , , , , , in, represents upsampling, represents matrix concatenation, Indicates the average pooling operation on the X dimension. Indicates the average value pooling operation on the Y dimension. Indicates the average pooling operation on the Z dimension. Indicates the convolution calculation with a convolution kernel of 1. represents the normalization operation, Represents the activation function Operation, Indicates the extraction operation on the X dimension. Indicates the extraction operation on the Y dimension. Indicates the extraction operation on the Z dimension. Represents the Sigmoid activation function; (2) Attention feature map Input to the second convolutional block of the decoder to obtain a high-level feature map , and then the high-level feature map and low-level feature maps Input to the second 3DAEM module, the high-level feature map Upsample by bilinear interpolation, and then upsample the high-level feature map With low-level feature maps Concatenate to get the feature map ; For feature maps The three dimensions X, Y, and Z are averaged and pooled respectively. The average values ​​of the pooling kernels in the three dimensions are (X, 1, 1), (1, Y, 1), and (1, 1, Z), respectively. X, Y, and Z represent the values ​​of the height, width, and depth, respectively. The one-dimensional features obtained after the pooling operation are then subjected to the Concat operation, the convolution operation with a kernel of 1, the BatchNorm normalization, the ReLU activation function, and the Split operation. The one-dimensional features obtained after the Concat operation, the convolution operation with a kernel of 1, the BatchNorm normalization, and the ReLU activation function are processed by the Split operation. According to the three dimensions of height h, width w, and depth d, the shape of the extracted one-dimensional feature is the same as the shape of the one-dimensional feature obtained after the pooling operation. Then the Sigmoid activation function is uniformly performed on the extracted one-dimensional features to obtain the attention coefficients of the three dimensions. , and , the attention coefficients of the three dimensions , , and feature map Multiply to get the attention feature map , the specific calculation process is: , , , , , , in, represents upsampling, represents matrix concatenation, Indicates the average pooling operation on the X dimension. Indicates the average value pooling operation on the Y dimension. Indicates the average pooling operation on the Z dimension. Indicates the convolution calculation with a convolution kernel of 1. represents the normalization operation, Represents the activation function Operation, Indicates the extraction operation on the X dimension. Indicates the extraction operation on the Y dimension. Indicates the extraction operation on the Z dimension. Represents the Sigmoid activation function; (3) Attention feature map Input to the third convolutional block of the decoder to obtain a high-level feature map , and then the high-level feature map and low-level feature maps Input to the third 3DAEM module, the high-level feature map Upsample by bilinear interpolation, and then upsample the high-level feature map With low-level feature maps Concatenate to get the feature map ; For feature maps The three dimensions X, Y, and Z are averaged and pooled respectively. The average values ​​of the pooling kernels in the three dimensions are (X, 1, 1), (1, Y, 1), and (1, 1, Z), respectively. X, Y, and Z represent the values ​​of the height, width, and depth, respectively. The one-dimensional vector obtained after the pooling operation is then subjected to the Concat splicing operation, the convolution operation with a kernel of 1, the BatchNorm normalization, the ReLU activation function, and the Split operation in sequence. The one-dimensional features obtained after the Concat splicing operation, the convolution operation with a kernel of 1, the BatchNorm normalization, and the ReLU activation function are extracted according to the three dimensions of height h, width w, and depth d through the Split operation. The shape of the extracted one-dimensional features is the same as the shape of the one-dimensional features obtained after the pooling operation. The Sigmoid activation function is then uniformly performed on the extracted one-dimensional features to obtain the attention coefficients of the three dimensions. , and , the attention coefficients of the three dimensions , , and feature map Multiply to get the attention feature map , the specific calculation process is: , , , , , , in, represents upsampling, represents matrix concatenation, Indicates the average pooling operation on the X dimension. Indicates the average value pooling operation on the Y dimension. Indicates the average pooling operation on the Z dimension. Indicates the convolution calculation with a convolution kernel of 1. represents the normalization operation, Represents the activation function Operation, Indicates the extraction operation on the X dimension. Indicates the extraction operation on the Y dimension. Indicates the extraction operation on the Z dimension. Represents the Sigmoid activation function; (4) Attention feature map Input to the fourth convolution block of the decoder to obtain the final image extraction result .

[0012] The present invention also provides an image extraction system based on a three-dimensional attention mechanism, which executes an image extraction method based on a three-dimensional attention mechanism.

[0013] The effects provided in the content of the invention are only the effects of the embodiments, rather than all the effects of the invention. The above technical solution has the following advantages or beneficial effects: The present invention applies the attention mechanism to the dimensionality of the image, and achieves the effect of enhancing the image's area of ​​interest by calculating the attention coefficient of each dimension. Secondly, the present invention fully integrates the feature map generated by the encoder end, solves the problem of poor extraction effect caused by image size difference, and at the same time, the dimensional attention mechanism also captures information on the three-dimensional spatial structure, improving the performance of the model in processing three-dimensional data; in addition, at the jump connection between the encoder and the decoder, the present invention uses the feature map generated by each encoder as a low-level feature branch to input into the 3DAEM module, and at the same time, the high-level features generated by the lower-level decoder are also input into the 3DAEM module as another branch, which enhances the model's attention to the relevant parts of the features generated by the encoder end, thereby improving the application of the encoder features in the decoder stage. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0015] Figure 1 It is a schematic diagram of the network structure in the present invention.

[0016] Figure 2 Schematic diagram of the 3DAEM structure in the present invention.

[0017] Figure 3 The figure shows the comparison of the image extraction results between the method in the present invention and the existing method. DETAILED DESCRIPTION

[0018] In order to clearly illustrate the technical features of the present invention, the present invention is described in detail below through specific implementation methods and in conjunction with the accompanying drawings. The following disclosure provides many different embodiments or examples for implementing different structures of the present invention. In order to simplify the disclosure of the present invention, the components and settings of specific examples are described below.

[0019] Example 1 An image extraction method based on a three-dimensional attention mechanism comprises the following steps: S1. Select existing three-dimensional medical image data as a dataset, and then divide the dataset into a training set and a test set in proportion; S2. Perform centroid cropping on the training set and the test set, and then convert the cropped image data into a unified format to obtain the preprocessed training set and the test set; S3. Build and configure the network structure, input the preprocessed training set into the network structure for training, iterate the training process multiple times, continuously update the network structure parameters, save the optimal weights and biases obtained in the training, and complete the training of the network structure; S4. Input the saved optimal weights and biases, as well as the image data in the preprocessed test set into the trained network structure, and output the final extraction result of the preprocessed test set.

[0020] S1 is as follows: The data set is divided into training sets in a ratio of 8:2. and test set , training set , test set ,in, represents the number of images in the training set, represents the number of images in the test set, , Indicates the training set images, Indicates the test set images.

[0021] S2 is as follows: Perform centroid cropping on the image data in the training set and test set, and then convert the cropped images into a unified format to obtain the preprocessed training set and test set , where the training set after preprocessing , the preprocessed test set , represents the first images, Represents the first images.

[0022] S3 is as follows: The network structure consists of four parts: an encoder, a decoder, three MFF multi-feature fusion modules and three 3DAEM three-dimensional attention enhancement mechanism modules. The encoder and decoder are composed of four convolution blocks. The four convolution blocks of the encoder are composed of two convolution layers with a convolution kernel size of 3, a step size of 1, and a padding of 1, an InstanceNorm layer, a ReLU layer and a maximum pooling layer with a downsampling coefficient of 2. The four convolution blocks of the decoder are composed of an upsampling block with a kernel size of 2 and two convolution layers with a convolution kernel size of 3, a step size of 2, and a padding of 1, an InstanceNorm layer and a ReLU activation function. After the network structure is constructed, the network structure is configured, and the CosineAnnealLingLR learning rate adjustment strategy, Adam optimizer and back propagation algorithm are used to optimize the network structure. Then the preprocessed training set is used. Input into the network structure for training until the final feature map result is output, and the weights of the network structure training process are saved and offset , and then iterate the training process multiple times to continuously update the weights of the network structure and offset , save the optimal weight and offset ; Among them, the preprocessed training set The image is input into the network structure for training. The training process is mainly divided into two parts: the encoder processing stage and the decoder processing stage.

[0023] Encoder processing stage: (1) The preprocessed training set Any image in Input to the first convolution block of the encoder and output the feature map , and then the feature map Input into the first MFF module to get the low-level feature map ; (2) Low-level feature maps Input to the second convolution block of the encoder to get the feature map , and then the feature map and feature map Input to the second MFF module, first the feature map Perform a maximum pooling operation with a kernel of 2 and then add it to the feature map Perform a convolution operation with a convolution kernel of 1 to obtain a low-level feature map , The calculation formula is: , in, represents the maximum pooling operation with a kernel of 2, represents a convolution operation with a convolution kernel of 1. Represents a splicing operation; (3) Low-level feature maps Input to the third convolution block of the encoder to get the feature map , and then the feature map , feature map and feature map Input to the third MFF module, first the feature map Perform the maximum pooling operation with a kernel of 4. Perform the maximum pooling operation with a kernel of 2, and then combine the two with the feature map Perform a convolution operation with a convolution kernel of 1 to obtain a low-level feature map , The calculation formula is: , in, represents the maximum pooling operation with a kernel of 2, represents the maximum value pooling operation with a kernel of 4. represents a convolution operation with a convolution kernel of 1. Represents a splicing operation; (4) Low-level feature maps Input to the fourth convolution block of the encoder to get the feature map .

[0024] Decoder processing stage: (1) Feature map Input to the first convolutional block of the decoder to obtain a high-level feature map , and then the high-level feature map and low-level feature maps Input to the first 3DAEM module, the high-level feature map Upsample by bilinear interpolation, and then upsample the high-level feature map With low-level feature maps Concatenate to get the feature map ; For feature maps The three dimensions X, Y, and Z are averaged and pooled respectively. The average values ​​of the pooling kernels in the three dimensions are (X, 1, 1), (1, Y, 1), and (1, 1, Z), respectively. X, Y, and Z represent the values ​​of the height, width, and depth, respectively. The one-dimensional features obtained after the pooling operation are then subjected to the Concat splicing operation, the convolution operation with a kernel of 1, the BatchNorm normalization operation, the ReLU activation function, and the Split extraction operation. The one-dimensional features obtained after the Concat splicing operation, the convolution operation with a kernel of 1, the BatchNorm normalization operation, and the ReLU activation function are processed through the Split operation. The extraction is performed according to the three dimensions of height h, width w, and depth d. The shape of the extracted one-dimensional feature is the same as the shape of the one-dimensional feature obtained after the pooling operation. Then the extracted one-dimensional feature is uniformly subjected to the convolution operation with a kernel of 1 and the Sigmoid activation function operation to obtain the attention coefficient of the three dimensions. , and , the attention coefficients of the three dimensions , , and feature map Multiply to get the attention feature map , the specific calculation process is: , , , , , , in, represents upsampling, represents matrix concatenation, Indicates the average pooling operation on the X dimension. Indicates the average value pooling operation on the Y dimension. Indicates the average pooling operation on the Z dimension. Indicates the convolution calculation with a convolution kernel of 1. represents the normalization operation, Represents the activation function Operation, Indicates the extraction operation on the X dimension. Indicates the extraction operation on the Y dimension. Indicates the extraction operation on the Z dimension. Represents the Sigmoid activation function; (2) Attention feature map Input to the second convolutional block of the decoder to obtain a high-level feature map , and then the high-level feature map and low-level feature maps Input to the second 3DAEM module, the high-level feature map Upsample by bilinear interpolation, and then upsample the high-level feature map With low-level feature maps Concatenate to get the feature map ; For feature maps The three dimensions X, Y, and Z are averaged and pooled respectively. The average values ​​of the pooling kernels in the three dimensions are (X, 1, 1), (1, Y, 1), and (1, 1, Z), respectively. X, Y, and Z represent the values ​​of the height, width, and depth, respectively. The one-dimensional features obtained after the pooling operation are then subjected to the Concat operation, the convolution operation with a kernel of 1, the BatchNorm normalization, the ReLU activation function, and the Split operation. The one-dimensional features obtained after the Concat operation, the convolution operation with a kernel of 1, the BatchNorm normalization, and the ReLU activation function are processed by the Split operation. According to the three dimensions of height h, width w, and depth d, the shape of the extracted one-dimensional feature is the same as the shape of the one-dimensional feature obtained after the pooling operation. Then the Sigmoid activation function is uniformly performed on the extracted one-dimensional features to obtain the attention coefficients of the three dimensions. , and , the attention coefficients of the three dimensions , , and feature map Multiply to get the attention feature map , the specific calculation process is: , , , , , , in, represents upsampling, represents matrix concatenation, Indicates the average pooling operation on the X dimension. Indicates the average value pooling operation on the Y dimension. Indicates the average pooling operation on the Z dimension. Indicates the convolution calculation with a convolution kernel of 1. represents the normalization operation, Represents the activation function Operation, Indicates the extraction operation on the X dimension. Indicates the extraction operation on the Y dimension. Indicates the extraction operation on the Z dimension. Represents the Sigmoid activation function; (3) Attention feature map Input to the third convolutional block of the decoder to obtain a high-level feature map , and then the high-level feature map and low-level feature maps Input to the third 3DAEM module, the high-level feature map Upsample by bilinear interpolation, and then upsample the high-level feature map With low-level feature maps Concatenate to get the feature map ; For feature maps The three dimensions X, Y, and Z are averaged and pooled respectively. The average values ​​of the pooling kernels in the three dimensions are (X, 1, 1), (1, Y, 1), and (1, 1, Z), respectively. X, Y, and Z represent the values ​​of the height, width, and depth, respectively. The one-dimensional vector obtained after the pooling operation is then subjected to the Concat splicing operation, the convolution operation with a kernel of 1, the BatchNorm normalization, the ReLU activation function, and the Split operation in sequence. The one-dimensional features obtained after the Concat splicing operation, the convolution operation with a kernel of 1, the BatchNorm normalization, and the ReLU activation function are extracted according to the three dimensions of height h, width w, and depth d through the Split operation. The shape of the extracted one-dimensional features is the same as the shape of the one-dimensional features obtained after the pooling operation. The Sigmoid activation function is then uniformly performed on the extracted one-dimensional features to obtain the attention coefficients of the three dimensions. , and , the attention coefficients of the three dimensions , , and feature map Multiply to get the attention feature map , the specific calculation process is: , , , , , , in, represents upsampling, represents matrix concatenation, Indicates the average pooling operation on the X dimension. Indicates the average value pooling operation on the Y dimension. Indicates the average pooling operation on the Z dimension. Indicates the convolution calculation with a convolution kernel of 1. represents the normalization operation, Represents the activation function Operation, Indicates the extraction operation on the X dimension. Indicates the extraction operation on the Y dimension. Indicates the extraction operation on the Z dimension. Represents the Sigmoid activation function; (4) Attention feature map Input to the fourth convolution block of the decoder to obtain the final image extraction result .

[0025] Example 2 A three-dimensional attention mechanism-based image extraction system implements a three-dimensional attention mechanism-based image extraction method.

[0026] Example 3 As shown in Table 1, the last line is the experimental results obtained by the method of the present invention, and the other three lines are existing methods, namely Unet (convolutional neural network for image extraction), AttUNet (deep learning method structure combining attention mechanism and Unet) and ResUNet (deep learning structure combining Unet and ResNet). The experimental results of these three methods are compared with the experimental results of the method of the present invention in terms of Dice (the value of the Dice index is usually between 0 and 1, and the higher the value, the better the performance of the extraction result), IOU (used to measure the degree of difference between the predicted value and the actual value) and Recall. According to the experimental data, it can be seen that the results obtained by the method of the present invention are all due to the other three methods. Therefore, the method of the present invention can achieve the effect of enhancing the image region of interest, and can also solve the problem of poor extraction effect caused by image size difference, thereby improving the performance of the model in processing three-dimensional data.

[0027] Table 1 Comparison results between the method of the present invention and the other three methods like Figure 3 As shown, the extraction results of the method of the present invention and the other three methods are compared. Figure 3 In the figure, two groups of images are given for comparison. The red line represents the actual label at the manually annotated location, and the green line represents the extraction results using Unet, AttUNet, ResUNet and the method of the present invention. Figure 3 It can be seen that the extraction results obtained by using the method of the present invention for the image are more consistent with the actual annotation. Therefore, the method of the present invention can help the network better understand the long-distance dependencies in the image when performing extraction tasks and improve the accuracy of extraction.

[0028] Although the above describes the specific implementation mode of the invention in conjunction with the drawings, it is not intended to limit the scope of protection of the invention. Based on the technical solution of the present invention, various modifications or variations that can be made by those skilled in the art without creative work are still within the scope of protection of the present invention.

Claims

1. An image extraction method based on a three-dimensional attention mechanism, characterized in that: The following steps are involved: S1. Select existing three-dimensional medical image data as a dataset, and then divide the dataset into a training set and a test set in proportion; S2. Perform centroid cropping on the training set and the test set, and then convert the cropped image data into a unified format to obtain the preprocessed training set and the test set; S3. Build and configure the network structure, input the preprocessed training set into the network structure for training, iterate the training process multiple times, continuously update the network structure parameters, save the optimal weights and biases obtained in the training, and complete the training of the network structure; S4. Input the saved optimal weights and biases, as well as the image data in the preprocessed test set into the trained network structure, and output the final extraction result of the preprocessed test set.

2. The image extraction method based on the three-dimensional attention mechanism according to claim 1 is characterized in that: S1 is as follows: The data set is divided into training sets in a ratio of 8:

2. and test set , training set , test set ,in, represents the number of images in the training set, represents the number of images in the test set, , Indicates the training set images, Indicates the test set images.

3. The image extraction method based on the three-dimensional attention mechanism according to claim 2 is characterized in that: S2 is as follows: Perform centroid cropping on the image data in the training set and test set, and then convert the cropped images into a unified format to obtain the preprocessed training set and test set , where the training set after preprocessing , the preprocessed test set , represents the first images, It represents the first images.

4. The image extraction method based on the three-dimensional attention mechanism according to claim 3, characterized in that S3 The details are as follows: The network structure consists of four parts: an encoder, a decoder, three MFF multi-feature fusion modules and three 3DAEM three-dimensional attention enhancement mechanism modules. The encoder and decoder are composed of four convolution blocks. The four convolution blocks of the encoder are composed of two convolution layers with a convolution kernel size of 3, a step size of 1, and a padding of 1, an InstanceNorm layer, a ReLU layer and a maximum pooling layer with a downsampling coefficient of 2. The four convolution blocks of the decoder are composed of an upsampling block with a kernel size of 2 and two convolution layers with a convolution kernel size of 3, a step size of 2, and a padding of 1, an InstanceNorm layer and a ReLU activation function. After the network structure is constructed, the network structure is configured, and the CosineAnnealLingLR learning rate adjustment strategy, Adam optimizer and back propagation algorithm are used to optimize the network structure. Then the preprocessed training set is used. Input into the network structure for training until the final feature map result is output, and the weights of the network structure training process are saved and offset , and then iterate the training process multiple times to continuously update the weights of the network structure and offset , save the optimal weight and offset ; Among them, the preprocessed training set The image is input into the network structure for training. The training process is mainly divided into two parts: the encoder processing stage and the decoder processing stage.

5. The image extraction method based on the three-dimensional attention mechanism according to claim 4 is characterized in that the encoder Processing stage: (1) The preprocessed training set Any image in Input to the first convolution block of the encoder and output the feature map , and then the feature map Input into the first MFF module to get the low-level feature map ; (2) Low-level feature maps Input to the second convolution block of the encoder to get the feature map , and then the feature map and feature map Input to the second MFF module, first the feature map Perform a maximum pooling operation with a kernel of 2 and then add it to the feature map Perform a convolution operation with a convolution kernel of 1 to obtain a low-level feature map , The calculation formula is: ， in, represents the maximum pooling operation with a kernel of 2, represents a convolution operation with a convolution kernel of 1. Represents a splicing operation; (3) Low-level feature maps Input to the third convolution block of the encoder to get the feature map , and then the feature map , feature map and feature map Input to the third MFF module, first the feature map Perform the maximum pooling operation with a kernel of 4. Perform the maximum pooling operation with a kernel of 2, and then combine the two with the feature map Perform a convolution operation with a convolution kernel of 1 to obtain a low-level feature map , The calculation formula is: ， in, represents the maximum pooling operation with a kernel of 2, represents the maximum value pooling operation with a kernel of 4. represents a convolution operation with a convolution kernel of 1. Represents a splicing operation; (4) Low-level feature maps Input to the fourth convolution block of the encoder to get the feature map .

6. The image extraction method based on the three-dimensional attention mechanism according to claim 5 is characterized in that: Decoder processing stage: (1) Feature map Input to the first convolutional block of the decoder to obtain a high-level feature map , and then the high-level feature map and low-level feature maps Input to the first 3DAEM module, the high-level feature map Upsample by bilinear interpolation, and then upsample the high-level feature map With low-level feature maps Concatenate to get the feature map ; For feature maps The three dimensions X, Y, and Z are averaged and pooled respectively. The average values ​​of the pooling kernels in the three dimensions are (X, 1, 1), (1, Y, 1), and (1, 1, Z), respectively. X, Y, and Z represent the values ​​of the height, width, and depth, respectively. The one-dimensional features obtained after the pooling operation are then subjected to the Concat splicing operation, the convolution operation with a kernel of 1, the BatchNorm normalization operation, the ReLU activation function, and the Split extraction operation. The one-dimensional features obtained after the Concat splicing operation, the convolution operation with a kernel of 1, the BatchNorm normalization operation, and the ReLU activation function are processed through the Split operation. The extraction is performed according to the three dimensions of height h, width w, and depth d. The shape of the extracted one-dimensional feature is the same as the shape of the one-dimensional feature obtained after the pooling operation. Then the extracted one-dimensional feature is uniformly subjected to the convolution operation with a kernel of 1 and the Sigmoid activation function operation to obtain the attention coefficient of the three dimensions. , and , the attention coefficients of the three dimensions , , and feature map Multiply to get the attention feature map , the specific calculation process is: ， ， ， ， ， ， in, represents upsampling, represents matrix concatenation, Indicates the average pooling operation on the X dimension. Indicates the average value pooling operation on the Y dimension. Indicates the average pooling operation on the Z dimension. Indicates the convolution calculation with a convolution kernel of 1. represents the normalization operation, Represents the activation function Operation, Indicates the extraction operation on the X dimension. Indicates the extraction operation on the Y dimension. Indicates the extraction operation on the Z dimension. Represents the Sigmoid activation function; (2) Attention feature map Input to the second convolutional block of the decoder to obtain a high-level feature map , and then the high-level feature map and low-level feature maps Input to the second 3DAEM module, the high-level feature map Upsample by bilinear interpolation, and then upsample the high-level feature map With low-level feature maps Concatenate to get the feature map ; For feature maps The three dimensions X, Y, and Z are averaged and pooled respectively. The average values ​​of the pooling kernels in the three dimensions are (X, 1, 1), (1, Y, 1), and (1, 1, Z), respectively. X, Y, and Z represent the values ​​of the height, width, and depth, respectively. The one-dimensional features obtained after the pooling operation are then subjected to the Concat operation, the convolution operation with a kernel of 1, the BatchNorm normalization, the ReLU activation function, and the Split operation. The one-dimensional features obtained after the Concat operation, the convolution operation with a kernel of 1, the BatchNorm normalization, and the ReLU activation function are processed by the Split operation. According to the three dimensions of height h, width w, and depth d, the shape of the extracted one-dimensional feature is the same as the shape of the one-dimensional feature obtained after the pooling operation. Then the Sigmoid activation function is uniformly performed on the extracted one-dimensional features to obtain the attention coefficients of the three dimensions. , and , the attention coefficients of the three dimensions 、 , and feature map Multiply to get the attention feature map , the specific calculation process is: ， ， ， ， ， ， in, represents upsampling, represents matrix concatenation, Indicates the average pooling operation on the X dimension. Indicates the average value pooling operation on the Y dimension. Indicates the average pooling operation on the Z dimension. Indicates the convolution calculation with a convolution kernel of 1. represents the normalization operation, Represents the activation function Operation, Indicates the extraction operation on the X dimension. Indicates the extraction operation on the Y dimension. Indicates the extraction operation on the Z dimension. Represents the Sigmoid activation function; (3) Attention feature map Input to the third convolutional block of the decoder to obtain a high-level feature map , and then the high-level feature map and low-level feature maps Input to the third 3DAEM module, the high-level feature map Upsample by bilinear interpolation, and then upsample the high-level feature map With low-level feature maps Concatenate to get the feature map ; For feature maps The three dimensions X, Y, and Z are averaged and pooled respectively. The average values ​​of the pooling kernels in the three dimensions are (X, 1, 1), (1, Y, 1), and (1, 1, Z), respectively. X, Y, and Z represent the values ​​of the height, width, and depth, respectively. The one-dimensional vector obtained after the pooling operation is then subjected to the Concat splicing operation, the convolution operation with a kernel of 1, the BatchNorm normalization, the ReLU activation function, and the Split operation in sequence. The one-dimensional features obtained after the Concat splicing operation, the convolution operation with a kernel of 1, the BatchNorm normalization, and the ReLU activation function are extracted according to the three dimensions of height h, width w, and depth d through the Split operation. The shape of the extracted one-dimensional features is the same as the shape of the one-dimensional features obtained after the pooling operation. The Sigmoid activation function is then uniformly performed on the extracted one-dimensional features to obtain the attention coefficients of the three dimensions. , and , the attention coefficients of the three dimensions , , and feature map Multiply to get the attention feature map , the specific calculation process is: ， ， ， ， ， ， in, represents upsampling, represents matrix concatenation, Indicates the average pooling operation on the X dimension. Indicates the average value pooling operation on the Y dimension. Indicates the average pooling operation on the Z dimension. Indicates the convolution calculation with a convolution kernel of 1. represents the normalization operation, Represents the activation function Operation, Indicates the extraction operation on the X dimension. Indicates the extraction operation on the Y dimension. Indicates the extraction operation on the Z dimension. Represents the Sigmoid activation function; (4) Attention feature map Input to the fourth convolution block of the decoder to obtain the final image extraction result .

7. An image extraction system based on a three-dimensional attention mechanism, characterized in that: Execute an image extraction method based on a three-dimensional attention mechanism as described in any one of claims 1-6.

Images