Terahertz medical image segmentation method based on Coordinate Attention module and U-Net + +

Abstract

The invention belongs to the technical field of server heat dissipation, and particularly relates to a terahertz medical image segmentation method based on a Coordinate Attention module and U-Net + +, and the method comprises the steps: obtaining original medical image data, and carrying out the data preprocessing of the original medical image data; the method comprises the following steps: creating a U-Net + + neural network model, and adding a Coordinate Attention module behind each convolution block of the U-Net + + neural network model; feature extraction is carried out on the original medical image data after data preprocessing through a U-Net + + neural network model added with a Coordinate Attention module, and output features are obtained; and S3, matching the determined organ labels with the output features in the step S3 to obtain a medical image segmentation result. According to the method, a semantic gap of a jump connection part is reduced by using a nested structure of U-Net + +, a Coordinate Attention module is added on this basis, and spatial information is embedded into channel attention so as to obtain information of a larger area, so that a model can identify a target area more accurately, introduction of large calculation overhead is avoided, and meanwhile, a better segmentation result can be obtained.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical image segmentation, and in particular to a terahertz medical image segmentation method based on a Coordinate Attention module and U-Net++. Background Art

[0002] Terahertz waves refer to electromagnetic waves with a frequency of 0.1THz to 10THz and a wavelength of 30μm to 3000μm. The photon energy in the terahertz band is low, and compared with X-rays, it does not have ionizing radiation, and will not cause obvious ionization damage to biological tissues; compared with ultrasonic imaging that requires the use of coupling agents to contact samples, terahertz imaging is a non-contact non-destructive test; and the frequency band of terahertz waves is at the energy level of vibration and rotation of many biological macromolecules, making it easy to identify substances with "fingerprint" characteristics.

[0003] There are many network structures that perform well in the field of medical image segmentation, among which U-Net is a widely used neural network. It is based on a fully convolutional neural network, has a simple structure, can perform more accurate segmentation with fewer samples, and uses skip connections to retrieve lost information. However, the feature semantic gap between its encoder and decoder is large, and the skip connection part of U-Net directly splices them, which may lead to a decrease in segmentation accuracy.

[0004] At present, some studies have applied U-Net to terahertz image segmentation, such as in terahertz security inspection and terahertz coaxial digital holography, but U-Net or its variants have not been applied to terahertz medical image segmentation. For the classic U-Net structure, the edge segmentation in terahertz image segmentation is inaccurate.

[0005] The U-Net network proposed for the cell segmentation task has a simple structure and combines shallow feature information and deep feature information through skip connections. It performs well in small sample image segmentation tasks. However, the large semantic gap between the decoding and encoding parts may lead to inaccurate segmentation results.

[0006] Therefore, how to improve the existing medical image segmentation methods to enhance the model's expressive power and context-awareness, while enhancing relevant features and suppressing useless features, is a technical problem that urgently needs to be solved. Summary of the invention

[0007] The purpose of the present invention is to provide a terahertz medical image segmentation method based on the Coordinate Attention module and U-Net++, so as to improve the medical image segmentation method in the prior art, so as to enhance the expression ability and context perception ability of the model, and at the same time enhance relevant features and suppress useless features.

[0008] In order to solve the above technical problems, the technical solution adopted by the present invention is as follows:

[0009] The terahertz medical image segmentation method based on the Coordinate Attention module and U-Net++ includes the following steps:

[0010] S1: Acquire original medical image data, and perform data preprocessing on the original medical image data;

[0011] S2: Create a U-Net++ neural network model, and add a Coordinate Attention module after each convolution block of the U-Net++ neural network model;

[0012] S3: The U-Net++ neural network model with the Coordinate Attention module is used to extract features from the original medical image data after data preprocessing, and output features are obtained;

[0013] S4: Matching the determined organ labels with the output features in step S3 to obtain a medical image segmentation result, wherein the determined organ labels are manually annotated labels obtained through supervised learning.

[0014] Preferably, the specific process of data preprocessing in step S1 is as follows:

[0015] S11: read the original medical image data, and split the image into three-channel feature maps according to the RGB value of each pixel;

[0016] S12: Convert the three-channel feature map into a grayscale image and perform normalization processing.

[0017] Preferably, the three-channel feature map in step S11 is three matrices, and the matrices are specifically as follows:

[0018]

[0019] Among them, x ij is the pixel value of each pixel, i, j are the pixel position numbers, i∈[1,m], j∈[1,n].

[0020] Preferably, the U-Net++ neural network model in step S2 includes multiple convolution blocks, a Coordinate Attention module is set after each convolution block, each of the convolution blocks includes 2 3×3 convolution kernels, and the U-Net++ neural network model has four layers, the input features of the latter layer are obtained by downsampling the first output variable of the previous layer, and the input features in the middle of each layer are obtained by upsampling the intermediate feature variables of the next layer and skipping the previous intermediate features of the layer.

[0021] Preferably, the specific process of feature extraction in step S3 is as follows:

[0022] S31: The feature map obtained in step S11 is used as the input layer of the U-Net++ neural network model with the Coordinate Attention module added, and a convolution operation is performed to obtain the intermediate feature map X 00 ;

[0023] S32: For the intermediate feature map X 00 Using the intermediate feature map X 00 Downsampling is performed to obtain the input of the next layer, and the input is subjected to the operation in step S31 to obtain the intermediate feature map X 10 ;

[0024] S33: For the intermediate feature map X 10 Upsample and combine with the intermediate feature map X 00 Perform skip connection fusion to obtain a fused feature map, and process the fused feature map with the convolution layer and Coordinate Attention module in step S31 to obtain X 01 ;

[0025] S34: Repeat steps S31-S33 to perform four-layer calculations to obtain a feature map X that combines shallow features and deep features. 04 .

[0026] Preferably, the specific process of step S31 is as follows:

[0027] S311: Select a convolution layer including 64 3×3 convolution kernels to perform a convolution operation on the input layer. The convolution layer is accompanied by a specified ReLU activation function layer and a BN layer. The activation function formula is as follows:

[0028] R(x) = max{0, x};

[0029] S312: Repeat the convolution operation of the convolution layer including 64 3×3 convolution kernels on the input layer and then output;

[0030] S313: Obtain a first set of intermediate feature maps through the Coordinate Attention module.

[0031] Preferably, the specific process of processing by the Coordinate Attention module in step S313 is as follows:

[0032] S3131: Coordinate information embedding: Input tensor X = (X 1 , ..., X C )∈R C×H×W Each channel is encoded along the horizontal and vertical directions respectively, and the tensors of size C×H×1 and C×1×W are obtained respectively. The calculation formula of the value of the h-th row of Zhang Liang obtained by encoding the C-th channel along the horizontal direction is as follows:

[0033]

[0034] The value of the wth column of the tensor encoded along the vertical direction of the Cth channel can be expressed as:

[0035]

[0036] Where C is the number of channels, H is the image height, and W is the image width;

[0037] S3132: Coordinate attention generation: First, concatenate the two feature maps obtained by embedding the coordinate information, and then use a 1×1 convolution F 1 The transformation obtains an intermediate feature map. The specific formula is as follows:

[0038] f=δ(F 1 ([z h , z w ]));

[0039] Where [·,·] represents the splicing on the spatial scale, δ is a nonlinear activation function, and f∈R C / r×(H+W) , r is the reduction rate;

[0040] S3133: Split f into two tensors f h and f w , using two 1×1 convolutions F h and F w Convert them into tensors g with the same number of channels as the input tensor X h and g w ;

[0041] g h =σ(F h (f h ));

[0042] gw =σ(F w (f w ));

[0043] Among them, σ is the activation function, and the output tensor Y is calculated based on it. The element of the tensor of the cth channel in the i-th row and j-th column can be expressed as:

[0044]

[0045] Preferably, the specific process of step S4 is: a 1× convolution kernel is obtained by pre-set label training, and the X obtained by feature extraction is 04 A convolution operation is performed to obtain the final output single-channel feature map, which is then converted into a picture output to achieve image segmentation.

[0046] The beneficial effects of the present invention include:

[0047] The terahertz medical image segmentation method based on the Coordinate Attention module and U-Net++ provided by the present invention obtains original medical image data and performs data preprocessing on the original medical image data; creates a U-Net++ neural network model, and adds a CoordinateAttention module after each convolution block of the U-Net++ neural network model; extracts features of the original medical image data after data preprocessing through the U-Net++ neural network model with the Coordinate Attention module, and obtains output features; uses the determined organ labels to match the output features in step S3 to obtain the medical image segmentation result. The "nested structure" of U-Net++ is used to reduce the semantic gap of the skipped part, and on this basis, the Coordinate Attention module is added to embed the spatial information into the channel attention to obtain information of a larger area, so that the model can more accurately identify the target area and avoid introducing large computational overhead, and at the same time, can obtain better segmentation results.

[0048] First, based on the "nested structure" introduced by the U-Net++ network, a Coordinate Attention module is added after each convolution block, and a CAU-Net++ structure is proposed. This structure combines position information and channel information, reduces the semantic gap of the skipped part, and focuses on both spatial and channel information to enable the model to obtain information from a larger area. It can more accurately locate and identify the target area, enhance the model's positioning ability, and enhance the model's expression ability and context perception ability. At the same time, it enhances relevant features and suppresses useless features.

[0049] Secondly, a 1×1 convolution kernel is obtained through the set label training, and the X obtained by feature extraction is 04 A convolution operation is performed to obtain the final output single-channel feature map, and the feature map is converted into a picture output to improve the accuracy of image segmentation.

[0050] Thirdly, based on the "nested structure" introduced in the U-Net++ network, a Coordinate Attention module is added after each convolution block to segment the terahertz medical images. Compared with other imaging technologies, segmentation is performed based on more information and better segmentation results can be obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 This is a schematic diagram of the network architecture of the present invention in which the Coordinate Attention module is added to U-Net++.

[0052] Figure 2 It is a schematic diagram of the process of processing by the Coordinate Attention module of the present invention.

[0053] Figure 3 This is a schematic diagram of the calculation process after adding the Coordinate Attention module to the U-Net++ of the present invention. DETAILED DESCRIPTION

[0054] The following is combined with Figure 1 to Figure 3 The present invention is further described in detail:

[0055] Example 1

[0056] The terahertz medical image segmentation method based on the Coordinate Attention module and U-Net++ includes the following steps:

[0057] S1: Acquire original medical image data and perform data preprocessing on the original medical image data. The purpose of the data preprocessing here is to convert the original medical image data into a matrix form that can be calculated to facilitate feature extraction in subsequent steps.

[0058] S2: Create a U-Net++ neural network model, and add a Coordinate Attention module after each convolution block of the U-Net++ neural network model to combine the position information and channel information to enhance the positioning ability of the model.

[0059] S3: The U-Net++ neural network model with the Coordinate Attention module is used to extract features from the original medical image data after data preprocessing and obtain output features.

[0060] S4: Use the determined organ labels to match the output features in step S3 to obtain the medical image segmentation result. The determined organ labels are obtained by supervised learning of manually labeled labels. Use a 1×1 convolution kernel obtained by pre-set label training to extract the X 04 A convolution operation is performed to obtain the final output single-channel feature map, and the feature map is converted into a picture output to achieve image segmentation.

[0061] In this embodiment, based on the "nested structure" introduced by the U-Net++ network, a Coordinate Attention module is added after each convolution block, and a CAU-Net++ structure is proposed, so that the position information and channel information are combined and considered, which can reduce the semantic gap of the skipped part and pay attention to both spatial and channel information so that the model can obtain information of a larger area, and can more accurately locate and identify the target area, enhance the positioning ability of the model, and can enhance the expression ability and context perception ability of the model, while enhancing related features and suppressing useless features.

[0062] In this embodiment, the specific process of data preprocessing in step S1 is as follows:

[0063] S11: read the original medical image data, and split the image into three-channel feature maps according to the RGB value of each pixel;

[0064] S12: Convert the three-channel feature map into a grayscale image and perform normalization processing.

[0065] The three-channel feature map in step S11 is three matrices, and the matrices are specifically as follows:

[0066]

[0067] Among them, x ij is the pixel value of each pixel, i, j are the pixel position numbers, i∈[1,m], j∈[1,n].

[0068] Example 2

[0069] On the basis of Example 1, the U-Net++ neural network model in step S2 includes multiple convolution blocks, and a Coordinate Attention module is set after each convolution block. Each of the convolution blocks includes 2 3×3 convolution kernels. The U-Net++ neural network model has four layers, and the input features of the latter layer are obtained by downsampling the first output variable of the previous layer, and the input features in the middle of each layer are obtained by upsampling the intermediate feature variables of the next layer and jumping the previous intermediate features of the layer. Based on the "nested structure" introduced in the U-Net++ network, the Coordinate Attention module is added after each convolution block to segment the terahertz medical image, and segmentation is performed on the basis of obtaining more information compared to other imaging technologies, which can obtain better segmentation results.

[0070] Example 3

[0071] Based on Example 1 or Example 2, see Figure 1 As shown, the specific process of feature extraction in step S3 is as follows:

[0072] S31: The feature map obtained in step S11 is used as the input layer of the U-Net++ neural network model with the Coordinate Attention module added, and a convolution operation is performed to obtain the intermediate feature map X 00 ;

[0073] S32: For the intermediate feature map X 00 Using the intermediate feature map X 00 Downsampling is performed to obtain the input of the next layer, and the input is subjected to the operation in step S31 to obtain the intermediate feature map X 10 ;

[0074] S33: For the intermediate feature map X 10 Upsample and combine with the intermediate feature map X 00 Perform skip connection fusion to obtain a fused feature map, and process the fused feature map with the convolution layer and Coordinate Attention module in step S31 to obtain X 01 ;

[0075] S34: Repeat steps S31-S33 to perform four-layer calculations to obtain a feature map X that combines shallow features and deep features. 04 .

[0076] For the input feature image, it is first processed by two 3×3 convolution blocks, and then processed by the CoordinateAttention module to merge the position and channel information, and output an intermediate feature map. This is a convolution block of the improved U-Net++ network, that is, the part marked with gray blocks in the figure above. The subsequent process in the figure above uses circles and intermediate variable symbols to represent this convolution block. The network has four layers. Starting from the second layer, the input features are obtained by downsampling the first output variable of the previous layer; the input features in the middle of each layer are obtained by upsampling the intermediate feature variables of the next layer and skipping the previous intermediate features of the layer. For example: the second intermediate feature X in the second layer 11 By X 10 and X 20 The feature jump connection after the last sampling is obtained after processing through the above convolution block. The structure has 4 layers and 15 convolution blocks in total, and the final output X is obtained. 04 After that, a 1×1 convolution kernel is used to process the image, and the final image segmentation result can be obtained.

[0077] In this embodiment, the specific process of step S31 is as follows:

[0078] S311: Select a convolution layer including 64 3×3 convolution kernels to perform a convolution operation on the input layer. The convolution layer is accompanied by a specified ReLU activation function layer and a BN layer. The activation function formula is as follows:

[0079] R(x) = max{0, x};

[0080] S312: Repeat the convolution operation of the convolution layer including 64 3×3 convolution kernels on the input layer and then output;

[0081] S313: Obtain a first set of intermediate feature maps through the Coordinate Attention module.

[0082] See also Figure 2 and Figure 3 As shown, the specific process of processing by the Coordinate Attention module in step S313 is as follows:

[0083] S3131: Coordinate information embedding: Input tensor X = (X 1 , ..., X C )∈R C×H×W Each channel is encoded along the horizontal and vertical directions respectively, and the tensors of size C×H×1 and C×1×W are obtained respectively. The calculation formula of the value of the hth row of Zhang Liang obtained by encoding the Cth channel along the horizontal direction is as follows:

[0084]

[0085] The value of the wth column of the tensor encoded along the vertical direction of the Cth channel can be expressed as:

[0086]

[0087] Where C is the number of channels, H is the image height, and W is the image width;

[0088] S3132: Coordinate attention generation: First, concatenate the two feature maps obtained by embedding the coordinate information, and then use a 1×1 convolution F 1 The transformation obtains an intermediate feature map. The specific formula is as follows:

[0089] f=δ(F 1 ([z h , z w ]));

[0090] Where [·,·] represents the splicing on the spatial scale, δ is a nonlinear activation function, and f∈R C / r×(H+W) , r is the reduction rate;

[0091] S3133: Split f into two tensors f h and f w , using two 1×1 convolutions F h and F w Convert them into tensors g with the same number of channels as the input tensor X h and g w ;

[0092] g h =σ(F h (f h ));

[0093] g w =σ(F w (f w ));

[0094] Among them, σ is the activation function, and the output tensor Y is calculated based on it. The element of the tensor of the cth channel in the i-th row and j-th column can be expressed as:

[0095]

[0096] The specific process of step S4 is: use a 1×1 convolution kernel obtained by pre-set label training to extract the X 04 The convolution operation is performed to obtain the final single-channel feature map, and the feature map is converted into a picture output to achieve image segmentation. A 1×1 convolution kernel is obtained through the set label training, and the X obtained by feature extraction is 04A convolution operation is performed to obtain the final output single-channel feature map, and the feature map is converted into a picture output to improve the accuracy of image segmentation.

[0097] In summary, the terahertz medical image segmentation method based on the Coordinate Attention module and U-Net++ provided by the present invention obtains the original medical image data and performs data preprocessing on the original medical image data; creates a U-Net++ neural network model, and adds the CoordinateAttention module after each convolution block of the U-Net++ neural network model; extracts features of the original medical image data after data preprocessing through the U-Net++ neural network model with the Coordinate Attention module, and obtains output features; uses the determined organ labels to match the output features in step S3 to obtain the medical image segmentation result. The "nested structure" of U-Net++ is used to reduce the semantic gap of the skipped part, and on this basis, the Coordinate Attention module is added to embed the spatial information into the channel attention to obtain information of a larger area, so that the model can more accurately identify the target area and avoid introducing large computational overhead, and at the same time, can obtain better segmentation results.

[0098] Based on the "nested structure" introduced by the U-Net++ network, the Coordinate Attention module is added after each convolution block, and the CAU-Net++ structure is proposed. This combines the position information and channel information, reduces the semantic gap of the skipped part, and pays attention to both spatial and channel information so that the model can obtain information from a larger area, more accurately locate and identify the target area, enhance the model's positioning ability, and enhance the model's expression ability and context perception ability. At the same time, it enhances related features and suppresses useless features. A 1×1 convolution kernel is obtained through the set label training, and the X obtained by feature extraction is used. 04 The convolution operation is performed to obtain the final output single-channel feature map, and the feature map is converted into a picture output to improve the accuracy of image segmentation. Based on the "nested structure" introduced in the U-Net++ network, the Coordinate Attention module is added after each convolution block to segment the terahertz medical image. Compared with other imaging technologies, segmentation is performed based on more information and better segmentation results can be obtained.

Claims

1. A terahertz medical image segmentation method based on Coordinate Attention module and U-Net++, characterized in that: The following steps are involved: S1: Acquire original medical image data, and perform data preprocessing on the original medical image data; S2: Create a U-Net++ neural network model, and add a Coordinate Attention module after each convolution block of the U-Net++ neural network model; S3: The U-Net++ neural network model with the Coordinate Attention module is used to extract features from the original medical image data after data preprocessing, and output features are obtained; S4: Matching the determined organ labels with the output features in step S3 to obtain a medical image segmentation result, wherein the determined organ labels are manually annotated labels obtained through supervised learning.

2. According to claim 1, the terahertz medical image segmentation method based on the Coordinate Attention module and U-Net++ is characterized in that: The specific process of data preprocessing in step S1 is as follows: S11: read the original medical image data, and split the image into three-channel feature maps according to the RGB value of each pixel; S12: Convert the three-channel feature map into a grayscale image and perform normalization processing.

3. The terahertz medical image segmentation method based on Coordinate Attention module and U-Net++ according to claim 2, characterized in that: The three-channel feature map in step S11 is three matrices, and the matrices are specifically as follows: Among them, x ij is the pixel value of each pixel, i, j are the pixel position numbers, i∈[1,m], j∈[1,n].

4. The terahertz medical image segmentation method based on Coordinate Attention module and U-Net++ according to claim 1, characterized in that: The U-Net++ neural network model in step S2 includes multiple convolution blocks, a Coordinate Attention module is set after each convolution block, each of the convolution blocks includes 2 3×3 convolution kernels, and the U-Net++ neural network model has four layers. The input features of the latter layer are obtained by downsampling the first output variable of the previous layer, and the input features in the middle of each layer are obtained by upsampling the intermediate feature variables of the next layer and skipping the previous intermediate features of the layer.

5. The terahertz medical image segmentation method based on Coordinate Attention module and U-Net++ according to claim 2, characterized in that: The specific process of feature extraction in step S3 is as follows: S31: The feature map obtained in step S11 is used as the input layer of the U-Net++ neural network model with the Coordinate Attention module added, and a convolution operation is performed to obtain the intermediate feature map X 00 ; S32: For the intermediate feature map X 00 Using the intermediate feature map X 00 Downsampling is performed to obtain the input of the next layer, and the input is subjected to the operation in step S31 to obtain the intermediate feature map X 10 ; S33: For the intermediate feature map X 10 Upsample and combine with the intermediate feature map X 00 Perform skip connection fusion to obtain a fused feature map, and process the fused feature map with the convolution layer and Coordinate Attention module in step S31 to obtain X 01 ; S34: Repeat steps S31-S33 to perform four-layer calculations to obtain a feature map X that combines shallow features and deep features. 04 .

6. The terahertz medical image segmentation method based on Coordinate Attention module and U-Net++ according to claim 5, characterized in that: The specific process of step S31 is as follows: S311: Select a convolution layer including 64 3×3 convolution kernels to perform a convolution operation on the input layer. The convolution layer is accompanied by a specified ReLU activation function layer and a BN layer. The activation function formula is as follows: R(x)=max{0,x}; S312: Repeat the convolution operation of the convolution layer including 64 3×3 convolution kernels on the input layer and then output; S313: Obtain a first set of intermediate feature maps through the Coordinate Attention module.

7. The terahertz medical image segmentation method based on Coordinate Attention module and U-Net++ according to claim 5, characterized in that: The specific process of processing by the Coordinate Attention module in step S313 is as follows: S3131: Coordinate information embedding: Input tensor X = (X1, ..., X C )∈R C×H×W Each channel is encoded along the horizontal and vertical directions respectively, and the tensors of size C×H×1 and C×1×W are obtained respectively. The calculation formula of the value of the hth row of Zhang Liang obtained by encoding the Cth channel along the horizontal direction is as follows: The value of the wth column of the tensor encoded along the vertical direction of the Cth channel can be expressed as: Where C is the number of channels, H is the image height, and W is the image width; S3132: Coordinate attention generation: First, concatenate the two feature maps obtained from the coordinate information embedding, and then use a 1×1 convolution F1 transformation to obtain an intermediate feature map. The specific formula is as follows: f=δ(F1([z h ,z w ])); Where [·,·] represents the splicing on the spatial scale, δ is a nonlinear activation function, and f∈R C / r×(H+W) , r is the reduction rate; S3133: Split f into two tensors f h and f w , using two 1×1 convolutions F h and F w Convert them into tensors g with the same number of channels as the input tensor X h and g w ; g h =σ(F h (f h )); g w =σ(F w (f w )); Among them, σ is the activation function, and the output tensor Y is calculated based on it. The element of the tensor of the cth channel in the i-th row and j-th column can be expressed as:

8. The terahertz medical image segmentation method based on Coordinate Attention module and U-Net++ according to claim 1, characterized in that: The specific process of step S4 is: use a 1×1 convolution kernel obtained by pre-set label training to extract the X 04 A convolution operation is performed to obtain the final output single-channel feature map, which is then converted into a picture output to achieve image segmentation.

Images