Terahertz medical image segmentation method based on Coordinate Attention module and U-Net++

Abstract

The present invention belongs to the field of server heat dissipation technology, and specifically relates to a terahertz medical image segmentation method based on a Coordinate Attention module and U-Net++. The method comprises obtaining raw medical image data and performing data preprocessing on the raw medical image data; creating a U-Net++ neural network model and adding a Coordinate Attention module after each convolution block of the U-Net++ neural network model; extracting features from the preprocessed raw medical image data using the U-Net++ neural network model to obtain output features; and matching the output features in step S3 with the determined organ labels to obtain a medical image segmentation result. The method utilizes the "nested structure" of U-Net++ to reduce the semantic gap between skipped connections, and on this basis, adds a Coordinate Attention module to embed spatial information into channel attention, thereby obtaining information on a larger area. This enables the model to more accurately identify the target area while avoiding large computational overhead and achieving better segmentation results.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical image segmentation, and in particular relates to a terahertz medical image segmentation method based on a Coordinate Attention module and U-Net++. Background Art

[0002] Terahertz waves are electromagnetic waves with frequencies between 0.1THz and 10THz and wavelengths between 30μm and 3000μm. Photons in the terahertz band have low energy and, unlike X-rays, lack ionizing radiation, causing no significant ionizing damage to biological tissue. Unlike ultrasound imaging, which requires contact with the sample using a coupling agent, terahertz imaging is a contactless, non-destructive testing method. Furthermore, the frequency of terahertz waves falls within the energy levels of many biological macromolecules, making it easier to identify substances with "fingerprint" properties.

[0003] There are many well-performing network architectures in the field of medical image segmentation, among which U-Net is a widely used neural network. Based on a fully convolutional neural network, it features a simple structure, enabling more accurate segmentation with fewer samples, and employs skip connections to recover lost information. However, the semantic gap between the features in its encoder and decoder is significant, and the skip connections in U-Net directly concatenate these features, which can lead to reduced segmentation accuracy.

[0004] While some research has applied U-Net to terahertz image segmentation, such as in terahertz security inspection and terahertz coaxial digital holography, U-Net or its variants have not been applied to terahertz medical image segmentation. The classic U-Net architecture suffers from inaccurate edge segmentation in terahertz images.

[0005] The U-Net network proposed for the cell segmentation task has a simple structure and combines shallow feature information and deep feature information through skip connections. It performs well in small-sample image segmentation tasks. However, the large semantic gap between the decoding and encoding parts may lead to inaccurate segmentation results.

[0006] Therefore, how to improve the existing medical image segmentation methods to enhance the model's expressiveness and context-awareness, while enhancing relevant features and suppressing useless features, is a technical problem that urgently needs to be solved. Summary of the Invention

[0007] The purpose of the present invention is to provide a terahertz medical image segmentation method based on the Coordinate Attention module and U-Net++, so as to improve the medical image segmentation method in the prior art, enhance the model's expressive power and context perception ability, and at the same time enhance relevant features and suppress useless features.

[0008] In order to solve the above technical problems, the technical solutions adopted by the present invention are as follows:

[0009] The terahertz medical image segmentation method based on the Coordinate Attention module and U-Net++ includes the following steps:

[0010] S1: Acquire original medical image data and perform data preprocessing on the original medical image data;

[0011] S2: Create a U-Net++ neural network model and add a Coordinate Attention module after each convolution block of the U-Net++ neural network model;

[0012] S3: Feature extraction is performed on the raw medical image data after data preprocessing using the U-Net++ neural network model with the Coordinate Attention module, and output features are obtained.

[0013] S4: Matching the determined organ labels with the output features in step S3 to obtain medical image segmentation results, wherein the determined organ labels are manually labeled labels obtained through supervised learning.

[0014] Preferably, the specific process of data preprocessing in step S1 is as follows:

[0015] S11: Read the original medical image data and split the image into three-channel feature maps according to the RGB value of each pixel;

[0016] S12: Convert the three-channel feature map into a grayscale image and perform normalization processing.

[0017] Preferably, the three-channel feature map in step S11 is three matrices, and the matrices are specifically as follows:

[0018]

[0019] Among them, x ij is the pixel value of each pixel, i, j is the pixel position number, i∈[1,m], j∈[1,n].

[0020] Preferably, the U-Net++ neural network model in step S2 includes multiple convolution blocks, a Coordinate Attention module is set after each convolution block, each of the convolution blocks includes two 3×3 convolution kernels, and the U-Net++ neural network model has four layers. The input features of the latter layer are obtained by downsampling the first output variable of the previous layer, and the input features in the middle of each layer are obtained by upsampling the intermediate feature variables of the next layer and skipping the previous intermediate features of the layer.

[0021] Preferably, the specific process of feature extraction in step S3 is as follows:

[0022] S31: The feature map obtained in step S11 is used as the input layer of the U-Net++ neural network model with the Coordinate Attention module added, and a convolution operation is performed to obtain the intermediate feature map X 00 ;

[0023] S32: Intermediate feature map X 00 Using the intermediate feature map X 00 Downsampling is performed to obtain the input of the next layer, and the operation in step S31 is performed on the input to obtain the intermediate feature map X 10 ;

[0024] S33: Intermediate feature map X 10 Upsample and combine with the intermediate feature map X 00 Perform skip connection fusion to obtain a fused feature map, and process the fused feature map with the convolution layer and Coordinate Attention module in step S31 to obtain X 01 ;

[0025] S34: Repeat steps S31-S33 to perform four-layer calculations to obtain a feature map X that combines shallow features and deep features. 04 .

[0026] Preferably, the specific process of step S31 is as follows:

[0027] S311: Select a convolution layer including 64 3×3 convolution kernels to perform a convolution operation on the input layer. The convolution layer is accompanied by a specified ReLU activation function layer and a BN layer. The activation function formula is as follows:

[0028] R(x) = max{0, x};

[0029] S312: Repeat the convolution operation of the input layer using 64 3×3 convolution kernels and output the result.

[0030] S313: Obtain the first set of intermediate feature maps through the Coordinate Attention module.

[0031] Preferably, the specific process of processing by the Coordinate Attention module in step S313 is as follows:

[0032] S3131: Coordinate information embedding: Input tensor X=(X1,...,X C )∈R C×H×W Each channel is encoded along the horizontal and vertical directions respectively, and the sizes of tensors are obtained respectively. The calculation formula of the h-th row of the Zhang Liang obtained by encoding the C-th channel along the horizontal direction is as follows:

[0033]

[0034] The value of the wth column of the tensor obtained by encoding the Cth channel along the vertical direction can be expressed as:

[0035]

[0036] Where C is the number of channels, H is the image height, and W is the image width;

[0037] S3132: Coordinate attention generation: First, the two feature maps obtained from the coordinate information embedding are concatenated together, and then a 1×1 convolution F1 transformation is used to obtain an intermediate feature map. The specific formula is as follows:

[0038] f=δ(F1([z h , z w ]));

[0039] Where [·,·] represents the splicing on the spatial scale, δ is the nonlinear activation function, f∈R C / r×(H+W) , r is the reduction rate;

[0040] S3133: Split f into two tensors f h and f w , using two 1×1 convolution F h and F w Convert them into tensors g with the same number of channels as the input tensor X h and g w ;

[0041] g h =σ(F h (f h ));

[0042] g w =σ(F w (fw ));

[0043] Among them, σ is the activation function, and the output tensor Y is calculated based on it. The element of the tensor of the cth channel in the i-th row and j-th column can be expressed as:

[0044]

[0045] Preferably, the specific process of step S4 is: using a 1× convolution kernel obtained by pre-set label training, and performing the X 04 Perform convolution operation to obtain the final output single-channel feature map, and convert the feature map into a picture output to achieve image segmentation.

[0046] The beneficial effects of the present invention include:

[0047] The terahertz medical image segmentation method provided by the present invention, based on the Coordinate Attention module and U-Net++, obtains raw medical image data and performs data preprocessing on the raw medical image data; creates a U-Net++ neural network model and adds the Coordinate Attention module after each convolution block of the U-Net++ neural network model; extracts features from the preprocessed raw medical image data using the U-Net++ neural network model to obtain output features; and matches the output features in step S3 with the determined organ labels to obtain a medical image segmentation result. The "nested structure" of U-Net++ is utilized to reduce the semantic gap of the skipped connection part, and on this basis, the Coordinate Attention module is added to embed spatial information into the channel attention to obtain information of a larger area, enabling the model to more accurately identify the target area while avoiding the introduction of large computational overhead, while also achieving better segmentation results.

[0048] First, based on the "nested structure" introduced by the U-Net++ network, a Coordinate Attention module is added after each convolutional block to propose the CAU-Net++ structure. This structure combines position information and channel information, reduces the semantic gap between skip connections, and focuses on both spatial and channel information, allowing the model to obtain information from a larger area. It can more accurately locate and identify target areas, enhance the model's positioning ability, and improve its expressiveness and contextual awareness. At the same time, it enhances relevant features and suppresses useless features.

[0049] Secondly, a 1×1 convolution kernel is obtained by training the set labels, and the X obtained by feature extraction is 04A convolution operation is performed to obtain the final output single-channel feature map, and the feature map is converted into an image output to improve the accuracy of image segmentation.

[0050] Thirdly, based on the "nested structure" introduced in the U-Net++ network, a Coordinate Attention module is added after each convolution block to segment terahertz medical images. Compared with other imaging technologies, segmentation is performed based on more information and better segmentation results can be obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 This is a schematic diagram of the network architecture of the present invention by adding the Coordinate Attention module to U-Net++.

[0052] Figure 2 This is a schematic diagram of the process of processing by the Coordinate Attention module of the present invention.

[0053] Figure 3 This is a schematic diagram of the calculation process after adding the Coordinate Attention module to the U-Net++ of the present invention. DETAILED DESCRIPTION

[0054] The following is combined with Figures 1 to 3 The present invention is described in further detail:

[0055] Example 1

[0056] The terahertz medical image segmentation method based on the Coordinate Attention module and U-Net++ includes the following steps:

[0057] S1: Acquire original medical image data and perform data preprocessing on the original medical image data. The purpose of the data preprocessing here is to convert the original medical image data into a matrix form that can be calculated to facilitate feature extraction in subsequent steps.

[0058] S2: Create a U-Net++ neural network model and add a Coordinate Attention module after each convolutional block in the U-Net++ neural network model to combine position information and channel information to enhance the model's positioning capabilities.

[0059] S3: The U-Net++ neural network model with the Coordinate Attention module is used to extract features from the original medical image data after data preprocessing and obtain output features.

[0060] S4: Use the determined organ labels to match the output features in step S3 to obtain the medical image segmentation results. The determined organ labels are obtained by supervised learning of manually labeled labels. A 1×1 convolution kernel is obtained by pre-set label training and the X 04 Perform convolution operation to obtain the final output single-channel feature map, and convert the feature map into a picture output to achieve image segmentation.

[0061] In this embodiment, based on the "nested structure" introduced by the U-Net++ network, a Coordinate Attention module is added after each convolution block to propose a CAU-Net++ structure. This structure combines position information and channel information, reduces the semantic gap of the skipped parts, and focuses on both spatial and channel information to enable the model to obtain information from a larger area. It can more accurately locate and identify the target area, enhance the model's positioning ability, and enhance the model's expressiveness and context perception capabilities. At the same time, it enhances relevant features and suppresses useless features.

[0062] In this embodiment, the specific process of data preprocessing in step S1 is as follows:

[0063] S11: Read the original medical image data and split the image into three-channel feature maps according to the RGB value of each pixel;

[0064] S12: Convert the three-channel feature map into a grayscale image and perform normalization processing.

[0065] The three-channel feature map in step S11 is three matrices, and the matrices are specifically as follows:

[0066]

[0067] Among them, x ij is the pixel value of each pixel, i, j is the pixel position number, i∈[1,m], j∈[1,n].

[0068] Example 2

[0069] Based on Example 1, the U-Net++ neural network model in step S2 includes multiple convolution blocks, and a Coordinate Attention module is set after each convolution block. Each convolution block includes two 3×3 convolution kernels. The U-Net++ neural network model has four layers. The input features of the latter layer are obtained by downsampling the first output variable of the previous layer, and the input features in each layer are obtained by upsampling the intermediate feature variables of the next layer and skipping the previous intermediate features of the layer. Based on the "nested structure" introduced in the U-Net++ network, the Coordinate Attention module is added after each convolution block to segment terahertz medical images. Compared with other imaging technologies, segmentation is performed based on more information, which can obtain better segmentation results.

[0070] Example 3

[0071] On the basis of Example 1 or Example 2, see Figure 1 As shown, the specific process of feature extraction in step S3 is as follows:

[0072] S31: The feature map obtained in step S11 is used as the input layer of the U-Net++ neural network model with the Coordinate Attention module added, and a convolution operation is performed to obtain the intermediate feature map X 00 ;

[0073] S32: Intermediate feature map X 00 Using the intermediate feature map X 00 Downsampling is performed to obtain the input of the next layer, and the operation in step S31 is performed on the input to obtain the intermediate feature map X 10 ;

[0074] S33: Intermediate feature map X 10 Upsample and combine with the intermediate feature map X 00 Perform skip connection fusion to obtain a fused feature map, and process the fused feature map with the convolution layer and Coordinate Attention module in step S31 to obtain X 01 ;

[0075] S34: Repeat steps S31-S33 to perform four-layer calculations to obtain a feature map X that combines shallow features and deep features. 04 .

[0076] The input feature image is first processed by two 3×3 convolution blocks, and then processed by the CoordinateAttention module to merge the position and channel information, outputting an intermediate feature map. This is a convolution block of the improved U-Net++ network, which is the gray block in the figure above. The subsequent process in the figure above uses circles and intermediate variable symbols to represent this convolution block. The network has four layers. Starting from the second layer, the input features are obtained by downsampling the first output variable of the previous layer; the intermediate input features of each layer are obtained by upsampling the intermediate feature variables of the next layer and skipping the previous intermediate features of the layer. For example, the second intermediate feature X in the second layer is obtained by upsampling the intermediate feature variables of the next layer and skipping the previous intermediate features of the layer. 11 By X 10 and X 20 The feature jump connection after the last sampling is processed by the above convolution block. The structure has 4 layers and 15 convolution blocks to obtain the final output X 04 Finally, a 1×1 convolution kernel is used to process the image, and the final image segmentation result can be obtained.

[0077] In this embodiment, the specific process of step S31 is as follows:

[0078] S311: Select a convolution layer including 64 3×3 convolution kernels to perform a convolution operation on the input layer. The convolution layer is accompanied by a specified ReLU activation function layer and a BN layer. The activation function formula is as follows:

[0079] R(x) = max{0, x};

[0080] S312: Repeat the convolution operation of the input layer using 64 3×3 convolution kernels and output the result.

[0081] S313: Obtain the first set of intermediate feature maps through the Coordinate Attention module.

[0082] See also Figure 2 and Figure 3 As shown, the specific process of processing by the Coordinate Attention module in step S313 is as follows:

[0083] S3131: Coordinate information embedding: Input tensor X=(X1,...,X C )∈R C×H×W Each channel is encoded along the horizontal and vertical directions respectively, and the sizes of tensors are obtained respectively. The calculation formula of the h-th row of the Zhang Liang obtained by encoding the C-th channel along the horizontal direction is as follows:

[0084]

[0085] The value of the wth column of the tensor obtained by encoding the Cth channel along the vertical direction can be expressed as:

[0086]

[0087] Where C is the number of channels, H is the image height, and W is the image width;

[0088] S3132: Coordinate attention generation: First, the two feature maps obtained from the coordinate information embedding are concatenated together, and then a 1×1 convolution F1 transformation is used to obtain an intermediate feature map. The specific formula is as follows:

[0089] f=δ(F1([z h , z w ]));

[0090] Where [·,·] represents the splicing on the spatial scale, δ is the nonlinear activation function, f∈R C / r×(H+W) , r is the reduction rate;

[0091] S3133: Split f into two tensors f h and f w , using two 1×1 convolution F h and F w Convert them into tensors g with the same number of channels as the input tensor X h and g w ;

[0092] g h =σ(F h (f h ));

[0093] g w =σ(F w (f w ));

[0094] Among them, σ is the activation function, and the output tensor Y is calculated based on it. The element of the tensor of the cth channel in the i-th row and j-th column can be expressed as:

[0095]

[0096] The specific process of step S4 is: use the pre-set label training to obtain a 1×1 convolution kernel, and perform the feature extraction on the X 04 Perform convolution operation to obtain the final output single-channel feature map, and convert the feature map into a picture output to achieve image segmentation. A 1×1 convolution kernel is obtained through the set label training, and the X obtained by feature extraction is 04A convolution operation is performed to obtain the final output single-channel feature map, and the feature map is converted into an image output to improve the accuracy of image segmentation.

[0097] In summary, the terahertz medical image segmentation method based on the Coordinate Attention module and U-Net++ provided by the present invention obtains raw medical image data and performs data preprocessing on the raw medical image data; creates a U-Net++ neural network model and adds a Coordinate Attention module after each convolution block of the U-Net++ neural network model; extracts features from the raw medical image data after data preprocessing using the U-Net++ neural network model with the Coordinate Attention module and obtains output features; and matches the output features in step S3 with the determined organ labels to obtain the medical image segmentation results. The "nested structure" of U-Net++ is utilized to reduce the semantic gap of the skipped connection part, and on this basis, the Coordinate Attention module is added to embed spatial information into the channel attention to obtain information of a larger area, enabling the model to more accurately identify the target area while avoiding the introduction of large computational overhead, while also obtaining better segmentation results.

[0098] Based on the "nested structure" introduced by the U-Net++ network, a Coordinate Attention module is added after each convolution block to propose the CAU-Net++ structure. This combines the consideration of position information and channel information, which can reduce the semantic gap of the skipped part and focus on both spatial and channel information to enable the model to obtain information from a larger area, more accurately locate and identify the target area, enhance the model's positioning ability, enhance the model's expressiveness and context perception, and at the same time enhance relevant features and suppress useless features. A 1×1 convolution kernel is obtained through set label training, and the X obtained by feature extraction is used to extract the image. 04 A convolution operation is performed to obtain a final single-channel feature map, which is then converted into an image for output, improving image segmentation accuracy. Leveraging the "nested structure" introduced in the U-Net++ network, a Coordinate Attention module is added after each convolutional block to segment terahertz medical images. This segmentation is based on more information than other imaging techniques, resulting in better segmentation results.

Claims

1. A terahertz medical image segmentation method based on the Coordinate Attention module and U-Net++ is characterized by: The following steps are involved: S1: Acquire original medical image data and perform data preprocessing on the original medical image data; S2: Create a U-Net++ neural network model and add a Coordinate Attention module after each convolution block of the U-Net++ neural network model; S3: Feature extraction is performed on the raw medical image data after data preprocessing using the U-Net++ neural network model with the Coordinate Attention module, and output features are obtained. S4: Matching the determined organ labels with the output features in step S3 to obtain a medical image segmentation result, wherein the determined organ labels are obtained by supervised learning using manually labeled labels; The specific process of feature extraction in step S3 is as follows: S31: The feature map obtained in step S11 is used as the input layer of the U-Net++ neural network model with the Coordinate Attention module added, and a convolution operation is performed to obtain the intermediate feature map X 00 ; S32: Intermediate feature map X 00 Using the intermediate feature map X 00 Downsampling is performed to obtain the input of the next layer, and the operation in step S31 is performed on the input to obtain the intermediate feature map X 10 ; S33: Intermediate feature map X 10 Upsample and combine with the intermediate feature map X 00 Perform skip connection fusion to obtain a fused feature map, and process the fused feature map with the convolution layer and Coordinate Attention module in step S31 to obtain X 01 ; S34: Repeat steps S31-S33 to perform four-layer calculations to obtain a feature map X that combines shallow features and deep features. 04 ; The specific process of step S31 is as follows: S311: Select a convolution layer including 64 3×3 convolution kernels to perform a convolution operation on the input layer. The convolution layer is accompanied by a specified ReLU activation function layer and a BN layer. The activation function formula is as follows: R(x)=max{0,x}; S312: Repeat the convolution operation of the convolution layer including 64 3×3 convolution kernels on the input layer and output it; S313: Obtain the first set of intermediate feature maps through the Coordinate Attention module; The specific process of processing by the Coordinate Attention module in step S313 is as follows: S3131: Coordinate information embedding: Input tensor X=(X1,...,X C )∈R C×H×W Each channel is encoded along the horizontal and vertical directions respectively, and the sizes of tensors are obtained respectively. The calculation formula of the h-th row of the Zhang Liang obtained by encoding the C-th channel along the horizontal direction is as follows: The value of the wth column of the tensor obtained by encoding the Cth channel along the vertical direction can be expressed as: Where C is the number of channels, H is the image height, and W is the image width; S3132: Coordinate attention generation: First, the two feature maps obtained from the coordinate information embedding are concatenated together, and then a 1×1 convolution F1 transformation is used to obtain an intermediate feature map. The specific formula is as follows: f=δ(F1([z h ,z w ])); Where [·,·] represents the splicing on the spatial scale, δ is the nonlinear activation function, f∈R C / r×(H+W) , r is the reduction rate; S3133: Split f into two tensors f h and f w , using two 1×1 convolution F h and F w Convert them into tensors g with the same number of channels as the input tensor X h and g w ; g h =σ(F h (f h )); g w =σ(F w (f w )); Among them, σ is the activation function, and the output tensor Y is calculated based on it. The element of the tensor of the cth channel in the i-th row and j-th column can be expressed as:

2. The terahertz medical image segmentation method based on the Coordinate Attention module and U-Net++ according to claim 1, characterized in that: The specific process of data preprocessing in step S1 is as follows: S11: Read the original medical image data and split the image into three-channel feature maps according to the RGB value of each pixel; S12: Convert the three-channel feature map into a grayscale image and perform normalization processing.

3. The terahertz medical image segmentation method based on the Coordinate Attention module and U-Net++ according to claim 2, characterized in that: The three-channel feature map in step S11 is three matrices, and the matrices are specifically as follows: Among them, x ij is the pixel value of each pixel, i, j is the pixel position number, i∈[1,m], j∈[1,n].

4. The terahertz medical image segmentation method based on the Coordinate Attention module and U-Net++ according to claim 1, characterized in that: The U-Net++ neural network model in step S2 includes multiple convolution blocks, and a Coordinate Attention module is set after each convolution block. Each convolution block includes two 3×3 convolution kernels. The U-Net++ neural network model has four layers. The input features of the latter layer are obtained by downsampling the first output variable of the previous layer, and the input features in the middle of each layer are obtained by upsampling the intermediate feature variables of the next layer and skipping the previous intermediate features of the layer.

5. The terahertz medical image segmentation method based on the Coordinate Attention module and U-Net++ according to claim 1, characterized in that: The specific process of step S4 is: use the pre-set label training to obtain a 1×1 convolution kernel, and perform the feature extraction on the X 04 Perform convolution operation to obtain the final output single-channel feature map, and convert the feature map into a picture output to achieve image segmentation.

Images