A medical image segmentation method based on adaptive scale

Abstract

The application provides a medical image segmentation method based on adaptive scale, which comprises the following steps: setting a parallel convolution layer module with different expansion convolutions to extract image features at different scales; introducing an attention module, which inputs a joint feature map that aggregates local and global features, and obtains an attention feature map through multi-layer convolution, wherein each layer of the attention feature map corresponds to the weight of the corresponding layer feature in the parallel convolution layer; and introducing a multi-scale fusion module to aggregate features at different scales from the parallel convolution layer through the attention feature map. The application aggregates features at different scales from the parallel convolution layer through the attention feature map, so that the network can adaptively select the scale, and the accuracy of medical image segmentation in a high-noise complex scene is ensured.

Description

Technical Field

[0001] This invention belongs to the field of computer vision, and in particular relates to a medical image segmentation method based on adaptive scale. Background Technology

[0002] In the past few years, various 2D and 3D deep learning segmentation algorithms have received widespread attention from researchers. The deep learning methods for semantic segmentation in the existing literature are mainly divided into two categories: one is based on encoder-decoder architecture, and the other is based on dilated convolution architecture.

[0003] Typically, the former extracts multi-layer features from the encoder and fuses them with the decoder to obtain a segmentation result image, while the latter extracts features from a larger receptive field by setting a certain dilation radius, finally obtaining a segmentation result image. Although these methods have made remarkable progress, some challenges remain. Specifically, the former is prone to losing some spatial information, potentially leading to inaccurate edge segmentation; the latter does not use downsampling, preserving spatial information, but requires pre-determining the dilation radius and is extremely sensitive to noise. Therefore, when image features are blurred or there is local noise, these methods often fail to achieve satisfactory results.

[0004] Therefore, an adaptive scale-based medical image segmentation method is proposed. By reducing the loss of spatial information through a small number of downsampling operations and fusing multi-scale features through an attention mechanism, the network can be applied to complex scenes with high noise, thus ensuring the accuracy of medical image segmentation in such scenes. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a medical image segmentation method based on adaptive scale, comprising the following steps: Step 1: Use a parallel convolutional layer module with different dilation convolutions to extract image features at different scales; Step 2: Introduce an attention module. The input to this module is a joint feature map, which aggregates local and global features. An attention feature map is obtained through multiple convolutions. Each attention feature map corresponds to the weight of the corresponding feature in the parallel convolutional layer. Step 3: A multi-scale fusion module is introduced to aggregate features from different scales of parallel convolutional layers through attention feature maps.

[0006] Preferably, the parallel convolutional layer module can be configured with parallel convolutional layers with different dilation convolutions to extract image features at different scales. Its main steps include: Given a feature map P extracted by a backbone CNN as input, a parallel convolutional layer with different dilation levels is used to extract image features at different scales, which can be defined as follows:

[0007] in, Indicated by Feature extraction of the image is performed to determine the radius of expansion. K is an adjustable parameter, used in experiments. Through different expansion radii Image features are extracted, and the resulting feature maps have different receptive fields. The receptive field of each layer is calculated as follows:

[0008] Where e is the kernel size and s is the stride. The receptive field during input.

[0009] Preferably, the attention module is capable of generating dependency coefficients from parallel convolutional layers based on local and global features of the image, and the main steps include: Given a feature map P extracted by the backbone CNN as input, a multi-scale joint feature is obtained by fusing the global features of the image, specifically defined as follows:

[0010] in For max pooling, Broadcast , representing the aggregation operation, and finally the attention coefficient is calculated using the following formula:

[0011]

[0012] in, This is a convolution operation with a kernel size of 1. The activation function is ReLU, which is a non-linear activation function with 256 output channels. This is a convolution operation with a kernel size of 1, f being the SoftMax activation function, and 4 output channels. , respectively, correspond to the fusion weights of the parallel convolutional layers, where, This represents the weight of the i-th layer, allowing the network to adaptively select an optimal feature map.

[0013] Preferably, the multi-scale fusion module merges feature maps from different scales of parallel convolutional layers, aggregating multi-scale feature maps using the following formula:

[0014] in, This represents the output of the i-th layer of the parallel convolutional layer module. For the i-th channel of the attention module, This represents element-wise multiplication, and the final F is the fused multi-scale feature map. The feature map has different degrees of dependence on different scales, and the network can learn an attention coefficient about scale through local and global features.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: By proposing a parallel convolutional layer module, the present invention enables the network to extract features at different scales, enhances the decision-making power of the model, avoids the network making incorrect decisions on fuzzy features, calculates weights at different scales through the proposed attention module, and enables the network to generate coefficients of each output from the parallel convolutional layer based on local and global image features. By aggregating features from different scales of the parallel convolutional layer through the attention map, the network can adaptively select the scale, thus ensuring the accuracy of medical image segmentation in high-noise and complex scenes. Attached Figure Description

[0016] Figure 1 This is a diagram illustrating the overall architecture of the present invention; Figure 2 This is the adaptive scaling diagram of the present invention; Figure 3 This is the attention coefficient diagram of the present invention; Figure 4 This is a comparison chart showing the accuracy of the method of the present invention in Comparative Example 1 compared with other methods. Detailed Implementation

[0017] The present invention will be further described below with reference to the accompanying drawings: Example: As attached Figure 1 As shown, this invention selects an encoder and a decoder as the backbone network. The decoder is used to extract image features to achieve target localization. By incorporating features from the encoder and fusing spatial information, the decoder achieves accurate image segmentation, as shown in the attached figure. Figure 2 As shown, the proposed adaptive module comprises three core components: a parallel convolutional layer module, an attention module, and a multi-scale fusion module.

[0018] A medical image segmentation method based on adaptive scale, the method steps are as follows: Step 1: By setting up a parallel convolutional layer module with different dilation levels, image features at different scales can be extracted, thereby improving the network's decision-making power and avoiding decision-making errors on blurred features.

[0019] Given a feature map P extracted by a backbone CNN as input, a parallel convolutional layer with different dilation levels is used to extract image features at different scales, which can be defined as follows:

[0020] in, Indicated by Feature extraction of the image is performed to determine the radius of expansion. K is an adjustable parameter, used in experiments. Through different expansion radii Image features are extracted, and the resulting feature maps have different receptive fields. The receptive field of each layer is calculated as follows:

[0021] Where e is the kernel size and s is the stride. This refers to the receptive field during input. (See attached image.) Figure 2 As shown, this paper sets up four convolutional layers, each with specific settings. , , , .

[0022] Step 2: To obtain the optimal scale for each center pixel, an attention module is introduced. The input to this module is a joint feature map, which aggregates local and global features. Through multiple convolutions, an attention feature map is obtained, with each attention feature map representing the weights of the corresponding features in the parallel layers.

[0023] Similarly, given a feature map P extracted by the backbone CNN as input, the global features of the image are fused to obtain a multi-scale joint feature, specifically defined as follows:

[0024] in For max pooling, Broadcast , representing the aggregation operation, and finally the attention coefficient is calculated using the following formula:

[0025]

[0026] in, This is a convolution operation with a kernel size of 1. The activation function is ReLU, which is a non-linear activation function with 256 output channels. This is a convolution operation with a kernel size of 1, f being the SoftMax activation function, and 4 output channels. , respectively, correspond to the fusion weights of the parallel convolutional layers, where, This represents the weight of the i-th layer, allowing the network to adaptively select an optimal feature map.

[0027] like Figure 3 As shown in the figure, (a) is the true label of the image, (b) is the attention coefficient corresponding to the first layer, (c) is the attention coefficient corresponding to the second layer, (d) is the attention coefficient corresponding to the third layer, and (e) is the attention coefficient corresponding to the fourth layer. It can be seen from the figure that the subject and the edge of the image depend on different scales. The subject mainly depends on a specific scale, while the edge of the object needs to make joint decisions based on multi-scale features to achieve accurate segmentation.

[0028] Step 3: A multi-scale fusion module is introduced to aggregate features from different scales of parallel convolutional layers through attention maps. This module merges feature maps from different scales of parallel convolutional layers using the following formula:

[0029] The above formula This represents the output of the i-th layer of the parallel convolutional layer module. For the i-th channel of the attention module, This represents element-wise multiplication, and the final F is the fused multi-scale feature map. Unlike traditional fusion, this feature map has different degrees of dependence on different scales. The network can learn an attention coefficient about scale through local and global features.

[0030] Comparative Example 1: To verify the superiority of the method of this invention, a comparative experiment was conducted with other methods on a set of cochlear datasets: The dataset includes 17 cases and 150 CT slices. All experiments were conducted using the PyTorch learning library in a GeForce RTX 2080Ti GPU environment.

[0031] like Figure 4The figures show the accuracy of the FCN method, UNet method, SegNet method, PSPNet method, MVNet method, ASPP method, AFN method, U-AFN method and the method of the present invention, respectively. These results demonstrate that it is feasible and effective to use the method of the present invention to achieve medical image segmentation in high-noise and complex scenes. In addition, the method of the present invention is also applicable to the identification of small, medium and large medical tissues and organs, as well as occluded and noisy targets.

Claims

1. A medical image segmentation method based on adaptive scale, characterized in that, The method includes the following steps: Step 1: Use a parallel convolutional layer module with different dilation convolutions to extract image features at different scales; Step 2: Introduce an attention module. This module takes a joint feature map as input, which aggregates local and global features. Through multiple convolutional layers, an attention feature map is obtained, with each layer corresponding to the weights of features from the corresponding layers in the parallel convolutional layers. The attention module can generate dependency coefficients on the features from the parallel convolutional layers based on the local and global features of the image. The main steps include: Given a feature map extracted by a backbone CNN P As input, the global features of the fused image are used to obtain a multi-scale joint feature, specifically defined as follows: in For max pooling, Broadcast , representing the aggregation operation, and finally the attention coefficient is calculated using the following formula: in, This is a convolution operation with a kernel size of 1. The activation function is ReLU, which is a non-linear activation function with 256 output channels. This is a convolution operation with a kernel size of 1. f This is the SoftMax activation function, with 4 output channels. , respectively, correspond to the fusion weights of the parallel convolutional layers, where, Indicates the first i The weights of the layers allow the network to adaptively select an optimal feature map. Step 3: A multi-scale fusion module is introduced to aggregate features from different scales of parallel convolutional layers through attention feature maps.

2. The medical image segmentation method based on adaptive scale as described in claim 1, characterized in that, The parallel convolutional layer module can be configured with parallel convolutional layers with different dilation levels to extract image features at different scales. Its main steps include: Given a feature map extracted by a backbone CNN P As input, image features at different scales are extracted through a parallel convolutional layer with different dilation levels, as defined below: in, Indicated by Feature extraction of the image is performed to determine the radius of expansion. K is an adjustable parameter, used in experiments. Through different expansion radii Image features are extracted, and the resulting feature maps have different receptive fields. The receptive field of each layer is calculated as follows: in e The kernel size is [size]. s Step size, The receptive field during input.

3. The medical image segmentation method based on adaptive scale as described in claim 1, characterized in that, The multi-scale fusion module merges feature maps from different scales of parallel convolutional layers, aggregating multi-scale feature maps using the following formula: in, Represents the parallel convolutional layer module. i The output of the layer, For the attention module i One channel, This represents element-wise multiplication, the final result. F The resulting multi-scale feature map has varying degrees of dependence on different scales, and the network can learn an attention coefficient related to scale through local and global features.

Images