A medical image segmentation system and a general and efficient medical image segmentation method

Abstract

The present invention discloses a medical image segmentation system and a general-purpose efficient medical image segmentation method. The medical image segmentation system includes a multi-layer encoder and a multi-layer decoder. Based on this system, a general-purpose efficient medical image segmentation method is designed. The method achieves high-precision recognition and segmentation of different lesion areas by encoding the input medical image, aggregating semantic information, automatically learning to select and utilize useful features and suppress irrelevant features through selective kernel operations, and finally capturing favorable features through a dilated spatial convolutional pooling pyramid. The technical solution of the present invention makes full use of the semantic features in medical images for effective extraction, and adaptively selects different receptive fields for decoding according to the characteristics of different data, which can effectively improve the segmentation performance of medical images.

Description

Technical Field

[0001] The present invention belongs to the field of bioinformatics, and specifically relates to a medical image segmentation system and a universal and efficient medical image segmentation method. Specifically, it is a medical image segmentation system constructed by a multi-layer encoder and a multi-layer decoder, and a universal and efficient medical image segmentation method implemented on the system. Background Art

[0002] Medical image segmentation plays a crucial role in early disease detection and diagnosis and is of great significance in the medical image analysis process. In the past, doctors manually segmented and labeled target regions, but this traditional method was highly subjective and inefficient. Efficient computational methods enabled by machine learning provide stronger guidance for improving the efficiency and accuracy of medical image segmentation. This improves medical professionals' understanding of disease progression and facilitates tracking of treatment effectiveness, further enhancing the accuracy, efficiency, and consistency of medical image analysis. Currently, extensive research is devoted to developing segmentation systems and automated segmentation methods for various medical images, with significant results.

[0003] Over the past few decades, a large number of medical image segmentation methods have been developed. Currently, these architectures can be broadly categorized into two types: U-Net-based and Transformer-based. U-Net-based architectures primarily employ a U-shaped encoder-decoder structure and incorporate various skip connections to fuse semantic information at different scales, thereby improving the accuracy of medical image segmentation. However, these architectures employ static kernels, which, due to their size, cannot extract all meaningful features from medical image datasets with diverse spatial contexts. Furthermore, standard skip connections directly propagate high-resolution feature maps from the encoder to the decoder, resulting in inefficient fusion of semantic information. Transformer-based architectures have the inherent advantage of predicting global information, effectively modeling long-range dependencies. However, these models also have drawbacks. First, because the self-attention mechanism focuses on global information, they are less able to retain detailed information within the image. Second, these models require significant computational and storage overhead, making Transformer-based architectures difficult to implement in clinical diagnosis and treatment.

[0004] Currently, a large number of network architectures are used to study medical image segmentation, but few segmentation architectures can simultaneously address the above-mentioned issues. Therefore, a key challenge for accurate medical image recognition and segmentation is to consider the different semantic features of different data, improve the above-mentioned deficiencies, extract all meaningful feature information from medical imaging datasets with different spatial backgrounds while retaining the structural and detail information in the images with low computational and storage overhead. In other words, to explore network architectures that can segment medical images efficiently, cost-effectively, and with high precision. Summary of the Invention

[0005] To overcome the above-mentioned deficiencies of the prior art, the present invention provides a medical image segmentation system and a general and efficient medical image segmentation method.

[0006] The medical image segmentation system of the present invention comprises a multi-layer encoder and a multi-layer decoder. The multi-layer encoder and the multi-layer decoder have the same number of layers and are cascaded in a multi-scale jump connection manner. The multi-layer encoder adopts a dual-branch structure. The first branch has three layers of encoders. The outputs of the encoders of each layer are respectively represented by symbols. Indicates that the second branch has n layers of encoders, and the outputs of each layer of encoders are respectively expressed as Said multi-layer decoder has n layers of decoders, and the output of each layer of decoder is represented by express.

[0007] In the image segmentation system of the present invention, a general and efficient medical image segmentation method is designed. The specific steps are as follows:

[0008] Step 1: The input medical image is encoded through a novel residual connection operation and a downsampling operation implemented by convolution, and the semantic information output of the first layer of the dual-branch encoder is obtained. and represents the output of the first layer of the first branch, Represents the output of the first layer of the second branch. The two are equal and can be expressed in a unified form

[0009] Step 2: Output the semantic information of the first layer encoder of the second branch in step 1 The output of the second layer encoder is obtained by the downsampling operation achieved through the novel residual connection operation and convolution Then you will get the output Through similar operations as above, the output of the third layer encoder is obtained Execute in sequence until the output of the n-th layer encoder is calculated

[0010] Step 3: Output the semantic information of the first layer of the first branch in step 1 The downsampling operation is achieved by a novel residual connection operation and convolution. Size and number of channels are related to the output of the n-1th layer encoder of the second branch consistent;

[0011] Step 4: Take the data obtained in step 3 and Obtained through selective kernel operations The purpose is to select and utilize more effective feature information;

[0012] Step 5: Replace the The three are respectively unified by bilinear interpolation operation, non-operation, and non-overlapping maximum pooling operation to achieve the unification of the three dimensions, and then perform splicing operation, and then sequentially pass convolution, batch normalization layer and nonlinear rectified linear unit activation function to obtain the semantic information after feature aggregation, and then with At the same time, a gating signal is generated through the soft attention module to control the importance of features at different spatial positions. Then, through the selective kernel operation, it automatically learns to select and utilize useful features and suppress irrelevant features to obtain the output of the penultimate layer decoder.

[0013] Step 6: Replace the and The three layers are unified by bilinear interpolation, non-operation, and non-overlapping maximum pooling operations, and then the splicing operation is performed. The semantic information after feature aggregation is obtained by convolution, batch normalization layer, and nonlinear rectified linear unit activation function. After that, the selective kernel operation is used to automatically learn to select and utilize useful features and suppress irrelevant features to obtain the output of the nk-th layer decoder.

[0014] Step 7: Let k = k + 1, and repeat step 6 until k = n - 1, and get the output of the 2nd ... n - 2nd layer decoder in turn

[0015] Step 8: Transform the output of the second layer decoder into Through bilinear interpolation operation, and then combined with the semantic information output in step 1 The two perform a splicing operation, and then sequentially pass through the convolution, batch normalization layer and nonlinear rectified linear unit activation function to obtain the semantic information after feature aggregation, which is the output of the first layer decoder.

[0016] Step 9: Transform the output of the first layer decoder in step 8 into Through the void space convolution pooling pyramid, the favorable features are finally captured to achieve the segmentation of medical images.

[0017] Specifically, the steps of the novel residual connection operation method described in the above steps are as follows:

[0018] (a) The initial input data is sequentially subjected to two sets of convolution operations with a kernel size of 3*3, a batch normalization layer, and a nonlinear rectified linear unit activation function to obtain the output;

[0019] (b) performing a residual connection on the output of step (a) and the output of the initial input data after a 3*3 convolution operation, so that the feature map of the initial input data is added to the output of step a to obtain a merged feature map;

[0020] (c) The feature map merged in step (b) is recalibrated through the squeeze and excitation module and passed to the next layer decoder block.

[0021] The present invention has the following beneficial effects: a medical image segmentation system and a general and efficient medical image segmentation method of the present invention can extract all meaningful feature information from medical imaging data sets with different spatial backgrounds under conditions of low computational complexity and storage overhead, and at the same time retain the structural information and detail information in the image, thereby realizing low-cost and high-precision medical image segmentation of deep learning networks under medical image data with different feature information. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a schematic diagram of the structural framework of the present invention.

[0023] Figure 2 It is a structural diagram of the novel residual connection operation method in the present invention.

[0024] Figure 3 This is a visualization comparison of the segmentation results of the present invention on the aneurysm dataset.

[0025] Figure 4 This is a visualization comparison of the segmentation results of the present invention on the liver dataset. DETAILED DESCRIPTION

[0026] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments. Those skilled in the art will readily appreciate the other advantages and benefits of the present invention from the contents disclosed herein. The present invention may also be implemented or applied through different specific embodiments, and the various details in this specification may be modified or altered based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments may be combined with each other unless they conflict.

[0027] Example 1:

[0028] Medical image segmentation requires more accurate extraction of the semantic feature information of the input data. Therefore, a dual-branch encoder can extract more comprehensive feature information than a single-branch encoder. Therefore, the present invention selects a dual-branch encoder encoding method with higher feature extraction performance. And through the concepts of detailed semantic information and overall semantic information, we have also found a new angle for the encoding method. In the present invention, we adopt a popular single-branch encoding strategy and add an encoding method that can pay more attention to the overall structure to improve the segmentation accuracy. According to current research, the source of medical image data that can be used for segmentation is not limited to this. Therefore, more sources are studied to obtain medical image data to verify the robustness of the strategy of the present invention.

[0029] like Figure 1 The figure below shows the framework of the present invention. The left half of the U-shaped structure consists of a dual-branch encoder. The first branch performs downsampling via a single large convolution, followed by a selective kernel module to generate feature representations. The second branch employs a popular layer-by-layer encoding strategy. Both branches extract semantic meanings from the raw data layer by layer, focusing on different aspects of the semantics to generate feature representations.

[0030] like Figure 2 As shown in the figure, each encoder layer uses a novel residual connection method. The input data is sequentially subjected to two sets of 3*3 convolution operations, a batch normalization layer, and a nonlinear rectified linear unit activation function. The resulting output is then residually connected with the output of the 3*3 convolution operation on the input data, so that the feature maps are added. The combined feature map is then recalibrated through the squeeze and excitation module and passed to the next encoder block, and so on until the deepest layer.

[0031] In a preferred embodiment of the present invention, the encoder is constructed in five layers.

[0032] Example 2:

[0033] Accurately segmenting medical images requires extracting and utilizing sufficient relevant semantic information. Therefore, in this paper, we employ multi-scale skip connections and feature aggregation to fully extract semantic information, and use adaptive receptive field selection decoding as the decoder. This allows the network to adaptively select and utilize information features captured by different receptive fields corresponding to different convolutional kernels that are effective for classification. In other words, it can selectively emphasize useful features and suppress irrelevant ones.

[0034] like Figure 1The figure shows the framework of the present invention. The dotted colored line in the U-shaped structure is a multi-scale jump connection, and the right half is an adaptive receptive field selection decoder. The multi-scale jump connection can be divided into two modes according to the layer where the decoder is located: the first layer decoder connects the output of the encoder of the same layer and the output of the decoder of the next layer; each layer decoder except the first layer decoder connects the output of the encoder of the same layer, the output of the encoder of the next layer, and the output of the decoder of the next layer. The specific implementation of these two modes can be achieved by distinguishing Figure 1 Seen in the dashed color.

[0035] The result of the multi-scale jump connection reaches the decoder through the feature aggregation operation. The specific implementation methods of the feature aggregation operation can be divided into three types according to the layer where the decoder is located: 1) The feature map of the encoder layer of the same scale directly passes the feature information to the target layer decoder, and then passes through the convolution, batch normalization layer and nonlinear rectified linear unit activation function in sequence; 2) The feature map of the shallow encoder passes the low-level detail information to the target layer decoder through non-overlapping maximum pooling operations, and then passes through the convolution, batch normalization layer and nonlinear rectified linear unit activation function in sequence; 3) The feature map of the deep decoder passes high-level semantic information to the target decoder layer through bilinear interpolation, and then passes through the convolution, batch normalization layer and nonlinear rectified linear unit activation function in sequence. Its calculation formula is

[0036]

[0037] Where H(X) = δ(LN(Conv(X))), Conv(·) represents a 3×3 convolution operation, LN(·) represents layer normalization, and δ(·) represents a nonlinear rectified linear unit activation function. Indicates i th layer encoder, Indicates i th Layer decoder, N represents the depth of the network, function C(·) represents the convolution operation, D(·) and U(·) represent the downsampling and upsampling operations respectively, and [·] represents the concatenation operation.

[0038] The specific implementation methods of the adaptive receptive field selection decoder can be divided into three types according to the layer where the decoder is located: 1) The feature map of the shallowest decoder is directly output after the aggregation operation without undergoing the adaptive receptive field selection decoding operation; 2) Before the feature map of the second deepest decoder enters the adaptive receptive field selection decoding operation, a soft attention module is used to generate a gating signal to adjust the semantic features of the decoder output by fusing the output of the first branch of the multi-scale skip connection and the dual-branch encoder, thereby controlling the importance of features at different spatial positions; 3) Other layers perform the adaptive receptive field selection decoding operation directly after the feature aggregation operation. The selective kernel block and attention gate ensure that meaningful features are used to segment the image. Its calculation formula is:

[0039]

[0040] in Indicates i th layer encoder, Indicates i th Layer decoder, N represents the depth of the network represents the output of the multi-scale skip connection, Ag(·) represents the attention gate operation, and SK(·) represents the selective kernel operation.

[0041] Finally, the output of the decoder block is passed through a dilated spatial convolutional pooling pyramid, where we apply 1×1 convolution and sigmoid activation to obtain the final segmentation map.

[0042] In a preferred embodiment of the present invention, the encoder or decoder is constructed in five layers.

[0043] Example 3:

[0044] Figure 3 This is a qualitative visual comparison of the model of the present invention and other models on the Aneurysm_data dataset. Figure 4 This is a qualitative visual comparison of the proposed model and other models on the LiTS dataset. Lesion areas are marked in white. A represents the original input medical image, B represents the manually segmented image, C to J represent the segmentation results of the U-Net model, Res U-Net model, U-Net++ model, Attention U-Net model, U-Net3+ model, Res U-Net++ model, UNETR model, and TranSEFusionNet model, respectively, and K represents the segmentation result of the proposed model.

[0045] It can be seen from the figure that compared with other models, the model of this method has better segmentation in details and does not segment the background into the target object, such as Figure 3 In H, Res U-Net++, the brain aneurysm is segmented into the background. Figure 4 The J, TranSEFusionNet model in the figure does not clearly segment the liver boundary, while the model of this method can correctly segment the lesion area. In summary, both quantitative and qualitative comparisons fully demonstrate the superiority of the present invention in medical image segmentation. By fusing features from different layers, the information of each layer of the network is fully utilized, which improves the final accuracy to a certain extent. Furthermore, the combination of the dual-branch encoder can more fully extract the semantic information of the image, and well handle the connection between the information, making the image segmentation results more accurate and smooth.

[0046] The embodiments described above are merely descriptions of preferred implementations of the present invention and are not intended to limit the scope of the present invention. Without departing from the spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should fall within the scope of protection of the present invention.

Claims

1. A general and efficient medical image segmentation method, characterized in that: The image segmentation method includes the following steps: Step 1: The input medical image is encoded through a novel residual connection operation and a downsampling operation implemented by convolution, and the semantic information output of the first layer of the dual-branch encoder is obtained. and , represents the output of the first layer of the first branch, Represents the output of the first layer of the second branch. The two are equal and expressed in a unified form ; Step 2: Output the semantic information of the first layer encoder of the second branch in step 1 The output of the second layer encoder is obtained by the downsampling operation achieved through the novel residual connection operation and convolution , then the output will be Through similar operations as above, the output of the third layer encoder is obtained , execute in sequence until the first The output of the layer encoder ; Step 3: Output the semantic information of the first layer of the first branch in step 1 The downsampling operation is achieved by a novel residual connection operation and convolution. , The size and number of channels are the same as those of the second branch. Layer encoder output consistent; Step 4: Take the data obtained in step 3 and Obtained through selective kernel operations ; Step 5: Replace the The three are respectively unified by bilinear interpolation operation, non-operation, and non-overlapping maximum pooling operation to achieve the unification of the three dimensions, and then perform splicing operation, and then sequentially pass convolution, batch normalization layer and nonlinear rectified linear unit activation function to obtain the semantic information after feature aggregation, and then with At the same time, a gating signal is generated through the soft attention module to control the importance of features at different spatial positions, and then the output of the penultimate layer decoder is obtained through selective kernel operation. ; Step 6: Replace the and The three are respectively unified by bilinear interpolation operation, non-operation, and non-overlapping maximum pooling operation to achieve the unification of the three dimensions, and then perform splicing operation, and then sequentially obtain the semantic information after feature aggregation through convolution, batch normalization layer and nonlinear rectified linear unit activation function, and then obtain the first Output of layer decoder ; Step 7: Make , repeat step 6 until , and get the Output of layer decoder ; Step 8: Transform the output of the second layer decoder into Through bilinear interpolation operation, and then combined with the semantic information in step 1 The two perform a splicing operation, and then sequentially pass through the convolution, batch normalization layer and nonlinear rectified linear unit activation function to obtain the semantic information after feature aggregation, which is the output of the first layer decoder. ; Step 9: Transform the output of the first layer decoder in step 8 into Through the void space convolution pooling pyramid, the favorable features are finally captured to achieve the segmentation of medical images; The steps of the new residual connection operation described in step 1 are as follows: Step a: The initial input data is sequentially subjected to two sets of convolution operations with a kernel size of 3*3, a batch normalization layer, and a nonlinear rectified linear unit activation function to obtain the output; Step b: Perform a residual connection on the output of step a and the output after the 3*3 convolution operation of the initial input data, so that the feature map of the initial input data is added to the output of step a to obtain a merged feature map; Step c: The feature map merged in step b is recalibrated through the squeeze and excitation module and passed to the next layer decoder block.

2. A medical image segmentation system for executing the method according to claim 1, characterized in that: The image segmentation system includes two parts: a multi-layer encoder and a multi-layer decoder. The multi-layer encoder and the multi-layer decoder have the same number of layers and are cascaded using a multi-scale jump connection. The multi-layer encoder adopts a dual-branch structure. The first branch has three layers of encoders. The outputs of the encoders in each layer are represented by symbols. , , Indicates that the second branch has Layer encoder, the output of each layer encoder is used Indicates that the multi-layer decoder has Layer decoder, the output of each layer decoder is used express.

Images