Medical image segmentation method

Abstract

The invention provides a medical image segmentation method. The method comprises the following steps: acquiring an original medical image; inputting the original medical image into a trained image segmentation model to obtain a final segmented image; the construction method of the trained image segmentation model comprises the following steps: constructing an initial network model; and training the initial network model by using an artificially marked medical gold standard image to obtain an image segmentation model. The problem that in the prior art, local details and global context information cannot be captured at the same time, and consequently segmented images are inaccurate is solved.

Description

Technical Field

[0001] This invention relates to the field of image segmentation technology, and in particular to a medical image segmentation method. Background Technology

[0002] Medical image segmentation is an important branch of medical image processing. It aims to accurately extract tissue structures or lesion areas of interest from medical image data, providing doctors with information for visualization, quantitative analysis, and diagnostic assistance. With the continuous advancement of medical imaging technology and the increasing level of digitization, medical image segmentation plays an increasingly important role in clinical diagnosis, treatment planning, and disease research.

[0003] Medical imaging data typically includes multiple modalities such as computed tomography (CT), magnetic resonance imaging (MRI), ultrasound imaging (US), and X-ray imaging. The high resolution, large information content, and complex dimensions of this data present challenges for medical image segmentation. In medical images, the differences in density, morphology, and grayscale characteristics of different tissue structures are often not significant, and the segmentation task is further complicated by factors such as noise, artifacts, and changes in the patient's anatomical structure.

[0004] In recent years, the development of deep learning technology has greatly boosted medical image segmentation. Deep learning models such as Convolutional Neural Networks (CNNs) have achieved remarkable results in image segmentation tasks. By utilizing feature representations learned by deep learning models, medical image segmentation algorithms can better adapt to different data characteristics and achieve higher accuracy and robustness. Commonly used deep learning models in the field of medical image segmentation include U-Net, FCN, SegNet, and DeepLab. These models have been widely applied in medical image segmentation tasks and are constantly being improved and optimized. For example, the U-Net structure alleviates the problem of information loss by introducing skip connections, improving segmentation performance; the DeepLab model increases the receptive field by introducing dilated convolution, improving the accuracy of segmenting large-scale structures. Besides deep learning models, traditional image segmentation methods also play an important role in medical image segmentation. For example, threshold-based segmentation, region growing, and edge detection methods are still widely used in medical image segmentation tasks and have certain advantages in some situations. However, current technologies still cannot simultaneously capture local details and global contextual information, leading to inaccurate image segmentation. Summary of the Invention

[0005] To overcome the shortcomings of existing technologies, the purpose of this invention is to provide a medical image segmentation method. This invention solves the problem in the prior art that the segmentation of images is inaccurate because local details and global context information cannot be captured simultaneously.

[0006] To achieve the above objectives, the present invention provides the following solution:

[0007] A medical image segmentation method, comprising:

[0008] Acquire raw medical images;

[0009] The original medical image is input into the trained image segmentation model to obtain the final segmented image;

[0010] The method for constructing the trained image segmentation model is as follows:

[0011] An initial network model is constructed, comprising: a convolutional coding module, a linear mapping layer, a multi-level coding module, a convolutional decoding module, and a classifier connected sequentially. The multi-level coding module includes multiple encoders and multi-level attention units. The convolutional coding module is connected to the linear mapping layer, which is connected to the first encoder in the multi-level coding module. The multi-level attention units are connected to the first encoder and the next encoder, respectively. The last multi-level attention unit in the multi-level coding module is connected to the convolutional decoding module. The convolutional coding module converts the original medical image into a multi-level feature map. The encoder encodes the multi-level features. The multi-level attention units fuse the current level features to obtain a fusion result. The convolutional decoding module obtains an initial segmentation image based on the fusion result. The classifier obtains a final segmentation image based on the initial segmentation image.

[0012] The initial network model was trained using artificially labeled medical gold standard images to obtain an image segmentation model.

[0013] Preferably, the multi-level attention unit is used to fuse the current level features to obtain a fusion result, including:

[0014] The current output feature maps are concatenated together to obtain the concatenated feature map;

[0015] Multiple 3x3 convolutional layers and fully connected layers are used to process the stitched feature map and generate an attention score for each feature channel.

[0016] Apply a softmax operation to the attention scores to obtain the weights for each feature map.

[0017] The original feature maps are weighted according to the weight of each feature map, and then summed to obtain the fusion result.

[0018] Preferably, the expression for the multi-level attention unit is:

[0019]

[0020] Where C represents context or task-related information, f(·) is a function used to adjust attention weights based on context information, and d k Q represents the dimension of the key vector, where Q is the query, K is the key, and V is the value.

[0021] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0022] This invention provides a medical image segmentation method, comprising: acquiring an original medical image; inputting the original medical image into a trained image segmentation model to obtain a final segmented image; the method for constructing the trained image segmentation model is as follows: constructing an initial network model; training the initial network model using artificially labeled medical gold standard images to obtain an image segmentation model. This invention can simultaneously capture local details and global contextual information of an image, which is crucial for handling complex medical image segmentation tasks and improves accuracy and universality. In clinical applications, this can greatly improve the accuracy and efficiency of image recognition and has significant value in guiding treatment plans and monitoring disease progression. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 A flowchart of a medical image segmentation method provided in an embodiment of the present invention;

[0025] Figure 2 This is an initial network model architecture diagram provided for an embodiment of the present invention;

[0026] Figure 3 This is a representative schematic diagram of deep learning-based human body component segmentation provided in an embodiment of the present invention, wherein, Figure 3 (a) is a schematic diagram of the first human body component segmentation. Figure 3 (b) is a schematic diagram of the second human body component segmentation. Figure 3 (c) is a schematic diagram of the third human body component segmentation. Detailed Implementation

[0027] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0028] The purpose of this invention is to provide a medical image segmentation method. This invention solves the problem in the prior art that the segmentation of images is inaccurate because local details and global context information cannot be captured simultaneously.

[0029] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0030] like Figure 1 As shown, the present invention provides a medical image segmentation method, comprising:

[0031] Step 100: Acquire the raw medical image;

[0032] Step 200: Input the original medical image into the trained image segmentation model to obtain the final segmented image;

[0033] The method for constructing the trained image segmentation model is as follows:

[0034] Step 201: Construct an initial network model; the initial network model includes: a convolutional coding module, a linear mapping layer, a multi-level coding module, a convolutional decoding module, and a classifier connected in sequence. The multi-level coding module includes: multiple encoders and multi-level attention units; the convolutional coding module is connected to the linear mapping layer, the linear mapping layer is connected to the first encoder in the multi-level coding module, the multi-level attention units are connected to the first encoder and the next encoder respectively, and the last multi-level attention unit in the multi-level coding module is connected to the convolutional decoding module. The convolutional coding module is used to convert the original medical image into a multi-level feature map. The encoder is used to encode the multi-level features. The multi-level attention units are used to fuse the current level features to obtain a fusion result. The convolutional decoding module is used to obtain an initial segmentation image based on the fusion result. The classifier obtains the final segmentation image based on the initial segmentation image.

[0035] Step 202: Train the initial network model using artificially labeled medical gold standard images to obtain an image segmentation model.

[0036] Specifically, given the differences between medical CT images and traditional natural images, we propose a visual hybrid architecture that combines the advantages of ConvNet (Convolutional Neural Network) and Transformer components. This architecture considers reconstructing images at different frequencies through multiple encoders and focusing attention on different parts through multiple channels. Sampling from CT images at different cone levels, we first implement frequency filtering of the original image using multi-kernel convolution. Similar to the pyramid structure in object detection, this multi-layer feature construction enables us to learn many features that are difficult for single-layer networks to notice, combining the spatial feature extraction capabilities of ConvNet with the ability of Transformer to capture long-range dependencies in sequence modeling.

[0037] The entire network follows an autoregressive encoder-decoder architecture. ① In the encoding stage, the network input is raw medical images, such as CT or MRI scans. These images are first processed by the encoder (the conv encoder module in the diagram), which typically consists of multiple convolutional layers capable of extracting features at different levels (low, medium, and high).

[0038] In this process, such as Figure 2 As shown, different levels of the encoder correspond to different feature levels of the image. For example: 1. Low-level features: Low-level features usually contain some local, detailed information, such as edges and textures. These features correspond to the basic structure and subtle changes of the image and are extracted by the shallower convolutional layers in the encoder.

[0039] 2. Mid-level features: Mid-level features encompass higher-level semantic information, such as shape and partial structure. These features are extracted by the intermediate convolutional layers in the encoder, and they correspond to local patterns and object parts in the image.

[0040] 3. High-level features: High-level features contain more abstract and semantic information, representing the overall structure, semantics, and context of the image. These features are typically extracted by deep convolutional layers in the encoder, and they correspond to high-level concepts such as object categories and scene information in the image.

[0041] Specifically, the mathematical expression for the BCEWithLogits loss used to optimize the model is as follows:

[0042]

[0043] Layer normalization is a standardization operation aimed at reducing internal covariate shift. Simply put, it involves standardizing the input data within each layer of the model, ensuring a mean of 0 and a variance of 1. This improves model training stability and accelerates convergence. In Transformer models, layer normalization is typically performed after self-attention and feedforward network modules.

[0044] A feedforward network (FFN) is a component of the Transformer model. It consists of two linear transformations with a non-linear activation function (such as ReLU) in between. The purpose of the FFN is to perform another transformation on the information processed by self-attention, giving the model the ability to learn more complex functions. Each encoder layer of the Transformer contains an FFN, which applies the same transformation independently to the representation at each location.

[0045] In the decoder stage, the mask attention mechanism fully demonstrates the network's flexibility and adaptability. This mechanism allows the network to switch freely between different mask configurations, whether some masks are present, all masks are present, or no masks are present. This design not only improves the model's ability to handle different scenarios but also further enhances the model's ability to understand and integrate information.

[0046] Furthermore, the masked attention mechanism endows the network with adaptive learning capabilities. The model can automatically adjust its focus based on different inputs and task requirements; this flexibility is key to achieving efficient learning and generalization. Through this mechanism, the model can not only work stably in complex and ever-changing data environments but also quickly adapt to new tasks and challenges.

[0047] The process of obtaining hierarchical features involves transforming the input image into a series of embedding representations at different levels to capture visual information from low to high. Below is an optimized flowchart illustrating this transformation using the OverlapPatchEmbed function:

[0048] Assuming the input image Where B is the batch size, C is the number of channels, and H0 and W0 are the height and width of the image, respectively.

[0049] First, the image X is transformed into an embedded feature map X' through a convolutional layer nn.Conv2d with padding:

[0050]

[0051] Conv2dproj This represents a convolution operation with parameters including input channels C, output embedding dimension D, kernel size of patch_size × patch_size, stride, and appropriate padding to maintain overlap. The result is a feature map of dimension D, where H' and W' are the height and width of the output feature map.

[0052] By adjusting the kernel size and stride of the convolutional layer, we can adjust the relationships between image hierarchical information and obtain the corresponding inputs for modules at different levels:

[0053] Low-Level: patch_size=7, stride=4, captures low-level detail information.

[0054] Mid-Level: patch_size=5, stride=3, captures mid-level features.

[0055] High-Level: Two settings, patch_size=3 and stride=2, are used to capture high-level abstract information. Two high-level layers are used because a single-layer network may not be able to learn the diversity of information; the complementarity of the two layers further enriches the corresponding embedded content.

[0056] Through the optimization process described above, information from the input image at different levels can be effectively converted into embedding vectors. These embedding vectors can then be used in subsequent processing steps of the model, such as feature fusion, classification, or other visual tasks.

[0057] These features are then fed into multiple Transformer encoding blocks (various multi-level encoder modules in the figure). The Transformer structure enables the network to capture the complex interactions and relationships between different parts of an image, which is crucial for understanding biological structures and pathological regions in images.

[0058] In this section, we employ the Adaptive Attention method: Adaptive attention optimizes the allocation of attention weights through a dynamic adjustment process, allowing the model to flexibly adjust its focus based on task requirements and the characteristics of the input data. In adaptive attention, the computation process involves not only the generation of queries, keys, and values, but also an adaptive mechanism, which can be an additional network layer or a module interacting with external signals. This mechanism influences how attention weights are calculated, enabling the model to focus more intently on the information most critical to the current task. A possible formulation of adaptive attention (an expression for multi-level attention units) is as follows:

[0059]

[0060] Where C represents context or task-related information, f(·) is a function used to adjust attention weights based on context information, and d k The dimension remains the same as the key vector, used to scale the dot product. This formula dynamically adjusts the attention weights by adding contextual information C, making the attention allocation more adaptable to the current input and task requirements.

[0061] In EncoderBlock, through a series of carefully designed steps, the model can effectively process and transform input information. Adaptive attention provides the ability to capture intra-sequence dependencies, MLP enhances the model's processing power, residual connections ensure effective information propagation, and regularization techniques improve the model's stability and generalization ability. The combined application of these techniques enables the encoder to process and transform input information more effectively.

[0062] Next, the feature attention mechanism aggregates key information from different levels, enabling the network to focus on the most important features in the final task. In medical image segmentation, this means accurately labeling pathological regions in the image. Furthermore, the embeddings produced by each encoder are fed into the same convolutional decoder, allowing full utilization of the information generated by the encoders (multi-level feature attention modules following multi-level encoders in the diagram).

[0063] The image is processed through multiple encoder stages (each containing several encoder blocks), with each stage processing feature maps at different resolutions. At the end of each stage, a FeatureAttention module is used to combine feature representations of different depths within the current stage. This is achieved by performing a weighted summation of the outputs of multiple encoder blocks within the same stage, where the weights are learned through the FeatureAttention mechanism.

[0064] Furthermore, the multi-level attention unit is used to fuse the current level features to obtain a fusion result, including:

[0065] The current output feature maps are concatenated together to obtain the concatenated feature map;

[0066] Multiple 3x3 convolutional layers and fully connected layers are used to process the stitched feature map and generate an attention score for each feature channel.

[0067] Apply a softmax operation to the attention scores to obtain the weights for each feature map.

[0068] The original feature maps are weighted according to the weight of each feature map, and then summed to obtain the fusion result.

[0069] Specifically, the FeatureAttention module's role is to fuse the outputs of different encoder blocks in a given stage. The specific process is as follows:

[0070] Feature concatenation: First, the output feature maps of all encoderblocks in the current stage are concatenated together to form a feature map with a larger number of channels.

[0071] Attention score calculation: The concatenated feature map is processed using multiple 3x3 convolutional layers and fully connected layers to generate an attention score for each feature channel.

[0072] Softmax normalization: Apply a softmax operation to the attention scores to obtain the weights of each feature map.

[0073] Weighted feature maps: The original feature maps are weighted according to the weights obtained from softmax, and then these weighted feature maps are summed to obtain the fused feature representation.

[0074] In each stage, the FeatureAttention module follows multiple encoder blocks. It first fuses the outputs of these blocks, then passes the fused feature map as input to the first encoder block of the next stage. In this way, the model can effectively integrate information at different scales while maintaining feature diversity. This structure allows each stage to not only capture feature information at its current scale but also integrate features from different levels in the previous stage through the feature attention mechanism, thereby capturing image details and contextual information at multiple scales.

[0075] The task of the decoding stage is to remap the abstract features from the encoding stage back to the original pixel space and generate a segmentation map that matches the size of the original image (the decoding conv module in the figure).

[0076] Finally, the image is processed by a classifier, which typically uses a softmax function to assign a label to each pixel, representing the probability that the pixel belongs to a specific class. (See the classifier and logits attention modules in the diagram.)

[0077] The purpose of the decoding head is to fuse the output multi-scale feature maps and map them to the target class space, ultimately generating a class prediction for each pixel. The main decoding conv module uses multiple 3x3 convolutions to reduce the number of channels in the fused feature map to `embedding_dim`. The logits attention module is implemented through multiple convolutional layers, where the number of input and output channels equals the number of classes `num_classes`, and the kernel size is set to 64x64 with a stride of 64. This means that for each input feature map, this convolutional layer generates a separate weight value, which is applied to the corresponding feature map channel. This mechanism allows the model to adjust the importance of each class channel based on global context information, thus assigning different levels of attention to different semantic classes. This method can be viewed as an attention mechanism that focuses on the channel dimension of the feature map, using weighted adjustments to enhance or suppress feature representations of certain specific classes.

[0078] The final visualization result of the model is as follows Figure 3 As shown, the original image (scan) is followed by images of subcutaneous adipose tissue (SAT), muscle, and visceral adipose tissue (VAT).

[0079] The beneficial effects of this invention are as follows:

[0080] This invention provides a medical image segmentation method, comprising: acquiring an original medical image; inputting the original medical image into a trained image segmentation model to obtain a final segmented image; the method for constructing the trained image segmentation model is as follows: constructing an initial network model; training the initial network model using artificially labeled medical gold standard images to obtain an image segmentation model. This invention can simultaneously capture local details and global contextual information of an image, which is crucial for handling complex medical image segmentation tasks and improves accuracy and universality. In clinical applications, this can greatly improve the accuracy and efficiency of image recognition and has significant value in guiding treatment plans and monitoring disease progression.

[0081] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0082] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A medical image segmentation method, characterized in that, include: Acquire raw medical images; The original medical image is input into the trained image segmentation model to obtain the final segmented image; The method for constructing the trained image segmentation model is as follows: Construct the initial network model; The initial network model includes: a convolutional coding module, a linear mapping layer, a multi-level coding module, a convolutional decoding module, and a classifier connected in sequence. The multi-level coding module includes: multiple encoders and multi-level attention units. The convolutional coding module is connected to the linear mapping layer, the linear mapping layer is connected to the first encoder in the multi-level coding module, the multi-level attention units are connected to the first encoder and the next encoder respectively, and the last multi-level attention unit in the multi-level coding module is connected to the convolutional decoding module. The convolutional coding module is used to convert the original medical image into a multi-level feature map. The encoder is used to encode the multi-level features. The multi-level attention units are used to fuse the current level features to obtain a fusion result. The convolutional decoding module is used to obtain an initial segmentation image based on the fusion result. The classifier obtains a final segmentation image based on the initial segmentation image. The initial network model was trained using artificially labeled medical gold standard images to obtain an image segmentation model.

2. The medical image segmentation method according to claim 1, characterized in that, The multi-level attention unit is used to fuse the current level features to obtain the fusion result, including: The current output feature maps are concatenated together to obtain the concatenated feature map; Multiple 3x3 convolutional layers and fully connected layers are used to process the stitched feature map and generate an attention score for each feature channel. Apply a softmax operation to the attention scores to obtain the weights for each feature map. The original feature maps are weighted according to the weight of each feature map, and then summed to obtain the fusion result.

3. The medical image segmentation method according to claim 1, characterized in that, The expression for the multi-level attention unit is: Where C represents context or task-related information, f(·) is a function used to adjust attention weights based on context information, and d k Q represents the dimension of the key vector, where Q is the query, K is the key, and V is the value.

Images