A medical image segmentation method based on edge-guided attention mechanism
By introducing edge-guided attention mechanism and multi-scale feature map fusion module in deep learning models, the problem of poor boundary feature capture when segmenting colorectal cancer osteogenic metastasis tumors is solved, achieving higher segmentation accuracy and detail retention.
Patent Information
- Application Number
- CN202411846115.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-16
- Publication Date
- 2025-04-08
- Estimated Expiration
- 2044-12-16
AI Technical Summary
When existing deep learning methods segment the osteogenetic metastasis of colorectal cancer, it is difficult to capture the boundary characteristics of small targets, resulting in low segmentation accuracy.
A multi-scale image segmentation model based on edge-guided attention mechanism is adopted, and the model's ability to capture edge details is enhanced by introducing edge-guided attention modules and multi-scale feature map fusion modules in the U-Net network.
It improves the effectiveness of accurate identification of osteogenic metastasis of colorectal cancer and boundary morphology segmentation, and improves the accuracy and detail retention ability of segmentation results.
Smart Images
Figure CN119295497B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of deep learning and image processing, and specifically relates to a medical image segmentation method based on an edge-guided attention mechanism. Background Art
[0002] In the field of medical image segmentation, there are mainly two types of image segmentation methods: traditional image segmentation methods and deep learning methods. Traditional image segmentation methods use shallow features such as grayscale and texture in medical images to complete segmentation tasks; commonly used methods include threshold method, region growing method, etc.; simply using shallow information in medical images does not produce good segmentation results. Deep learning methods can achieve end-to-end learning, and can automatically extract more abstract features during the training process, greatly improving segmentation accuracy.
[0003] The most widely used deep learning segmentation method is the U-Net network, which can achieve efficient segmentation with fewer training samples. Later, networks such as ResUNet and UNet++, which are improved based on the U-Net architecture, have shown good results in medical segmentation. However, these segmentation networks cannot capture effective features well for complex medical scenes such as colorectal cancer bone metastases with small targets and fuzzy boundaries. There are problems such as inaccurate boundary prediction and low segmentation accuracy.
[0004] Colorectal cancer (CRC) is one of the most common malignant tumors in the world. Distal metastasis of colorectal cancer most often occurs in internal organs and bone sites such as the thoracic and lumbar vertebrae. Colorectal cancer bone metastasis includes osteolytic bone metastasis and osteoblastic bone metastasis. Osteoblastic metastasis usually manifests as local increased bone density and bone sclerosis on CT (computed tomography) images, while osteolytic metastasis manifests as bone destruction, dissolution and decreased bone density. Osteoblastic metastasis is common in colorectal cancer. Accurate identification and segmentation of osteoblastic metastases of colorectal cancer can help doctors better understand the progression of the disease and develop more accurate treatment plans.
[0005] The segmentation results of existing deep learning methods are usually better than those of traditional image processing methods, but they are still insufficient for specific segmentation such as colorectal cancer bone metastases. Colorectal cancer bone metastases are small targets in the entire CT image and have blurred boundaries, variable positions, and complex backgrounds, making it difficult to achieve accurate segmentation. Summary of the invention
[0006] The object of the present invention is to provide a medical image segmentation method based on edge-guided attention mechanism.
[0007] In a first aspect, the present invention provides a medical image segmentation method based on an edge-guided attention mechanism, which comprises the following steps:
[0008] Step 1: Construct a data set; the data set includes the original medical image and the labels of the regions of interest in the original medical image;
[0009] Step 2: construct a multi-scale edge-guided attention image segmentation model; the multi-scale edge-guided attention image segmentation model uses a U-Net network as a basic framework, and the U-Net network includes an encoder and a decoder;
[0010] The encoder includes multiple encoding layers, and the decoder includes multiple decoding layers; the encoding layers and decoding layers at the same layer are connected by skip connections, and the first four encoding layers and the corresponding decoding layers are connected in series with an edge-guided attention module on the skip connections;
[0011] The output of the last encoding layer is fused with the output of each decoding layer through a multi-scale feature map fusion module to obtain the output of the multi-scale edge-guided attention image segmentation model;
[0012] Step 3: Use the dataset to train a multi-scale edge-guided attention image segmentation model;
[0013] Step 4: Use the trained model to segment the medical image under test.
[0014] Preferably, in the encoder, all encoding layers except the last encoding layer input their output feature maps to the next encoding layer after downsampling;
[0015] In the decoder, the input of each decoding layer is the concatenation of the output feature map of the previous decoding layer after upsampling and the output feature map of the same encoding layer after edge-guided attention module processing; the outputs of each decoding layer and the different-scale feature maps of the last encoding layer are convolved through multiple convolutional layers, and the sizes of the feature maps output by multiple convolutional layers are kept consistent with the size of the original medical image through bilinear interpolation and input into the multi-scale feature map fusion module; the number of convolutional layers is the same as the number of encoding layers.
[0016] Preferably, both the encoding layer and the decoding layer in the multi-scale edge-guided attention image segmentation model adopt feature extraction modules;
[0017] The feature extraction module uses the U-Net network as the basic framework, including multiple sub-coding layers and multiple sub-decoding layers; direct jump connections are used between the sub-coding layers and sub-decoding layers in the same layer; the input of each sub-decoding layer is the concatenation of the output feature map of the previous sub-decoding layer and the output feature map of the same sub-coding layer.
[0018] Preferably, the sub-coding layer and the sub-decoding layer in the feature extraction module have the same structure, both comprising a dilated convolution layer, a batch normalization layer and a ReLU activation function connected in sequence.
[0019] Preferably, the multi-scale feature map fusion module includes a splicing layer, a convolution layer and a Sigmoid activation function; wherein, the feature maps output by the multiple convolution layers are spliced along the channel through the splicing layer, and the spliced feature maps are sequentially passed through the convolution layer and the Sigmoid activation function to obtain a prediction probability map.
[0020] Preferably, the edge-guided attention module comprises a first branch, a second branch, and a feature fusion block;
[0021] The first branch specifically multiplies the input feature map by the result of processing the input feature map with the maximum pooling operation, the first convolution, the ReLU activation function, the second convolution and the Sigmoid activation function in sequence, to obtain the first output feature map;
[0022] The second branch specifically multiplies the input feature map by the result of processing the input feature map by the maximum pooling operation, convolution and Sigmoid activation function in sequence, to obtain a second output feature map;
[0023] The feature fusion block specifically adds the first output feature map, the second output feature map and the original input feature map to obtain a preliminary fused feature map, and then sequentially performs edge detection, batch normalization layer, ReLU activation function and convolution operation on the preliminary fused feature map before adding and fusing it with the preliminary fused feature map.
[0024] Preferably, in step 3, the expression of the total loss L of the multi-scale edge-guided attention image segmentation model is:
[0025]
[0026] in, It is the loss between the feature map and the label of the input multi-scale feature map fusion module; The loss between the feature map output by the multi-scale edge-guided attention image segmentation model and the label; and The losses and loss The weight of Indicates the number of coding layers in the encoder.
[0027] Preferably, the medical image is a computed tomography image.
[0028] In a second aspect, the present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the memory stores the computer program; the processor executes the above-mentioned medical image segmentation method based on the edge-guided attention mechanism.
[0029] In a third aspect, the present invention provides a readable storage medium storing a computer program; when the computer program is executed by a processor, it is used to implement the above-mentioned medical image segmentation method based on the edge-guided attention mechanism.
[0030] The present invention has the following beneficial effects:
[0031] 1. The present invention uses a multi-scale edge-guided attention image segmentation model to segment medical images. By introducing the edge-guided attention mechanism into the multi-scale edge-guided attention image segmentation model, compared with the existing methods, the present invention can retain more edge details when segmenting images with small targets and blurred boundaries, thereby improving the effectiveness of the multi-scale edge-guided attention image segmentation model in accurately identifying bone metastases of colorectal cancer and segmenting boundary morphology.
[0032] 2. The present invention solves the problem that the existing methods do not fully consider the correlation between features and effectively screen important features in the process of segmenting colorectal cancer osteogenic metastases through a multi-scale edge-guided attention image segmentation model, thereby improving the accuracy of image segmentation. BRIEF DESCRIPTION OF THE DRAWINGS
[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without creative work.
[0034] Figure 1 It is the overall flow chart of the present invention.
[0035] Figure 2 Schematic diagram of the structure of the multi-scale edge-guided attention image segmentation model in the present invention.
[0036] Figure 3 It is a schematic diagram of the structure of the feature extraction module in the present invention.
[0037] Figure 4 Schematic diagram of the structure of the edge-guided attention module in the present invention.
[0038] Figure 5 This is a visual comparison diagram of the segmentation results of the present invention and the existing image segmentation model. DETAILED DESCRIPTION
[0039] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. The present invention is further described below in conjunction with the drawings.
[0040] like Figure 1 As shown, a medical image segmentation method based on edge-guided attention mechanism includes the following steps:
[0041] Step 1: Get the dataset
[0042] 1-1. Obtain a CT image of colorectal cancer bone metastasis, and use the doctor's manual outline file of the ROI area (region of interest) of the colorectal cancer bone metastasis as a label;
[0043] 1-2. Add window according to the HU value (Houinck unit) distribution histogram of the CT image; the original CT value distribution range without window adjustment is wide, which will lead to poor image contrast and difficulty in identifying the region of interest. Since colorectal cancer osteogenic metastases are distributed inside the bone or at the edge of the bone, the overall CT value distribution range is large. Combined with the CT value distribution range of colorectal cancer osteogenic metastases and the CT value distribution range of bones, the window width is set to [-1000,1000];
[0044] 1-3. Perform normalization on the windowed CT image data; the value range of the bone metastasis CT image after normalization becomes between 0 and 1;
[0045] 1-4. Slice the CT three-dimensional image data along the Z axis, retain the slices containing the bone metastasis of colorectal cancer, and then splice the retained slices together along the Z axis; use the spliced images as the original medical images to construct a data set, and divide the data set into a training set and a test set at an 8:2 ratio.
[0046] Step 2: Build a multi-scale edge-guided attention image segmentation model
[0047] like Figure 2As shown in the figure, the multi-scale edge-guided attention image segmentation model (MEDANet) takes the U-Net network as the basic framework, which includes an encoder, a decoder, an edge-guided attention module (EDAM) and a multi-scale feature map fusion module; the encoder includes six encoding layers En01, En02, En03, En04, En05 and En06 connected in sequence; each encoding layer reduces the size of the output feature map by half through maximum pooling downsampling and then inputs it into the next encoding layer; the decoder includes five decoding layers De05, De04, De03, De02, De01 connected in sequence and six convolution layers (the convolution kernel is 3×3); each decoding layer enlarges the size of the output feature map by two times through bilinear interpolation upsampling and then inputs it into the next decoding layer. The feature maps output by the encoding layers En01, En02, En03 and En04 are processed by the edge-guided attention module, and then concatenated with the results obtained by bilinear interpolation upsampling of the decoding layers De02, De03, De04 and De05, and the feature maps obtained after concatenation are used as the input of the next decoding layer; the feature maps obtained by concatenating the result of upsampling the output of the encoding layer En06 with the encoding layer En05 are used as the input of the decoding layer De05; the feature maps of different scales output by each decoding layer and the last encoding layer are convoluted through six convolutional layers, and the sizes of the feature maps output by the six convolutional layers are kept consistent with the size of the original medical image through bilinear interpolation and input into the multi-scale feature map fusion module. The feature extraction module (FEB) is used in both the encoding layer and the decoding layer.
[0048] like Figure 3 As shown in the figure, the feature extraction module uses the U-Net network as the basic framework, which includes five sub-encoding layers and three sub-decoding layers; the feature map after the output of the fourth and fifth sub-encoding layers is concatenated as the input of the first sub-decoding layer; the feature map after the output of the second and third sub-encoding layers is concatenated with the output of the first and second sub-decoding layers respectively, is used as the input of the second and third sub-decoding layers; the feature map after the output of the first sub-encoding layer is added to the output of the third sub-decoding layer is used as the output of the feature extraction module. Both the sub-encoding layer and the sub-decoding layer include a sequentially connected dilated convolution layer (the convolution kernel is 3×3 and the step size is 1), a batch normalization layer (BN) and a ReLU activation function; the dilation coefficients used in the dilated convolution layer in the sub-encoding layer and the sub-decoding layer are 1, 1, 2, 3, 4, 3, 2, 1 respectively, so that the size of the feature map remains unchanged while the receptive field is increased.
[0049] like Figure 4As shown in the figure, the edge-guided attention module includes a first branch, a second branch and a feature fusion block; the first branch includes a maximum pooling layer, a first convolution layer (the convolution kernel size is 1×1), a ReLU activation function, a second convolution layer (the convolution kernel size is 1×1) and a Sigmoid activation function connected in sequence; the first branch is used to screen important features on the feature map channel. The first branch performs a global maximum pooling operation on each channel of the feature map input to the edge-guided attention module, and sends the obtained feature map to the first convolution layer for channel dimensionality reduction, and then sends it to the second convolution layer after passing through the ReLU activation function. The result is input into the Sigmoid activation function, and the feature map obtained by multiplying the weight output by the Sigmoid activation function with the feature map of the input edge-guided attention module is used as the output of the first branch.
[0050] The second branch includes a maximum pooling layer, a convolution layer (the convolution kernel size is 7×7) and a Sigmoid activation function connected in sequence; the second branch is used to screen important features in the feature map space. The second branch performs a maximum pooling operation on the feature map of the input edge-guided attention module along the channel dimension in space; the features after maximum pooling are weighted through the convolution layer, and the results are input into the Sigmoid activation function to obtain the weights, and the feature map obtained by multiplying the weights output by the Sigmoid activation function with the feature map of the input edge-guided attention module is used as the output of the second branch.
[0051] The feature fusion block includes an edge detection module, a batch normalization layer, a ReLU activation function and a convolution layer (the convolution kernel size is 3×3) connected in sequence; the feature fusion block adds and fuses the output of the first branch, the output of the second branch and the input of the edge-guided attention module to obtain a preliminary fused feature map, and then adds and fuses the preliminary fused feature map with the preliminary fused feature map after edge detection, batch normalization layer, ReLU activation function and convolution operation to obtain an edge-guided attention feature map as the output of the edge-guided attention module.
[0052] The edge detection module uses the Sobel operator, which is a first-order differential operator. It includes two 3×3 convolution kernels, filter_x and filter_y, which are used to calculate the gradient of the image in the horizontal and vertical directions respectively.
[0053] in,
[0054] The squares of the horizontal and vertical gradient values are added together, and then the square root is taken to obtain a comprehensive gradient that reflects the edge strength of the image. The result is weighted by the Sigmoid function in the edge detection module, and then multiplied with the input feature map to obtain an edge-guided feature map to extract boundary information.
[0055] The multi-scale feature map fusion module includes a splicing layer, a convolution layer and a Sigmoid activation function. The images processed by the six convolution layers in the decoder are spliced along the channel through the splicing layer, and the spliced feature maps are sequentially passed through the convolution layer and the Sigmoid activation function to obtain the prediction probability map Sfuse.
[0056] Step 3: Input the training set into the MEDANet model for training, and test the trained model with the test set; use multiple Dice Loss weighted summation methods as the loss function of the MEDANet model. The loss function is divided into two parts. One part is the loss between the feature map and the label of the input multi-scale feature map fusion module. ; The other part is the loss between the feature map output by the multi-scale edge-guided attention image segmentation model and the label The total loss of the MEDANet model is obtained by weighted summing the two losses. The expression of the total loss L is:
[0057]
[0058] in, and The losses and loss The weights of are all set to 1; .
[0059] In this embodiment, the input image size is 512×512, the Batch Size is set to 10, the number of iterations is 200, the Adam optimizer is used for optimization, and the initial learning rate is set to 0.01.
[0060] Step 4: Use the trained model to identify and segment bone metastases in CT images of colorectal cancer bone metastases.
[0061] Step 5: Model Evaluation
[0062] The present invention and the existing model are used to segment the colorectal cancer bone metastasis images. Figure 5 It can be seen that the segmentation results obtained by the present invention have a high degree of overlap with the labels, and there is no mis-segmentation of large areas. Compared with other models, the present invention performs better in the segmentation of the edge area of colorectal cancer osteogenic metastases.
[0063] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.
Claims
1. A medical image segmentation method based on edge-guided attention mechanism, characterized in that: The method comprises the following steps: Step 1: Construct a data set; the data set includes the original medical image and the labels of the regions of interest in the original medical image; Step 2: construct a multi-scale edge-guided attention image segmentation model; the multi-scale edge-guided attention image segmentation model uses a U-Net network as a basic framework, and the U-Net network includes an encoder and a decoder; The encoder includes multiple encoding layers, and the decoder includes multiple decoding layers; the encoding layers and decoding layers at the same layer are connected by skip connections, and the first four encoding layers and the corresponding decoding layers are connected in series with an edge-guided attention module on the skip connections; The edge-guided attention module includes a first branch, a second branch, and a feature fusion block; The first branch specifically multiplies the input feature map by the result of processing the input feature map with the maximum pooling operation, the first convolution, the ReLU activation function, the second convolution and the Sigmoid activation function in sequence, to obtain the first output feature map; The second branch specifically multiplies the input feature map by the result of processing the input feature map by the maximum pooling operation, convolution and Sigmoid activation function in sequence, to obtain a second output feature map; The feature fusion block specifically adds the first output feature map, the second output feature map and the original input feature map to obtain a preliminary fused feature map, and then sequentially performs edge detection, batch normalization layer, ReLU activation function, and convolution operation on the preliminary fused feature map before adding and fusing it with the preliminary fused feature map; The output of the last encoding layer is fused with the output of each decoding layer through a multi-scale feature map fusion module to obtain the output of the multi-scale edge-guided attention image segmentation model; The encoding layer and the decoding layer of the multi-scale edge-guided attention image segmentation model both use feature extraction modules; The feature extraction module uses the U-Net network as the basic framework, including multiple sub-coding layers and multiple sub-decoding layers; direct jump connections are used between the sub-coding layers and sub-decoding layers in the same layer; the input of each sub-decoding layer is the concatenation result of the output feature map of the previous sub-decoding layer and the output feature map of the same sub-coding layer; Step 3: Use the dataset to train a multi-scale edge-guided attention image segmentation model; Step 4: Use the trained model to segment the medical image under test.
2. The medical image segmentation method based on edge-guided attention mechanism according to claim 1, characterized in that: In the encoder, except for the last encoding layer, all encoding layers downsample their output feature maps and input them to the next encoding layer; In the decoder, the input of each decoding layer is the concatenation of the output feature map of the previous decoding layer after upsampling and the output feature map of the same encoding layer after edge-guided attention module processing; the outputs of each decoding layer and the different-scale feature maps of the last encoding layer are convolved through multiple convolutional layers, and the sizes of the feature maps output by multiple convolutional layers are kept consistent with the size of the original medical image through bilinear interpolation and input into the multi-scale feature map fusion module; the number of convolutional layers is the same as the number of encoding layers.
3. The medical image segmentation method based on edge-guided attention mechanism according to claim 1, characterized in that: The sub-coding layer and the sub-decoding layer in the feature extraction module have the same structure, both of which include a dilated convolution layer, a batch normalization layer and a ReLU activation function connected in sequence.
4. The medical image segmentation method based on edge-guided attention mechanism according to claim 1, characterized in that: The multi-scale feature map fusion module includes a splicing layer, a convolution layer and a Sigmoid activation function; wherein, the feature maps output by multiple convolution layers are spliced along the channel through the splicing layer, and the spliced feature maps are sequentially passed through the convolution layer and the Sigmoid activation function to obtain a prediction probability map.
5. The medical image segmentation method based on edge-guided attention mechanism according to claim 1, characterized in that: In step 3, the total loss of the multi-scale edge-guided attention image segmentation model L The expression is: ； in, l m It is the loss between the feature map and the label of the input multi-scale feature map fusion module; l fuse The loss between the feature map output by the multi-scale edge-guided attention image segmentation model and the label; ω m and ω fuse The losses l m and loss l fuse The weight of M Indicates the number of coding layers in the encoder.
6. The medical image segmentation method based on edge-guided attention mechanism according to claim 1, characterized in that: The medical image is a computer tomography image.
7. A computer device, characterized in that: It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the memory stores the computer program; and the processor executes a medical image segmentation method based on an edge-guided attention mechanism as described in claim 1.
8. A readable storage medium, characterized in that: A computer program is stored; when the computer program is executed by a processor, it is used to implement the medical image segmentation method based on the edge-guided attention mechanism described in claim 1.