A medical image segmentation method and system based on edge feature attention guidance

Abstract

The application belongs to the technical field of medical image processing, and provides a medical image segmentation method and system based on edge feature attention guidance. The method comprises the following steps: encoding a medical image by using an encoder; fusing a feature map output by a first layer of the encoder and a feature map output by a second layer to obtain a fused feature map, and guiding the fused feature map by using an edge feature attention module to obtain an attention feature map; performing cross-level pixel reconstruction on the feature map segmented by the encoder by using a global feature reconstruction module between the encoder and a decoder to obtain a global reconstruction feature; based on the global reconstruction feature, aggregating scale information and edge attention representation of the feature map output by each layer of the decoder by using a weight module to obtain a weight feature map; and fusing the attention feature map and the weight feature map to obtain a segmentation result.

Description

Technical Field

[0001] This invention belongs to the field of medical image processing technology, and in particular relates to a medical image segmentation method and system based on edge feature attention guidance. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Medical image segmentation is a crucial step in medical image analysis. Currently, an increasing number of excellent deep learning-based solutions are being proposed to address the problem of medical image segmentation. Existing medical image segmentation methods can be mainly categorized into two types: edge detection and object segmentation.

[0004] In recent years, U-Net has been widely used in medical image segmentation. However, U-Net and most of its variants directly combine low-level and high-level features without extracting edge features. Existing convolutional neural networks (CNNs) actually learn texture features, not shape features. Learning shape features can improve the segmentation accuracy of the network. Many studies have attempted to integrate low-level and high-level features using attention mechanisms, which have yielded significant improvements but still suffer from redundant information and numerous network parameters. The ET-Net method combines edge detection and semantic segmentation, utilizing edge information. However, it ignores global information, and the structure for extracting edge information is relatively simple, resulting in fewer extracted features. Summary of the Invention

[0005] To address the technical problems mentioned above, this invention provides a medical image segmentation method and system based on edge feature attention guidance. While reconstructing global information, it uses shallow edge information in the encoder to guide the segmentation of the decoder. In the decoder, a weighted aggregation module is used to emphasize valuable features and aggregate multi-scale information, effectively solving the problems of insufficient edge features and the inadequate global feature extraction capability of traditional U-Net.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] The first aspect of the present invention provides a medical image segmentation method based on edge feature attention guidance.

[0008] A medical image segmentation method based on edge feature attention guidance includes:

[0009] Encoders are used to encode medical images;

[0010] The feature map output from the first layer encoder and the feature map output from the second layer are fused to obtain a fused feature map. An edge feature attention module is used to guide the fused feature map to obtain an attention feature map.

[0011] Between the encoder and decoder, the global feature reconstruction module performs cross-level pixel reconstruction on the feature map segmented by the encoder to obtain the global reconstructed features.

[0012] Based on global reconstruction features, a weighted feature map is obtained by aggregating the scale information and edge attention representation of the feature map output by each decoder layer using a weighted module.

[0013] The attention feature map and the weight feature map are fused to obtain the segmentation result.

[0014] Furthermore, the process of obtaining the fused feature map includes: connecting the feature map output by the first layer encoder and the feature map output by the second layer, and then fusing them using a convolutional layer to obtain the fused feature map.

[0015] Furthermore, the process of using the edge feature attention module to guide the fusion feature map to obtain the attention feature map includes: based on the fusion feature map, using the ReLU activation function to introduce nonlinearity, then using a convolutional layer to unify the number of feature channels, and using a softmax layer to obtain the attention map; multiplying the attention map and the fusion feature map, and then fusing them through a convolutional layer to obtain the attention feature map.

[0016] Furthermore, the process of performing cross-level pixel reconstruction on the feature map segmented by the encoder through the global feature reconstruction module includes: generating a set of global descriptors representing each class; for the l-th level feature map, if the global descriptor Z extracted at that level is used... l When performing feature reconstruction, the pixel at each location will be represented by Z... l Interrelated; combine the global descriptor at this level with all higher-level global descriptors to obtain a cross-level global descriptor Z. l ', using a cross-level global descriptor Z l Reconstruct the pixels at each location in the feature map to obtain the global reconstructed features.

[0017] Furthermore, the use of a cross-level global descriptor Z l The process of reconstructing the global reconstructed features for each pixel in the feature map includes: predicting the global reconstruction of the current feature map, and using a cross-level global descriptor Z. l Reconstruct the pixels at each location in the globally reconstructed feature map to obtain the globally reconstructed features.

[0018] Furthermore, the process of obtaining a weighted feature map by aggregating the scale information and edge attention representation of the feature map output by each decoder layer based on global reconstruction features includes: inputting the feature map output by each decoder layer into the weighting block based on global reconstruction features; firstly, aggregating the global context information in the feature map of each encoder layer using global average pooling; then, estimating the correlation of each decoder layer using ReLU and Sigmoid nonlinear activation functions and generating weights along the channel dimension; and multiplying the weights by the output of each encoder layer to obtain the weighted feature map.

[0019] Furthermore, if the medical image is a historical medical image, the historical medical image is preprocessed to construct a training set and a validation set; the training set is used to train the neural network consisting of the encoder, decoder, edge feature attention module, global feature reconstruction module and weight module; the validation set is used to validate the trained neural network. When the loss value between the real result and the result output by the neural network is less than the minimum loss value during the training process, the trained neural network is obtained.

[0020] A second aspect of the present invention provides a medical image segmentation system based on edge feature attention guidance.

[0021] A medical image segmentation system based on edge feature attention guidance, comprising:

[0022] The encoder module is configured to encode medical images;

[0023] The edge feature attention module is configured to fuse the feature map output by the first layer encoder and the feature map output by the second layer to obtain a fused feature map, and guide the fused feature map to obtain an attention feature map.

[0024] The global feature reconstruction module is configured to be located between the encoder and decoder to perform cross-level pixel reconstruction on the feature maps segmented by the encoder to obtain globally reconstructed features.

[0025] The weighting module is configured to: aggregate the scale information and edge attention representation of the feature maps output by each decoder layer based on the global reconstructed features to obtain a weighted feature map;

[0026] The fusion module is configured to fuse the attention feature map and the weight feature map to obtain the segmentation result.

[0027] A third aspect of the present invention provides a computer-readable storage medium.

[0028] A computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of the edge feature-based attention-guided medical image segmentation method as described in the first aspect above.

[0029] A fourth aspect of the present invention provides a computer device.

[0030] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the steps of the edge feature-based attention-guided medical image segmentation method as described in the first aspect above.

[0031] Compared with the prior art, the beneficial effects of the present invention are:

[0032] This invention, while reconstructing global information, uses shallow edge information in the encoder to guide the segmentation of the decoder. In the decoder, a weighted aggregation module is used to emphasize valuable features and aggregate multi-scale information, effectively solving the problems of insufficient learning edge features and the inadequate global feature extraction capability of traditional U-Net. Attached Figure Description

[0033] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0034] Figure 1 This is a flowchart illustrating the medical image segmentation method based on edge feature attention guidance as shown in this invention;

[0035] Figure 2 This is a flowchart of medical image data preprocessing as shown in this invention;

[0036] Figure 3 This is a schematic diagram of the medical image segmentation method guided by edge feature attention as shown in this invention;

[0037] Figure 4 This is a schematic diagram of the global feature reconstruction module shown in this invention;

[0038] Figure 5 This is a schematic diagram of the weighting module shown in this invention. Detailed Implementation

[0039] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0040] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0041] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0042] It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of methods and systems according to various embodiments of this disclosure. It should be noted that each block in a flowchart or block diagram may represent a module, segment, or portion of code, which may include one or more executable instructions for implementing the logical functions specified in the various embodiments. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that shown in the drawings. For example, two consecutively represented blocks may actually be executed substantially in parallel, or they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, may be implemented using a dedicated hardware-based system that performs the specified functions or operations, or using a combination of dedicated hardware and computer instructions.

[0043] Example 1

[0044] like Figure 1 As shown, this embodiment provides a medical image segmentation method based on edge feature attention guidance. This embodiment uses the application of this method to a server as an example for illustration. It is understood that this method can also be applied to terminals, and can also be applied to systems including terminals, servers, and other components, and implemented through interaction between the terminal and the server. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network servers, cloud communication, middleware services, domain name services, CDN security services, and big data and artificial intelligence platforms. The terminal can be a smartphone, tablet, laptop, desktop computer, etc., but is not limited to these. The terminal and server can be directly or indirectly connected via wired or wireless communication, which is not limited herein. In this embodiment, the method includes the following steps:

[0045] Encoders are used to encode medical images;

[0046] The feature map output from the first layer encoder and the feature map output from the second layer are fused to obtain a fused feature map. An edge feature attention module is used to guide the fused feature map to obtain an attention feature map.

[0047] Between the encoder and decoder, the global feature reconstruction module performs cross-level pixel reconstruction on the feature map segmented by the encoder to obtain the global reconstructed features.

[0048] Based on global reconstruction features, a weighted feature map is obtained by aggregating the scale information and edge attention representation of the feature map output by each decoder layer using a weighted module.

[0049] The attention feature map and the weight feature map are fused to obtain the segmentation result.

[0050] The specific steps of this embodiment will be described in detail below with reference to the accompanying drawings:

[0051] Step (1): As Figure 2 As shown, the acquired medical image data is preprocessed to obtain the training dataset.

[0052] The acquired images were uniformly changed to PNG or JPG format, with pixel values ​​set to 224×224. The images were then normalized to transform pixel values ​​to the range of 0-1, and divided into training and test sets according to the ratio.

[0053] Because acquiring medical image data is more difficult than acquiring ordinary image data, and data annotation is time-consuming and labor-intensive, requiring guidance from professional doctors, the embodiments of this invention use a five-fold cross-training method. After randomly dividing the training set and the test set, online data augmentation strategies are adopted for the training set, including random rotation and random flipping.

[0054] Random rotation: Set angles of 90°, 180°, and 270° to rotate the image;

[0055] Random Flip: Flip the image along the horizontal or vertical direction.

[0056] Step (2): As Figure 3 As shown, a neural network model based on edge feature attention guidance is constructed, including an encoder-decoder part, an edge attention guidance module, a global feature reconstruction module, and a weight module.

[0057] like Figure 3As shown, we utilize an edge feature attention guidance module in the early stages of the encoder to retain more useful edge features, better guiding feature extraction during segmentation. The steps are as follows: First, the features of the second-layer encoder are upsampled to a resolution common to the features of the first-layer encoder and then concatenated. Next, a 1×1 convolutional layer is used to fuse the features. A ReLU activation function is used to introduce non-linearity, followed by another convolutional operation to unify the number of feature channels. Then, a softmax layer is used to obtain the attention map. After multiplying the attention map with the fused features, a convolutional layer is used to generate the final attention features.

[0058] like Figure 4 As shown, building a Global Feature Reconstruction Module (GER) between the encoder and decoder involves three steps: (1) generating a set of global descriptors representing each class; (2) combining the global descriptors at this level with global descriptors at all higher levels to obtain cross-level global descriptors; and (3) predicting the weights at each location and reconstructing each pixel using the cross-level global descriptors.

[0059] For the l-th level feature map, if the global descriptor Z extracted at this level is used... l When performing feature reconstruction, the pixel at each location will be represented by Z... l They are interconnected. Furthermore, combining the global descriptor at this level with all higher-level global descriptors yields a cross-level global descriptor set, which can be expressed as:

[0060] Z l =concat(Z) l Z l+1 ,...,Z l+m )

[0061] Among them, Z l Z l+1 ,...,Z l+m It is a global descriptor generated from the l+1 to l+m level feature maps. In this way, descriptors from high-level features with strong semantic information can be used to reconstruct low-level feature maps. Therefore, the reconstructed feature map will have rich spatial details and semantic information.

[0062] The next step is to use the cross-level global descriptor Z. l To reconstruct the features at each location. Predict the global reconstruction weights V using a 1×1 algorithm on the current feature map. l The softmax function in the channel dimension is used to normalize the reconstruction weights, enhancing the selection ability of the global descriptor. Feature map X l The reconstruction process can be described as follows:

[0063]

[0064] in, This represents the reconstructed feature map. Finally, to prevent degradation during network training, the reconstructed feature map... Perform a 1×1 convolution, and then... Add input feature X l This yields the final global reconstruction features.

[0065] like Figure 5 As shown, the weighting module is used at each stage of the decoder to emphasize valuable features and aggregate multi-scale information and edge attention representations to improve segmentation performance. The output of each decoder is input to the weighted block to highlight valuable information. First, global average pooling is used to aggregate global contextual information from the input. Then, ReLU and Sigmoid nonlinear activation functions are used to estimate the relevance of the layers and generate weights along the channel dimension. The generated weights are multiplied by the output to produce more representative features.

[0066] Step (3): Train the training dataset with an attention-guided neural network based on edge features to obtain a trained attention-guided neural network model.

[0067] This embodiment uses a five-fold cross-training method to randomly divide the training set and the validation set. The constructed neural network is used to train the training set and the test set, and the cross-entropy loss function is used to calculate the loss value between the true label and the predicted result. During the training of the validation set, the change of the loss value is recorded. If the loss value is less than the minimum loss value during the training process, the current network model needs to be saved and the network parameters need to be updated. Otherwise, a new round of training is continued. The training epoch is set to 150. If the number of training iterations is less than 150, the process returns to step (4) to continue a new round of training. After the epoch reaches 150, the edge feature attention-guided neural network model training is completed.

[0068] Step (4): Input the medical image to be segmented into the trained edge attention-guided neural network model to obtain the predicted segmentation results and evaluation metrics.

[0069] Example 2

[0070] This embodiment provides a medical image segmentation system based on edge feature attention guidance.

[0071] A medical image segmentation system based on edge feature attention guidance, comprising:

[0072] The encoder module is configured to encode medical images;

[0073] The edge feature attention module is configured to fuse the feature map output by the first layer encoder and the feature map output by the second layer to obtain a fused feature map, and guide the fused feature map to obtain an attention feature map.

[0074] The global feature reconstruction module is configured to be located between the encoder and decoder to perform cross-level pixel reconstruction on the feature maps segmented by the encoder to obtain globally reconstructed features.

[0075] The weighting module is configured to: aggregate the scale information and edge attention representation of the feature maps output by each decoder layer based on the global reconstructed features to obtain a weighted feature map;

[0076] The fusion module is configured to fuse the attention feature map and the weight feature map to obtain the segmentation result.

[0077] It should be noted that the encoder module, edge feature attention module, global feature reconstruction module, weight module, and fusion module described above are the same as those implemented in the examples and application scenarios corresponding to the steps in Embodiment 1, but are not limited to the content disclosed in Embodiment 1. It should also be noted that the above modules, as part of the system, can be executed in a computer system such as a set of computer-executable instructions.

[0078] Example 3

[0079] This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in the medical image segmentation method based on edge feature attention as described in Embodiment 1 above.

[0080] Example 4

[0081] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in the medical image segmentation method based on edge feature attention as described in Embodiment 1 above.

[0082] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0083] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0084] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0085] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0086] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0087] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A medical image segmentation method based on edge feature attention guidance, characterized in that, include: Encoders are used to encode medical images; The feature map output from the first layer encoder and the feature map output from the second layer are fused to obtain a fused feature map. An edge feature attention module is used to guide the fused feature map to obtain an attention feature map. Between the encoder and decoder, the global feature reconstruction module performs cross-level pixel reconstruction on the feature map segmented by the encoder to obtain the global reconstructed features. The process of performing cross-level pixel reconstruction on the feature map segmented by the encoder through the global feature reconstruction module includes: generating a set of global descriptors representing each class; for the l-th level feature map, if the global descriptor extracted at this level is used... Z l When performing feature reconstruction, the pixels at each location will be... Z l Interrelated; combine the global descriptor at this level with all higher-level global descriptors to obtain a cross-level global descriptor. Use cross-level global descriptors Reconstruct the pixels at each location in the feature map to obtain the global reconstructed features; Based on global reconstruction features, a weighted feature map is obtained by aggregating the scale information and edge attention representation of the feature map output by each decoder layer using a weighted module. The attention feature map and the weight feature map are fused to obtain the segmentation result.

2. The medical image segmentation method based on edge feature attention guidance according to claim 1, characterized in that, The process of obtaining the fused feature map includes: connecting the feature map output by the first layer encoder and the feature map output by the second layer, and then using a convolutional layer to fuse them to obtain the fused feature map.

3. The medical image segmentation method based on edge feature attention guidance according to claim 1, characterized in that, The process of using an edge feature attention module to guide the fusion feature map to obtain an attention feature map includes: based on the fusion feature map, a ReLU activation function is used to introduce non-linearity, then a convolutional layer is used to unify the number of feature channels, and a softmax layer is used to obtain the attention map; after multiplying the attention map and the fusion feature map, they are fused through a convolutional layer to obtain the attention feature map.

4. The medical image segmentation method based on edge feature attention guidance according to claim 1, characterized in that, The use of cross-level global descriptors The process of reconstructing the global reconstructed features for each pixel in the feature map includes: predicting the global reconstruction of the current feature map, and using cross-level global descriptors. Reconstruct the pixels at each location in the globally reconstructed feature map to obtain the globally reconstructed features.

5. The medical image segmentation method based on edge feature attention guidance according to claim 1, characterized in that, The process of obtaining a weighted feature map by aggregating the scale information and edge attention representation of the feature map output by each decoder layer based on global reconstruction features includes: inputting the feature map output by each decoder layer into the weighting block based on global reconstruction features; firstly, aggregating the global context information in the feature map of each encoder layer using global average pooling; then, estimating the correlation of each decoder layer using ReLU and Sigmoid nonlinear activation functions and generating weights along the channel dimension; and multiplying the weights by the output of each encoder layer to obtain the weighted feature map.

6. The medical image segmentation method based on edge feature attention guidance according to claim 1, characterized in that, If the medical image is a historical medical image, the historical medical image is preprocessed to construct a training set and a validation set. The training set is used to train the neural network consisting of the encoder, decoder, edge feature attention module, global feature reconstruction module and weight module. The validation set is used to validate the trained neural network. When the loss value between the real result and the result output by the neural network is less than the minimum loss value during the training process, the trained neural network is obtained.

7. A medical image segmentation system based on edge feature attention guidance, characterized in that, include: The encoder module is configured to encode medical images; The edge feature attention module is configured to fuse the feature map output by the first layer encoder and the feature map output by the second layer to obtain a fused feature map, and guide the fused feature map to obtain an attention feature map. The global feature reconstruction module is configured to be located between the encoder and decoder to perform cross-level pixel reconstruction on the feature maps segmented by the encoder to obtain globally reconstructed features. The process of performing cross-level pixel reconstruction on the feature map segmented by the encoder through the global feature reconstruction module includes: generating a set of global descriptors representing each class; for the l-th level feature map, if the global descriptor extracted at this level is used... Z l When performing feature reconstruction, the pixels at each location will be... Z l Interrelated; combine the global descriptor at this level with all higher-level global descriptors to obtain a cross-level global descriptor. Use cross-level global descriptors Reconstruct the pixels at each location in the feature map to obtain the global reconstructed features; The weighting module is configured to: aggregate the scale information and edge attention representation of the feature maps output by each decoder layer based on the global reconstructed features to obtain a weighted feature map; The fusion module is configured to fuse the attention feature map and the weight feature map to obtain the segmentation result.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in the medical image segmentation method based on edge feature attention as described in any one of claims 1-6.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the medical image segmentation method based on edge feature attention as described in any one of claims 1-6.

Images