Medical image segmentation method, device, system, and storage medium

Abstract

The present invention discloses a medical image segmentation method, device, system, and storage medium, including: step S1, obtaining historical medical images; step S2, training a BMSFE-SegNet network based on the historical medical images; step S3, inputting the medical image to be processed into the trained BMSFE-SegNet network for image segmentation. The technical solution of the present invention solves the problem of edge multi-scale feature fusion mismatch caused by blurred edge information of medical images and the introduction of noise during the fusion process.

Description

Technical Field

[0001] The present invention belongs to the field of image processing technology, and in particular relates to a medical image segmentation method and device, system, and storage medium based on edge perception. Background Art

[0002] Medical image analysis and processing play an important role in medical diagnosis tasks. As one of the most common tasks in the field of medical image processing, medical image segmentation can accurately locate lesions and organ structures in images and separate them from images, providing important clues for clinicians to conduct further diagnosis or surgical treatment. However, in the field of medical image segmentation, compared with natural images, medical images have the characteristics of more noise, low resolution and contrast, and complex and diverse imaging principles. As a result, most medical images do not have obvious structural edges, making it difficult to accurately segment the target area. Therefore, medical image segmentation is still a challenging task.

[0003] Traditional medical image segmentation algorithms use manually designed features to identify segmentation targets in images. This type of method requires different features to be designed for the diagnosis of different diseases, and is easily disturbed by noise in the image, resulting in erroneous prediction results. Therefore, it is not practical in actual clinical practice. Deep learning technology, which has emerged in recent years, has gradually become the preferred method in the field of image segmentation due to its powerful feature expression ability. In the task of medical image segmentation, in order to solve the problem of blurred edge information, most methods restore edge details by enforcing inter-pixel dependencies. However, these methods only focus on local edge areas and cannot effectively learn global multi-scale information. Secondly, the multi-scale features of the edge area cannot be effectively registered, which affects the pixel-level fusion of features and thus the segmentation effect of the model. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a medical image segmentation method and device, system, and storage medium to solve the problem of edge multi-scale feature fusion mismatch caused by blurred edge information of medical images and the introduction of noise during the fusion process.

[0005] To achieve the above object, the present invention adopts the following technical solution:

[0006] A medical image segmentation method, comprising:

[0007] Step S1, obtaining historical medical images;

[0008] Step S2, training the BMSFE-SegNet network based on historical medical images;

[0009] Step S3: input the medical image to be processed into the trained BMSFE-SegNet network for image segmentation.

[0010] Preferably, the BMSFE-SegNet network comprises: an encoder, a multi-scale receptive field module MRFB module, a medical image edge prediction module and a medical image segmentation module; wherein the medical image segmentation module comprises: an edge-aware multi-scale feature fusion module RMSF and an edge-aware attention enhancement module BAE.

[0011] As a preference, the total loss function for training the BMSFE-SegNet network for: in, is the IoU loss, Loss for BCE is the edge prediction graph loss.

[0012] The present invention also provides a medical image segmentation device, comprising:

[0013] An acquisition module, used to acquire historical medical images;

[0014] A training module for training the BMSFE-SegNet network based on historical medical images;

[0015] The segmentation module is used to input the medical image to be processed into the trained BMSFE-SegNet network for image segmentation.

[0016] Preferably, the BMSFE-SegNet network comprises: an encoder, a multi-scale receptive field module MRFB module, a medical image edge prediction module and a medical image segmentation module; wherein the medical image segmentation module comprises: an edge-aware multi-scale feature fusion module RMSF and an edge-aware attention enhancement module BAE.

[0017] As a preference, the total loss function for training the BMSFE-SegNet network for: in, is the IoU loss, Loss for BCE is the edge prediction graph loss.

[0018] The present invention also provides a medical image segmentation system, comprising: a memory and a processor, wherein the memory stores a computer program executed by the processor, and the computer program executes the medical image segmentation method when executed by the processor.

[0019] The present invention also provides a storage medium, on which a computer program is stored, and the computer program executes the medical image segmentation method when running.

[0020] The present invention introduces an edge-aware attention enhancement module and an edge-aware multi-scale fusion module to achieve more accurate segmentation, and solve the problem of edge multi-scale feature fusion mismatch caused by blurred edge information of medical images and the introduction of noise during the fusion process. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0022] Figure 1 is a flow chart of a medical image segmentation method according to an embodiment of the present invention;

[0023] Figure 2 A schematic diagram of a network model used in a medical image segmentation method according to an embodiment of the present invention;

[0024] Figure 3 Schematic diagram of edge-aware multi-scale feature fusion module (BMSF);

[0025] Figure 4 Schematic diagram of the edge-aware attention enhancement module (BAE). DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0028] Embodiment 1:

[0029] like Figure 1 , 2 As shown, an embodiment of the present invention provides a medical image segmentation method.

[0030] Step S1, obtaining historical medical images;

[0031] Step S2, training the BMSFE-SegNet network based on historical medical images;

[0032] Step S3: input the medical image to be processed into the trained BMSFE-SegNet network for image segmentation.

[0033] As an implementation method of an embodiment of the present invention, the BMSFE-SegNet network uses PVTv2 as the backbone network of the model, and the BMSFE-SegNet network includes: an encoder, a multi-scale receptive field module (MRFB module), a medical image edge prediction module and a medical image segmentation module; wherein the medical image segmentation module includes: an edge-aware multi-scale feature fusion module RMSF and an edge-aware attention enhancement module BAE.

[0034] Furthermore, the specific process of training the BMSFE-SegNet network based on historical medical images is as follows:

[0035] First, historical medical images Through the four-layer encoder, the pyramid feature map is obtained Among them, i represents the number of network layers, C i represents the number of channels of the feature map of the i-th layer. Then, in order to obtain a multi-scale receptive field and dimensionality reduction, the present invention converts U i Input into the multi-scale receptive field module (i.e., MRFB module) for feature aggregation, and obtain Next, U i ′ will be input into the medical image edge prediction module and the medical image segmentation module respectively. Specifically, in order to obtain the edge features of the medical image Will U i ’ performs 3×3 convolution operation, batch normalization operation and ReLU activation function. It is divided into two branches. The first branch passes through 1 × 1 After the convolution operation, it becomes a single-channel feature, where Upsampled to The same spatial size, finally through the Sigmoid function, aggregate the four-layer single-channel features to obtain the edge prediction output E0. The second branch obtains the edge perception feature through a 1×1 convolution operation Used to guide medical image segmentation.

[0036] For the medical image segmentation module, it consists of an edge-aware multi-scale feature fusion module RMSF and an edge-aware attention enhancement module BAE. First, the U extracted by MRFB is i ′ and the BMSF module of the previous layer after fusion Input to the BMSF fusion module and perceive features at the edge The BMSF module of the current layer is input under the action of to perform multi-scale feature alignment and fusion to obtain the fused features Subsequently, in order to suppress the noise in the fusion process, the edge-aware features Also as a guiding feature, Input the BAE module to obtain the attention enhanced feature map V i Finally, V i After a series of convolution operations and Sigmoid functions, the medical image segmentation prediction map E is obtained. i , the highest resolution E1 is used as the final medical image segmentation output in the BMSFE-SegNet network training stage.

[0037] The working process of the edge-aware multi-scale feature fusion module (BMSF) is as follows:

[0038] like Figure 3 As shown in Figure 1, the edge-aware multi-scale fusion module is used to align and accurately fuse multi-scale features on a spatial scale. Specifically, in the edge-aware feature Under the guidance of i 'The spatial dimensions do not match Through bilinear interpolation, it is obtained by upsampling by 2 times. Similarly, It is also necessary to follow the U i ′ is downsampled. Then, U i 'and Edge-aware features Add together to get the feature R with edge information i With R i+1 . R i With R i+1 Splicing, and after a 3 × 3 convolution operation to get the offset Then, guided by the offset, Input to the deformable convolution module to get the alignment feature The motivation for using deformable convolution [1] in the embodiment of the present invention is to make the feature information not only have the same spatial size but also be aligned in the spatial dimension. In order to further efficiently fuse multi-scale information, the present invention proposes a multi-scale adaptive fusion module (MSAF) to perform adaptive weighted summation on the aligned multi-scale features to obtain a multi-scale fused feature. Among them, the adaptive matrix The feature U to be fused i 'and Input a 1×1 convolution unit respectively, and transform its channel number to 1. After that, the two single-channel features are concatenated and passed through a Softmax function to obtain two corresponding weight matrices α and α′, and the relationship between the two is α+α′=1. The specific formula is as follows:

[0039]

[0040] Among them, Conv a (·) represents a 1×1 convolution unit, and Concat(·) represents a concatenation operation. i 'and Multiply them with their respective weight matrices and then add them together to get the final fusion feature The formula is as follows:

[0041]

[0042] The working process of the edge-aware attention enhancement module (BAE) is as follows:

[0043] In order to suppress noise in the feature fusion process and reduce the impact of noise on segmentation accuracy, an embodiment of the present invention proposes an edge-aware attention enhancement module. At the channel level and spatial level, the attention enhancement module learns the correlation of different channel dimensions and the dependency of different spatial dimensions through the attention mechanism, so that the network can highlight the importance of the segmented area, strengthen the perception of edge information, accurately focus on local details, and reduce the interference of channel noise and the redundancy of spatial information.

[0044] like Figure 4 As shown, the attention enhancement module uses edge-aware features and edge-aware multi-scale fusion features As input, the edge-aware attention-enhanced features are finally output. Specifically, in the channel dimension, the multi-scale fused features It is divided into two branches. On the one hand, global average pooling is performed to obtain one-dimensional features. Then input it into the fully connected layer to get the query vector on the other hand Also through a 1 × 1 convolutional layer and flattened into value features At the same time, edge-aware features As a guiding feature, The first branch is also input into the global pooling layer and the fully connected layer to obtain With key vector The specific formula is as follows:

[0045]

[0046] q i =f1(G i ), k F =f2(G F ) (4)

[0047] The embodiment of the present invention considers that q i Represent the features that need to be enhanced from the channel dimension And k F Representing guiding features that carry edge information from the channel dimension Therefore, q i and the transposed k F Multiply and then perform Softmax operation to get the channel similarity matrix This matrix reflects A channel with The degree of match between the specific channels of i With value feature v i After matrix multiplication and shape restoration, the enhanced features in the channel dimension are obtained. The formula is as follows:

[0048]

[0049]

[0050]

[0051] In the spatial dimension, according to The size of the spatial scale is matched, and then and The average values ​​are calculated and concatenated in the channel dimension to obtain the dual-channel features.

[0052]

[0053] Afterwards, H i Input to the 7×7 convolution layer and the Sigmoid layer to obtain the spatial attention weight The spatial attention weights and the features enhanced by the channel dimension Multiply and then Add together to get the final enhanced feature V i The formula is as follows:

[0054] k i =Sigmoid(Conv 7×7 (H i )) (9)

[0055]

[0056] The loss function for training the BMSFE-SegNet network is as follows:

[0057] Calculate edge prediction graph loss

[0058] In the embodiment of the present invention, when the network is trained, the edge prediction graph E0 is supervised by using Dice Loss. The specific calculation method is to use the obtained E0 and the edge truth label mask graph G E Calculate the loss, recorded as:

[0059]

[0060] Calculate the segmentation prediction map loss:

[0061] In the embodiment of the present invention, when the BMSFE-SegNet network is trained, the segmentation prediction graph E generated finally is i Calculate IoU loss using weighted IoU loss function and weighted BCE loss function and BCE loss The specific calculation method is to use the obtained multi-scale E i and segmentation truth label mask G S Calculate the loss, recorded as

[0062]

[0063]

[0064] Finally, the loss function of the model during training is (i.e., the segmentation prediction map loss) is

[0065]

[0066] The embodiment of the present invention uses the medical image public dataset Kvasir-SEG, which is a colorectal polyp segmentation dataset based on endoscopic images. There are 1000 groups of images in total, each of which contains an endoscopic image containing polyps and a corresponding true value label mask map, and the image resolution ranges from 332*487 to 1920*1072. To facilitate the experiment, the embodiment of the present invention adjusts the size of all images and true value label mask maps to 332*332 in the network training experiment, uses 800 groups of images for model training, and the remaining 200 groups of images for testing.

[0067] The embodiment of the present invention is implemented under the PyTorch deep learning framework and runs on the Ubuntu 18.04 operating system. The network model uses the AdamW optimizer, the learning rate is 0.0001, the batch size of the training data is 32 during training, and the epoch is set to 100.

[0068] The embodiment of the present invention uses the Dice similarity coefficient (Dice), an evaluation index widely recognized in the field of medical image segmentation, to measure network performance. Generally, the larger the Dice, the better the performance.

[0069] The experimental results of the embodiment of the present invention show that the Dice coefficient of the test result of the embodiment of the present invention on the Kvasir data set is 0.920.

[0070] Embodiment 2:

[0071] The embodiment of the present invention further provides a medical image segmentation device, comprising:

[0072] An acquisition module, used to acquire historical medical images;

[0073] A training module for training the BMSFE-SegNet network based on historical medical images;

[0074] The segmentation module is used to input the medical image to be processed into the trained BMSFE-SegNet network for image segmentation.

[0075] As an implementation method of an embodiment of the present invention, the BMSFE-SegNet network includes: an encoder, a multi-scale receptive field module MRFB module, a medical image edge prediction module and a medical image segmentation module; wherein the medical image segmentation module includes: an edge-aware multi-scale feature fusion module RMSF and an edge-aware attention enhancement module BAE.

[0076] As an implementation method of the present invention, the total loss function of training the BMSFE-SegNet network is for:

[0077] Embodiment 3:

[0078] An embodiment of the present invention further provides a medical image segmentation system, comprising: a memory and a processor, wherein the memory stores a computer program executed by the processor, and the computer program executes a medical image segmentation method when executed by the processor.

[0079] Embodiment 4:

[0080] An embodiment of the present invention further provides a storage medium, on which a computer program is stored, and the computer program executes the medical image segmentation method when running.

[0081] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A medical image segmentation method, characterized in that: include: Step S1, obtaining historical medical images; Step S2, training a BMSFE-SegNet network according to historical medical images; the BMSFE-SegNet network comprises: an encoder, a multi-scale receptive field module MRFB module, a medical image edge prediction module and a medical image segmentation module; wherein the medical image segmentation module comprises: an edge-aware multi-scale feature fusion module BMSF and an edge-aware attention enhancement module BAE; Step S3, inputting the medical image to be processed into the trained BMSFE-SegNet network for image segmentation; Among them, historical medical images Through the four-layer encoder, the pyramid feature map is obtained Among them, i represents the number of network layers, C i Represents the number of channels of the feature map of the i-th layer; U i Input into the multi-scale receptive field module for feature aggregation, and get U i ′ will be input into the medical image edge prediction module and the medical image segmentation module respectively, and U i 'Perform 3×3 convolution operation, batch normalization operation and ReLU activation function; It is divided into two branches. The first branch passes through 1 × 1 After the convolution operation, it becomes a single-channel feature. Upsampled to With the same spatial size, the four-layer single-channel features are aggregated through the Sigmoid function to obtain the edge prediction output E0; the second branch obtains the edge perception feature through a 1×1 convolution operation Used to guide medical image segmentation; The medical image segmentation module extracts the U i ′ and the BMSF module of the previous layer after fusion Input to the BMSF fusion module and perceive features at the edge The BMSF module of the current layer is input under the action of to perform multi-scale feature alignment and fusion to obtain the fused features Edge-aware features Also as a guiding feature, Input the BAE module to obtain the attention enhanced feature map V i ; V i After convolution operation and Sigmoid function, the medical image segmentation prediction map E is obtained. i , the highest resolution E1 is used as the final medical image segmentation output in the BMSFE-SegNet network training stage.

2. The medical image segmentation method according to claim 1, characterized in that: The total loss function for training the BMSFE-SegNet network for: in, is the IoU loss, Loss for BCE is the edge prediction graph loss.

3. A medical image segmentation device, characterized in that: include: An acquisition module, used to acquire historical medical images; A training module, used for training a BMSFE-SegNet network according to historical medical images; the BMSFE-SegNet network comprises: an encoder, a multi-scale receptive field module MRFB module, a medical image edge prediction module and a medical image segmentation module; wherein the medical image segmentation module comprises: an edge-aware multi-scale feature fusion module BMSF and an edge-aware attention enhancement module BAE; The segmentation module is used to input the medical image to be processed into the trained BMSFE-SegNet network for image segmentation; Among them, historical medical images Through the four-layer encoder, the pyramid feature map is obtained Among them, i represents the number of network layers, C i Represents the number of channels of the feature map of the i-th layer; U i Input into the multi-scale receptive field module for feature aggregation, and get U i ′ will be input into the medical image edge prediction module and the medical image segmentation module respectively, and U i 'Perform 3×3 convolution operation, batch normalization operation and ReLU activation function; It is divided into two branches. The first branch becomes a single-channel feature after a 1×1 convolution operation. Upsampled to With the same spatial size, the four-layer single-channel features are aggregated through the Sigmoid function to obtain the edge prediction output E0; the second branch obtains the edge perception feature through a 1×1 convolution operation Used to guide medical image segmentation; The medical image segmentation module extracts the U i ′ and the BMSF module of the previous layer after fusion Input to the BMSF fusion module and perceive features at the edge The BMSF module of the current layer is input under the action of to perform multi-scale feature alignment and fusion to obtain the fused features Edge-aware features Also as a guiding feature, Input the BAE module to obtain the attention enhanced feature map V i ; V i After convolution operation and Sigmoid function, the medical image segmentation prediction map E is obtained. i , the highest resolution E1 is used as the final medical image segmentation output in the BMSFE-SegNet network training stage.

4. The medical image segmentation device according to claim 3, characterized in that: The total loss function for training the BMSFE-SegNet network for: in, is the IoU loss, Loss for BCE is the edge prediction graph loss.

5. A medical image segmentation system, characterized in that: include: A memory and a processor, wherein the memory stores a computer program executed by the processor, and when the computer program is executed by the processor, the medical image segmentation method according to any one of claims 1 to 2 is executed.

6. A storage medium, characterized in that: The storage medium stores a computer program, which executes the medical image segmentation method according to any one of claims 1 to 2 when running.

Images