Medical image segmentation method and device, electronic equipment and readable storage medium

Abstract

The invention provides a medical image segmentation method and device, electronic equipment and a readable storage medium, and belongs to the technical field of image processing, and the method comprises the steps: carrying out the multi-scale local feature extraction and global feature extraction of a medical image, and obtaining a first local feature map and a first global feature map corresponding to each scale; performing feature fusion on the first local feature map and the first global feature map corresponding to each scale to obtain a first fusion feature map corresponding to each scale; fusing the first fusion feature maps corresponding to the scales to obtain a second fusion feature map; and decoding the second fusion feature map to obtain an image segmentation result. According to the medical image segmentation method and device, the electronic equipment and the readable storage medium provided by the invention, the segmentation precision of the medical image can be improved.

Description

Technical Field

[0001] The present disclosure belongs to the field of image processing technology, and more specifically, relates to a medical image segmentation method and device, an electronic device, and a readable storage medium. Background Art

[0002] Medical image segmentation is one of the core tasks of computer-aided diagnosis systems (CADs), and its accuracy directly affects the diagnosis, treatment, and prognosis of patients. Although manual annotation of lesions or targets by clinical experts can provide high-quality segmentation results, this method is time-consuming, labor-intensive, and highly dependent on the knowledge and experience of experts, making it prone to subjective errors. Therefore, automated and accurate medical image segmentation technology is essential to improve diagnostic efficiency, reduce doctors' workload, and achieve standardized and objective medical image analysis. Medical image segmentation is a key task in computer-aided diagnosis, and is of great significance for the early diagnosis of diseases and the formulation of treatment plans.

[0003] With the rapid development of computer hardware and deep learning technology, automatic paired medical image segmentation has shown great promise. In particular, convolutional neural networks (CNNs) have shown excellent performance in medical image segmentation tasks and significantly improved segmentation accuracy. The UNet architecture and its U-shaped derivatives have gradually become the standard paradigm for medical image segmentation, achieving state-of-the-art performance in various tasks. Drawing inspiration from U-Net, subsequent models have improved segmentation accuracy by designing unique feature enhancement modules, combining attention mechanisms, and improving U-Net structures. However, due to the locality of convolution operations, CNNs have limitations in modeling global information, resulting in poor performance in some complex tasks. Traditional CNN-based segmentation methods mainly rely on local feature extraction and have difficulty capturing long-range dependencies in images. Although the receptive field can be expanded by stacking more convolutional layers, it also brings problems such as increased computational complexity and optimization difficulties. Summary of the invention

[0004] The purpose of the present disclosure is to provide a medical image segmentation method and device, an electronic device, and a readable storage medium to improve segmentation accuracy.

[0005] A first aspect of the embodiments of the present disclosure provides a medical image segmentation method, comprising: Performing multi-scale local feature extraction and global feature extraction on the medical image to obtain a first local feature map and a first global feature map corresponding to each scale; The first local feature map corresponding to each scale and the first global feature map are feature-fused to obtain the first fused feature map corresponding to each scale; the first fused feature maps corresponding to each scale are fused to obtain the second fused feature map; The second fused feature map is decoded to obtain an image segmentation result.

[0006] A second aspect of the embodiments of the present disclosure provides a medical image segmentation device, comprising: A dual-branch feature extraction module is used to extract multi-scale local features of medical images to obtain a first local feature map corresponding to each scale; The dual-branch fusion module is used to extract multi-scale global features of the medical image to obtain a first global feature map corresponding to each scale; perform feature fusion on the first global feature map corresponding to each scale and the first local feature map to obtain a first fused feature map corresponding to each scale; A decoder is used to decode the first fused feature map to obtain an image segmentation result.

[0007] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor implements the steps of the above-mentioned medical image segmentation method when executing the computer program.

[0008] According to a fourth aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned medical image segmentation method are implemented.

[0009] The medical image segmentation method and device, electronic device, and readable storage medium provided by the embodiments of the present disclosure have the following beneficial effects: In the disclosed embodiment, by performing multi-scale local feature extraction on the medical image, the first local feature map obtained can capture the detail information of the local area in the medical image; by performing multi-scale global feature extraction on the medical image, the first global feature map obtained can capture a wider range of contextual information. The first local feature map corresponding to each scale and the first global feature map are subjected to feature fusion, and the obtained first fused feature map can comprehensively reflect the local and global features. The first fused feature maps corresponding to each scale are fused so that features from low to high layers can be fully utilized. In summary, the expression ability and segmentation performance of the model are enhanced by extracting and fusing multi-scale features. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0011] Figure 1 A schematic diagram of a flow chart of a medical image segmentation method provided by an embodiment of the present disclosure; Figure 2 A block diagram of the principle of a medical image segmentation device provided by an embodiment of the present disclosure; Figure 3 A principle block diagram of a dual-branch fusion module provided in an embodiment of the present disclosure; Figure 4 A block diagram of a decoder according to an embodiment of the present disclosure; Figure 5 A summary diagram of experimental results provided for an embodiment of the present disclosure; Figure 6 A schematic block diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0012] In the following description, specific details such as specific system structures and technologies are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present disclosure. However, it should be clear to those skilled in the art that the present disclosure may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obstructing the description of the present disclosure with unnecessary details.

[0013] In order to make the purpose, technical solutions and advantages of the present disclosure more clear, specific embodiments will be described below in conjunction with the accompanying drawings.

[0014] Please refer to Figure 1 , Figure 1 A schematic flow chart of a medical image segmentation method provided in an embodiment of the present disclosure, the method is applied to a medical image segmentation device, and the method includes: S101: Perform multi-scale local feature extraction and global feature extraction on the medical image to obtain a first local feature map and a first global feature map corresponding to each scale.

[0015] Please refer to Figure 2In this embodiment, the dual-branch feature extraction module in the medical image segmentation device includes a spatial branch and a state-space branch. Based on the spatial branch, multi-scale local feature extraction is performed on the medical image. The spatial branch uses ConvNeXt-T as its main feature extraction network. ConvNeXt combines the advantages of Transformer and CNN architectures, maintaining the efficiency and simplicity of traditional CNN, and integrating the powerful feature representation ability and flexibility of Transformer. Compared with VGG, ResNet, DenseNet, etc., ConvNeXt uses a larger convolution kernel (7×7) and introduces a concept similar to the attention mechanism, aiming to balance the global and local features of image processing. This method enables the network to achieve stronger feature representation capabilities when processing image features of different scales.

[0016] In this embodiment, the spatial branch includes four ConvNeXt Blocks, where: The first ConvNeXt Block: processes features of size h×768; The second ConvNeXt Block: processes features of size h×384; The third ConvNeXt Block: processes features of size h×192; The fourth ConvNeXt Block: processes features of size h×96; There is a downsampling operation between each Block.

[0017] The input image Image ∈ Rb×3×h×w is processed and the first local feature maps of different scales are generated through four stages: ∈ Rb×96×h / 4×w / 4; ∈ Rb×192×h / 8×w / 8; ∈ Rb×384×h / 16×w / 16; ∈ Rb×768×h / 32×w / 32.

[0018] In this embodiment, a state space branch is used to perform multi-scale global feature extraction on medical images. The state space branch integrates the core function SS2D in VMamba and is responsible for capturing more extensive contextual information.

[0019] Specifically, the state space branch includes four VSSBlocks. The input of the VSS Block passes through the initial linear embedding layer, and the output is divided into two information streams. One stream passes through a 3 × 3 deep convolution layer and then enters the core SS2D module through the Silu activation function. The output of SS2D passes through a normalization layer and then is added to the output of the other information stream, which is activated by Silu. Due to the causal nature of VMamba, position embedding bias is not used. Among them: The first VSS Block: processes features of size h×768; The second VSS Block: processes features of h×384 size; The third VSS Block: processes features of size h×192; The fourth VSS Block: processes features of size h×96; There is a downsampling operation between each Block.

[0020] The input image is also processed in four stages to generate the first global feature maps of different scales: ∈ Rb×96×h / 4×w / 4; ∈ Rb×192×h / 8×w / 8; ∈ Rb×384×h / 16×w / 16; ∈ Rb×768×h / 32×w / 32; The processing process of each stage can be expressed as: patches = PatchPartition(images); = VSSBlock(patches); =VSSBlock(DownSample( )); =VSSBlock(DownSample( )); =VSSBlock(DownSample( )).

[0021] S102: Fusing the first local feature map and the first global feature map corresponding to each scale to obtain a first fused feature map corresponding to each scale; fusing the first fused feature map corresponding to each scale to obtain a second fused feature map.

[0022] In this embodiment, local feature extraction helps to capture detailed information of local areas in medical images, such as the specific texture of a certain organ, tiny lesions and other details in a CT image; while global feature extraction can grasp the characteristics of the image as a whole, such as the general shape of the entire body part, the relative position relationship between different organs, etc. By fusing the first local feature map corresponding to each scale with the first global feature map, the information in the medical image can be more comprehensively mined.

[0023] In this embodiment, the feature maps of each scale can represent the image features at different resolutions, and the features at different scales have different effects on medical image segmentation. For example, the features at a high-resolution scale may be more conducive to discovering tiny diseased tissues, but the features at a low-resolution scale can quickly locate the approximate position of the organ and provide a framework for subsequent fine segmentation.

[0024] By fusing the first fusion feature maps corresponding to each scale again to obtain the second fusion feature map, cross-scale feature integration is achieved, and the advantageous features of different scales are fused. For example, the fine lesion detail features at a high-resolution scale are fused with the overall layout features of organs at a low-resolution scale, which can provide a rich and comprehensive feature basis for the final segmentation result.

[0025] S103: Decode the second fused feature map to obtain an image segmentation result.

[0026] In this embodiment, by decoding the second fused feature map, for example, by upsampling to restore the resolution, converting the features into pixel category information, etc., the second fused feature map can be converted into a specific medical image segmentation situation, clearly dividing different tissues, lesions and other areas, and obtaining an image segmentation result.

[0027] From the above, it can be concluded that in this embodiment, by performing multi-scale local feature extraction on medical images, the first local feature map obtained can capture the detailed information of the local area in the medical image; by performing multi-scale global feature extraction on medical images, the first global feature map obtained can capture a wider range of contextual information. The first local feature map corresponding to each scale and the first global feature map are fused to obtain the first fused feature map that can comprehensively reflect the local and global features. The first fused feature maps corresponding to each scale are fused so that the features from low to high layers can be fully utilized. In summary, the expression ability and segmentation performance of the model are enhanced through the extraction and fusion of multi-scale features.

[0028] Please refer to Figure 3 In one embodiment of the present disclosure, the first local feature map corresponding to each scale and the first global feature map are subjected to feature fusion to obtain the first fused feature map corresponding to each scale, including: Perform channel pooling on the first local feature map, and aggregate the information of each channel to obtain a second local feature map; The first local feature map is passed through an activation function to output a third local feature map; Obtain a fourth local feature map based on the second local feature map and the third local feature map; Multiply the first local feature map and the first global feature map point by point to obtain a third fused feature map; Processing the first global feature map based on the channel attention module and the spatial attention module to obtain a second global feature map; The second global feature map, the third fused feature map and the fourth local feature map are fused to obtain a first fused feature map.

[0029] In this embodiment, for the first local feature generated by the spatial branch , first perform channel pooling, and then learn ( Figure 3 The CBR module in the example, convolution + normalization + ReLU) is used to aggregate information from different channels, thereby enhancing the robustness of the model. Next, this embodiment applies the Sigmoid function to the input features and multiplies the results, thereby compressing the feature representation while retaining important information. This method helps retain features that are beneficial to medical segmentation tasks while suppressing noise that may affect the segmentation performance of the model. The process can be expressed as: = Sigmoid( ) × Conv(Channelpool( )) At the same time, the Hadamard product is directly applied to perform point-by-point multiplication of features Mf and Cf, followed by a convolution operation to achieve feature learning. The Hadamard product emphasizes the features that are considered important in the two branches, pointing to relevant information. The process can be expressed as: = Conv( ⊙ ) The first global feature for state-space branching , based on the channel attention module and the spatial attention module, the first global feature map is processed to obtain the second global feature map .

[0030] Finally, the three features obtained , and They are fused by splicing and input into the residual module for automatic feature integration: = Residual(Concat( , , )).

[0031] It can be concluded from the above that this embodiment further refines the global features through two attention strategies, and then performs channel-level feature fusion and residual connection. Multiple feature fusion methods can effectively extract and integrate feature information at different levels.

[0032] In one embodiment of the present disclosure, the first global feature map is processed based on the channel attention module and the spatial attention module to obtain the second global feature map, including: Input the first global feature map into the channel attention module to obtain a third global feature map; Obtain a fourth global feature map based on the third global feature map and the first global feature map; Input the fourth global feature map into the spatial attention module to obtain a fifth global feature map; A second global feature map is obtained based on the fourth global feature map and the fifth global feature map.

[0033] In this embodiment, the channel attention module includes: (1) Feature pooling: including maximum pooling operation and average pooling operation; (2) Feature processing: including shared MLP processing, addition operation and Sigmoid activation function; (3) Output: Generate channel attention feature map.

[0034] The spatial attention module includes: (1) Input: channel redefinition features; (2) Processing flow: [max pooling, average pooling] operations, convolutional layer processing, and Sigmoid activation function; (3) Output: Generate spatial attention feature map.

[0035] From the above, it can be concluded that this embodiment adopts the channel and space attention mechanism (CBAM) to more effectively select the medical image features extracted by the state space branch from the two dimensions of channel and space. , discarding irrelevant information.

[0036] Please refer to Figure 4In one embodiment of the present disclosure, the first fused feature maps corresponding to each scale are fused to obtain a second fused feature map, including: Dividing the first fused feature map corresponding to each scale into a first fine-grained feature map and a first coarse-grained feature map; wherein the scale of the first fine-grained feature map is larger than the size of the first coarse-grained feature map; The first coarse-grained feature map is sequentially subjected to an adaptive maximum pooling layer and a convolution process to obtain a second coarse-grained feature map; The first fine-grained feature map and the second coarse-grained feature map corresponding to each scale are fused to obtain a second fused feature map.

[0037] In this embodiment, in the first fusion feature map corresponding to each scale, ( , , ) are considered as fine-grained (low-level) features that provide structural outlines and contextual information such as localization, while ( ) are coarse-grained (high-level) features with richer and more precise semantic details, which are crucial for generating accurate segmentation predictions.

[0038] When fusing the first fused feature maps corresponding to each scale, firstly, the first coarse-grained feature map ( ) A second coarse-grained feature map is obtained through a PPU unit (pyramid pooling unit), and then the second coarse-grained feature map is fused with the first fine-grained feature map corresponding to each scale to obtain a second fused feature map.

[0039] Specifically, in the PPU unit, it is processed through three adaptive maximum pooling layers to adjust it to the specified size, and then learned through convolution. This method captures the detailed texture of coarse-grained features at different scales, enhances the semantic information at each level, and integrates features of different scales to improve the accuracy and robustness of segmentation.

[0040] The specific operation details of PPU are as follows: (1) Using pooling kernels of different sizes for feature extraction; (2) Align features at different scales by upsampling; (3) Concatenate all features and perform convolution fusion; A1 = Conv(AMPamp=1( )) ; A2 = Conv(AMPamp=3( )) ; A3 = Conv(AMPamp=6( )) ; = Concat( , up(A1), up(A2), up(A3); Among them, AMP represents the adaptive maximum pooling operation, amp represents the specified size of the feature, and up represents upsampling the features generated by AMP to ( ) size for concatenation. Subsequently, the features generated by the PPU unit are combined with the low-level features ( , , ) aggregation. Finally, features at all scales are fused to generate the prediction map for the segmentation task.

[0041] It can be concluded from the above that this embodiment effectively combines coarse-grained and fine-grained feature information through multi-scale feature extraction and progressive pooling, thereby enhancing the feature extraction capability.

[0042] In one embodiment of the present disclosure, before performing feature fusion on the first fine-grained feature map and the second coarse-grained feature map corresponding to each scale, the medical image segmentation method further includes: Convolutionally processing each first fine-grained feature map through multiple paths to obtain multiple second fine-grained feature maps corresponding to each first fine-grained feature map; The multiple second fine-grained feature maps are fused to obtain an updated first fine-grained feature map.

[0043] In this embodiment, before the first fine-grained feature map corresponding to each scale is feature-fused with the second coarse-grained feature map, the first fine-grained feature map corresponding to each scale is simultaneously input into a separation dilation unit (SDU). The first fine-grained feature map is processed through four different paths in the SDU: the first path uses (1x1) convolution, while the second, third, and fourth paths combine standard convolution, striped convolution, and dilated convolution, with dilation rates of 3, respectively, and the convolution kernel sizes of each path are 3, 5, and 7, respectively.

[0044] Specifically, the structure of the separation expansion unit includes: (1) Input processing: initial Conv1×1 convolution; (2) Three parallel branches: First branch: Conv3×3 (d=3), Conv1×1; Second branch: Conv1×5, Conv5×5 (d=3); The third branch: Conv1×7, Conv7×7 (d=3); (3) Output processing: All branch results are merged through Concat and output .

[0045] = Conv1×1( ) ; = D3Conv3×3(Conv3×1(Conv1×3(Conv3×3( )))) ; = D3Conv5×5(Conv5×1(Conv1×5(Conv5×5( )))) ; = D3Conv7×7(Conv7×1(Conv1×7(Conv7×7( )))) ; = Concat( , , , , ).

[0046] It can be concluded from the above that the decoder of this embodiment integrates a separation dilation unit (SDU) and a pyramid pooling unit (PPU) to more accurately identify detailed textures and structural features in images of different sizes, thereby improving the accuracy of the prediction map.

[0047] In one embodiment of the present disclosure, before the first coarse-grained feature map is sequentially subjected to an adaptive maximum pooling layer and a convolution process, the medical image segmentation method further includes: The first coarse-grained feature map is subjected to convolution processing of multiple paths respectively to obtain multiple third fine-grained feature maps corresponding to the first coarse-grained feature map; The multiple third fine-grained feature maps are fused to obtain an updated first coarse-grained feature map.

[0048] In this embodiment, the same processing method as the first fine-grained feature map is adopted. Before the first coarse-grained feature map is sequentially subjected to the adaptive maximum pooling layer and the convolution processing, the first coarse-grained feature map is subjected to the convolution processing of multiple paths respectively. By using the split hole unit to expand the receptive field, the feature extraction capability can be further enhanced.

[0049] In one embodiment of the present disclosure, the convolution processing of multiple paths includes: 1x1 convolution, standard convolution, strip convolution and dilated convolution.

[0050] In summary, the processing flow of the decoder for mixed coarse and fine features in this embodiment is as follows: (1) Top branch ( ): SDU module processes input, passes through CBR module, performs addition operation, passes through CBR module again and outputs ; (2) Middle branch (D f2 ): Similar processing flow to the top branch, SDU → CBR → Addition → CBR, output .

[0051] (3) The third branch (D f3 ): Same process, SDU → CBR → addition → CBR, Output .

[0052] (4) Bottom branch ( and PPU): SDU module processing; Enter the PPU (Progressive Pooling Unit) module; Contains three different AMP (AdaptiveMaxPool) layers: AMP=1, AMP=3, and AMP=6; Characteristic dimensions: B×C×H×W.

[0053] (5) Feature fusion: All branches are connected through the Concat operation; the final output is the prediction graph.

[0054] The method of this embodiment and the existing method are used to conduct experiments on four medical image datasets: BUSI, DDTI, TN3K and ISIC2016. The experimental results are shown in the following table. Figure 5 As shown. According to the experimental results, the method of the present invention has significant advantages over the existing methods: In the BUSI dataset, YMamba proposed in this embodiment achieves state-of-the-art results in both indicators. It is 0.01 higher than PolypPVT in Dice indicator and reaches 0.768 in IoU, which is 1.1% and 1.8% higher than EH-Former and NPDNet respectively. Although CMUNeXt performs well in global context modeling, YMamba improves both indicators by an average of 6.75%, including a 7.4% increase in IoU.

[0055] In the DDTI dataset, EH-Former also achieved SOTA in two evaluation indicators. However, YMamba in this embodiment further surpassed or was on par with EH-Former in these two indicators, with Dice improved by 1.2% and IoU improved by 0.8%. It is worth noting that compared with VMUNet, which also uses VMamba as the backbone network, YMamba in this embodiment exceeded Dice and IoU indicators by 0.11 and 0.19 respectively.

[0056] For TN3K, YMamba performs well in both Dice and IoU, with scores of 0.859 and 0.778. Compared with M2SNet, the method of this embodiment improves Dice by 0.019. In the ISIC2016 dataset, YMamba performs best in IoU, only 0.006 lower than the leading VMUNet. It is worth mentioning that compared with the recently proposed CMUNet, the method of this embodiment improves the average performance by 11.5%, and especially improves by 15% in IoU. Compared with PraNet, which also uses a method combining coarse and fine segmentation, the method of this embodiment improves by 3.8% in Dice and 5.7% in IoU. These results show that YMamba has made significant progress in medical image segmentation tasks, demonstrating its wide applicability and superior performance on a variety of datasets.

[0057] Corresponding to the medical image segmentation method of the above embodiment, Figure 2 This is a block diagram of a medical image segmentation device provided by an embodiment of the present disclosure. For ease of explanation, only the parts related to the embodiment of the present disclosure are shown. Figure 2 The medical image segmentation device includes: a dual-branch feature extraction module, a dual-branch fusion module and a decoder. The dual-branch feature extraction module is used to perform multi-scale local feature extraction and global feature extraction on the medical image to obtain a first local feature map and a first global feature map corresponding to each scale; A dual-branch fusion module is used to fuse the first local feature map corresponding to each scale with the first global feature map to obtain a first fused feature map corresponding to each scale; and fuse the first fused feature maps corresponding to each scale to obtain a second fused feature map; The decoder is used to decode the second fused feature map to obtain an image segmentation result.

[0058] In one embodiment of the present disclosure, the dual-branch fusion module is specifically used for: Perform channel pooling on the first local feature map, and aggregate the information of each channel to obtain a second local feature map; The first local feature map is passed through an activation function to output a third local feature map; Obtain a fourth local feature map based on the second local feature map and the third local feature map; Multiply the first local feature map and the first global feature map point by point to obtain a third fused feature map; Processing the first global feature map based on the channel attention module and the spatial attention module to obtain a second global feature map; The second global feature map, the third fused feature map and the fourth local feature map are fused to obtain a first fused feature map.

[0059] In one embodiment of the present disclosure, the dual-branch fusion module is further used to: Input the first global feature map into the channel attention module to obtain a third global feature map; Obtain a fourth global feature map based on the third global feature map and the first global feature map; Input the fourth global feature map into the spatial attention module to obtain a fifth global feature map; A second global feature map is obtained based on the fourth global feature map and the fifth global feature map.

[0060] In one embodiment of the present disclosure, the dual-branch fusion module is specifically used for: Dividing the first fused feature map corresponding to each scale into a first fine-grained feature map and a first coarse-grained feature map; wherein the scale of the first fine-grained feature map is larger than the size of the first coarse-grained feature map; The first coarse-grained feature map is sequentially subjected to an adaptive maximum pooling layer and a convolution process to obtain a second coarse-grained feature map; The first fine-grained feature map and the second coarse-grained feature map corresponding to each scale are fused to obtain a second fused feature map.

[0061] In one embodiment of the present disclosure, the dual-branch fusion module is further used to: Convolutionally processing each first fine-grained feature map through multiple paths to obtain multiple second fine-grained feature maps corresponding to each first fine-grained feature map; The multiple second fine-grained feature maps are fused to obtain an updated first fine-grained feature map.

[0062] In one embodiment of the present disclosure, the dual-branch fusion module is further used to: The first coarse-grained feature map is subjected to convolution processing of multiple paths respectively to obtain multiple third fine-grained feature maps corresponding to the first coarse-grained feature map; The multiple third fine-grained feature maps are fused to obtain an updated first coarse-grained feature map.

[0063] In one embodiment of the present disclosure, the convolution processing of multiple paths includes: 1x1 convolution, standard convolution, strip convolution and dilated convolution.

[0064] See also Figure 6 , Figure 6 A schematic block diagram of an electronic device provided by an embodiment of the present disclosure. Figure 6 The electronic device 300 in the embodiment shown may include: one or more processors 301, one or more input devices 302, one or more output devices 303 and one or more memories 304. The processors 301, input devices 302, output devices 303 and memories 304 communicate with each other via a communication bus 305. The memory 304 is used to store computer programs, which include program instructions. The processor 301 is used to execute the program instructions stored in the memory 304. Among them, the processor 301 is configured to call the program instructions to execute the functions of each module / unit in the above-mentioned device embodiments.

[0065] It should be understood that in the embodiment of the present disclosure, the processor 301 may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0066] The input device 302 may include a touch panel, a fingerprint collection sensor (for collecting the user's fingerprint information and fingerprint direction information), a microphone, etc., and the output device 303 may include a display (LCD, etc.), a speaker, etc.

[0067] The memory 304 may include a read-only memory and a random access memory, and provide instructions and data to the processor 301. A portion of the memory 304 may also include a non-volatile random access memory. For example, the memory 304 may also store information about the device type.

[0068] In a specific implementation, the processor 301, input device 302, and output device 303 described in the embodiments of the present disclosure can execute the implementation methods described in the first and second embodiments of the medical image segmentation method provided in the embodiments of the present disclosure, and can also execute the implementation methods of the electronic device described in the embodiments of the present disclosure, which will not be repeated here.

[0069] In another embodiment of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by the processor, all or part of the processes in the above-mentioned embodiment method are implemented, and the computer program can also be completed by instructing the relevant hardware through the computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor, the steps of each of the above-mentioned method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0070] The computer-readable storage medium may be an internal storage unit of the electronic device of any of the aforementioned embodiments, such as a hard disk or memory of the electronic device. The computer-readable storage medium may also be an external storage device of the electronic device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device. Furthermore, the computer-readable storage medium may also include both an internal storage unit of the electronic device and an external storage device. The computer-readable storage medium is used to store computer programs and other programs and data required by the electronic device. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0071] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this disclosure.

[0072] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the electronic devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0073] In the several embodiments provided in the present application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces or units, or it can be an electrical, mechanical or other form of connection.

[0074] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present disclosure.

[0075] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0076] The above are only specific embodiments of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present disclosure, and these modifications or replacements should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A medical image segmentation method, characterized in that: include: Performing multi-scale local feature extraction and global feature extraction on the medical image to obtain a first local feature map and a first global feature map corresponding to each scale; Perform feature fusion on the first local feature map corresponding to each scale and the first global feature map to obtain a first fused feature map corresponding to each scale; The first fused feature maps corresponding to each scale are fused to obtain a second fused feature map; The second fused feature map is decoded to obtain an image segmentation result.

2. The medical image segmentation method according to claim 1, characterized in that: The step of fusing the first local feature map corresponding to each scale with the first global feature map to obtain the first fused feature map corresponding to each scale comprises: Perform channel pooling on the first local feature map, and aggregate information of each channel to obtain a second local feature map; Pass the first local feature map through an activation function to output a third local feature map; Obtain a fourth local feature map based on the second local feature map and the third local feature map; Multiplying the first local feature map and the first global feature map point by point to obtain a third fused feature map; Processing the first global feature map based on a channel attention module and a spatial attention module to obtain a second global feature map; The second global feature map, the third fused feature map and the fourth local feature map are fused to obtain the first fused feature map.

3. The medical image segmentation method according to claim 2, characterized in that: The processing of the first global feature map based on the channel attention module and the spatial attention module to obtain a second global feature map includes: Inputting the first global feature map into a channel attention module to obtain a third global feature map; Obtain a fourth global feature map based on the third global feature map and the first global feature map; Inputting the fourth global feature map into the spatial attention module to obtain a fifth global feature map; The second global feature map is obtained based on the fourth global feature map and the fifth global feature map.

4. The medical image segmentation method according to claim 1, characterized in that: The step of fusing the first fused feature maps corresponding to each scale to obtain a second fused feature map includes: Dividing the first fused feature map corresponding to each scale into a first fine-grained feature map and a first coarse-grained feature map; wherein the scale of the first fine-grained feature map is larger than the size of the first coarse-grained feature map; The first coarse-grained feature map is sequentially subjected to an adaptive maximum pooling layer and a convolution process to obtain a second coarse-grained feature map; The first fine-grained feature map and the second coarse-grained feature map corresponding to each scale are fused to obtain a second fused feature map.

5. The medical image segmentation method according to claim 4, characterized in that: Before performing feature fusion on the first fine-grained feature map and the second coarse-grained feature map corresponding to each scale, the medical image segmentation method further includes: Convolutionally processing each first fine-grained feature map through multiple paths to obtain multiple second fine-grained feature maps corresponding to each first fine-grained feature map; The multiple second fine-grained feature maps are fused to obtain an updated first fine-grained feature map.

6. The medical image segmentation method according to claim 4, characterized in that: Before the first coarse-grained feature map is sequentially subjected to an adaptive maximum pooling layer and a convolution process, the medical image segmentation method further includes: The first coarse-grained feature map is subjected to convolution processing of multiple paths respectively to obtain multiple third fine-grained feature maps corresponding to the first coarse-grained feature map; The multiple third fine-grained feature maps are fused to obtain an updated first coarse-grained feature map.

7. The medical image segmentation method according to claim 5 or 6, characterized in that: The convolution processing of the multiple paths includes: 1x1 convolution, standard convolution, strip convolution and dilated convolution.

8. A medical image segmentation device, characterized in that: include: A dual-branch feature extraction module is used to extract multi-scale local features of medical images to obtain a first local feature map corresponding to each scale; The dual-branch fusion module is used to extract multi-scale global features of the medical image to obtain a first global feature map corresponding to each scale; perform feature fusion on the first global feature map corresponding to each scale and the first local feature map to obtain a first fused feature map corresponding to each scale; A decoder is used to decode the first fused feature map to obtain an image segmentation result.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images