Medical image segmentation method and model based on dynamic layer-view-scale fusion

Abstract

The application discloses a medical image segmentation method and model based on dynamic layer-view-scale fusion, which comprises initial feature extraction on an input image to obtain an initial feature map, multi-round encoding iterative processing of the initial feature map in sequence to obtain encoding feature maps of various scales in sequence, self-adaptive scale calibration based on view arbitration on the encoding feature map output by the last round of encoding iteration in the encoding feature maps of various scales to obtain a feature map after self-adaptive receptive field calibration, multi-round decoding iterative processing of the feature map after self-adaptive receptive field calibration in sequence, weighting of the output after processing through a cross-scale attention gate to obtain decoding feature maps of various scales, convolution of the decoding feature map output by the last round of decoding iteration in the decoding feature maps of various scales through a prediction head to obtain a segmentation result, realization of pixel-level dynamic routing and self-adaptive receptive field calibration, and accurate alignment of feature representation and the diversity and heterogeneity of anatomical structures.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and in particular to a medical image segmentation method and model based on dynamic layer-view-scale fusion. Background Technology

[0002] Automated medical image segmentation is fundamental to quantitative analysis in computational imaging, crucial for lesion delineation, volume assessment, and longitudinal efficacy evaluation. In practical clinical applications, models must operate stably under highly heterogeneous conditions, including handling multimodal inputs (such as magnetic resonance imaging (MRI), optical coherence tomography (OCT), and computed tomography (CT)), addressing domain shifts from multicenter data, overcoming low tissue contrast, handling severe class imbalance, and adapting to significant differences in anatomical scale. These factors collectively reveal a persistent core methodological challenge, largely independent of the type of base model chosen: how to adaptively select, weight, and fuse information from different feature levels (layers), complementary representational perspectives (views), and different receptive field ranges (scales).

[0003] Despite recent significant advancements in model architecture and training paradigms—for example, convolutional encoders and U-Net-style decoders remain competitive due to their effectiveness, Transformer variants can capture long-range dependencies, state-space models (such as Mamba) offer linear-complexity sequence modeling capabilities, and semi-supervised / weakly supervised and few-shot learning strategies alleviate the problem of scarce labeled data; basic model adaptation techniques are also exploring their application in the medical field—most existing solutions still rely on static or heuristic approaches for feature fusion. Specifically, cross-layer connections often indiscriminately merge features, leading to the propagation of redundant information and background noise; multi-scale modules typically weight each branch equally, failing to dynamically align the receptive field with the size of local targets; and attention mechanisms are often applied sequentially or from a single perspective, making it difficult to simultaneously optimize global context and local boundary details at the same location. These observations suggest that a unified and input-adaptive fusion principle capable of cross-layer, cross-scale, and cross-view fusion remains to be explored. Summary of the Invention

[0004] The purpose of this invention is to address the problem in the prior art that there is a lack of a unified fusion mechanism that can dynamically and pixel-level weigh cross-layer features, multi-scale branches, and global-local view contributions, which makes it impossible to accurately align feature representations with the diversity and heterogeneity of anatomical structures. Therefore, this invention proposes a medical image segmentation method and model based on dynamic layer-view-scale fusion.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: A medical image segmentation method based on dynamic layer-view-scale fusion includes: Step 1: Perform initial feature extraction on the input image to obtain an initial feature map; Step 2: The initial feature map is processed through multiple rounds of encoding iteration. In each round of encoding iteration, dynamic dense fusion and pooling downsampling are performed on the current input feature map to obtain encoded feature maps at various scales. Step 3: Perform view arbitration-based adaptive scale calibration on the encoded feature maps output from the last round of encoding iterations at each scale to obtain the feature maps after adaptive receptive field calibration; Step 4: Perform multiple rounds of decoding iteration on the feature map after adaptive receptive field calibration. In each round of decoding iteration, perform dynamic dense fusion processing and pooling upsampling processing on the current input feature map. Then, weight the processed output through cross-scale attention gates to obtain decoded feature maps at each scale. Step 5: Input the decoded feature map output from the last decoding iteration of the decoded feature maps at each scale into the prediction head for convolution to obtain the segmentation result.

[0006] Furthermore, step 3 specifically includes the following steps: Step 31: Perform dilated convolution and global pooling operations of 1×1, 1×3, 3×1, and 3×3 on the encoded feature map output from the last round of encoding iteration to obtain five branch feature maps; Step 32: Perform view arbitration on the five branch feature maps respectively to obtain five weighted feature maps; Step 33: Concatenate the five weighted feature maps to obtain the concatenated feature map; Step 34: The concatenated feature maps are processed sequentially by 1×1 convolution, batch normalization, and ReLU function to obtain the feature maps after adaptive receptive field calibration.

[0007] Furthermore, the specific steps of performing view arbitration on each branch feature map in step 33 include: Step 331: Perform 3×3 grouped convolution and global pooling on the branch feature maps respectively to obtain local feature maps and global feature maps; Step 332: Concatenate the local feature map and the global feature map, and then perform expansion and compression sequentially to generate attention weights for view arbitration; Step 333: Multiply the attention weights used for view arbitration with the branch feature map to obtain the weighted feature map.

[0008] Furthermore, at least three rounds of encoding iteration are performed in step 2, and at least three rounds of decoding iteration are performed in step 4.

[0009] A model for performing the medical image segmentation method based on dynamic layer-view-scale fusion as described in any one of claims 1-5, comprising: The initial feature extraction module is used to extract initial features from the input image; The dynamic dense fusion coding module is used to perform multiple rounds of coding iteration on the initial feature map sequentially; The cross-scale attention gate module is used to weight the output after dynamic dense fusion and pooling upsampling in each decoding iteration; The hole feature extraction module is used to perform view arbitration-based adaptive scale calibration on the encoded feature map output from the last round of encoding iteration; The dynamic dense fusion decoding module is used to perform multiple rounds of decoding iteration on the feature map after adaptive receptive field calibration. The prediction head module is used to perform 1×1 convolution on the decoded feature map output from the last round of decoding iteration.

[0010] Furthermore, the dynamic dense fusion coding module specifically includes: multiple dynamic dense fusion modules and a two-dimensional max pooling layer; The dynamic dense fusion module is used to perform dynamic dense fusion processing on the input feature maps; A two-dimensional max-pooling layer is used to perform max-pooling downsampling on the output of the dynamic dense fusion module; Among them, multiple dynamic dense fusion modules and two-dimensional maximum pooling layers are arranged in a preset order and electrically connected in sequence.

[0011] Furthermore, the hole feature extraction module includes: The first dilated convolutional layer is used to perform a 1×1 dilated convolution operation with a dilation rate of 1 on the input feature map; The second dilated convolutional layer is used to perform a 1×3 dilated convolution operation with a dilation rate of 2 on the input feature map; The third dilated convolutional layer is used to perform a 3×1 dilated convolution operation with a dilation rate of 3 on the input feature map; The fourth dilated convolutional layer is used to perform a 3×3 dilated convolution operation with a dilation rate of 4 on the input feature map; The global pooling layer is used to perform global average pooling or global max pooling operations on the input feature map.

[0012] Multiple global-local fusion modules are used to perform view arbitration operations on the outputs of the first dilated convolutional layer, the second dilated convolutional layer, the third dilated convolutional layer, the fourth dilated convolutional layer, and the global pooling layer, respectively. The splicing layer is used to perform splicing operations on the outputs of multiple global and local fusion modules; A 1×1 convolutional layer is used to perform a 1×1 convolution operation on the output of the splicing layer; The normalization layer is used to perform batch normalization on the output of the 1×1 convolutional layer. The ReLU activation function layer is used to perform an element-wise nonlinear transformation on the output of the normalization layer; The outputs of the first dilated convolutional layer, the second dilated convolutional layer, the third dilated convolutional layer, the fourth dilated convolutional layer, and the global pooling layer flow into the corresponding global-local fusion modules. The outputs of all global-local fusion modules flow into the splicing layer, and then pass through the 1×1 convolutional layer, the normalization layer, and the ReLU activation function layer in sequence before being output.

[0013] Furthermore, the global-local fusion module includes: The global feature extraction submodule is used to perform global pooling and upsampling on the input feature map; The fusion submodule is used to concatenate the output of the local feature extraction submodule with the output of the global feature extraction submodule, and then sequentially perform channel doubling expansion, channel halving compression and Sigmoid activation to generate attention weights for view arbitration; The modulation submodule is used to combine the attention weights used for view arbitration with the input feature map.

[0014] Furthermore, the dynamic dense fusion decoding module includes: multiple dynamic dense fusion modules and a scale-based projector; The dynamic dense fusion module is used to perform dynamic dense fusion processing on the input feature maps; A scale-up projector is used to upsample the output of the dynamic dense fusion module; Among them, multiple dynamic dense fusion modules and scale projectors are arranged in a preset order and electrically connected in sequence.

[0015] Compared with existing technologies, the advantages of this invention are: This invention calculates pixel-wise attention coefficients by using cross-scale attention gates to compute the encoded features at each scale level and the outputs of the corresponding decoding layers after dynamic dense fusion and pooling upsampling. These pixel-wise attention coefficients are then multiplied element-wise with the outputs of the corresponding decoding layers after dynamic dense fusion and pooling upsampling, achieving pixel-level dynamic routing to enhance effective detail regions and suppress redundant noise. Adaptive receptive field calibration is achieved by performing view arbitration-based adaptive scale calibration on the encoded feature map output from the last encoding iteration. This enables more accurate adaptation to anatomical structures and lesions of different scales and contrasts, significantly reducing missed detections of small lesions and missegmentation of large structures. It also exhibits stronger robustness on variable, multi-center, and heterogeneous data, solving the problem of feature representations failing to accurately align with the diversity and heterogeneity of anatomical structures. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the overall structure of the model proposed in this invention.

[0017] Figure 2 This is a schematic diagram of the dynamic dense fusion module structure proposed in this invention.

[0018] Figure 3 This is a schematic diagram of the cross-scale attention gate module structure proposed in this invention.

[0019] Figure 4 This is a schematic diagram of the hole feature extraction module proposed in this invention.

[0020] Figure 5 This is a graph showing the results of verification using the Parse 2022 dataset in the implementation of this invention.

[0021] Figure 6 The model proposed in this invention outputs segmentation results compared to other models. Detailed Implementation

[0022] The invention will now be further explained with reference to the accompanying drawings.

[0023] A medical image segmentation method based on dynamic layer-view-scale fusion includes: Step 1: Perform initial feature extraction on the input image to obtain an initial feature map; Step 2: The initial feature map is processed through multiple rounds of encoding iteration. In each round of encoding iteration, dynamic dense fusion and pooling downsampling are performed on the current input feature map to obtain encoded feature maps at various scales. Step 3: Perform view arbitration-based adaptive scale calibration on the encoded feature maps output from the last round of encoding iterations at each scale to obtain the feature maps after adaptive receptive field calibration; Step 4: Perform multiple rounds of decoding iteration on the feature map after adaptive receptive field calibration. In each round of decoding iteration, perform dynamic dense fusion processing and pooling upsampling processing on the current input feature map. Then, weight the processed output through cross-scale attention gates to obtain decoded feature maps at each scale. Step 5: Input the decoded feature map output from the last decoding iteration of the decoded feature maps at each scale into the prediction head for convolution to obtain the segmentation result.

[0024] In step 3, the encoded feature map output from the last round of encoding iteration is first input and subjected to 1×1, 1×3, 3×1, and 3×3 dilated convolutions and global pooling operations to obtain five branch feature maps. Each of the five branch feature maps is subjected to 3×3 grouped convolutions and global pooling, and then concatenated, expanded, and compressed in sequence to generate attention weights for view arbitration. The attention weights for view arbitration are then multiplied by the branch feature maps to obtain five weighted feature maps. The five weighted feature maps are then concatenated to obtain a concatenated feature map. Finally, the concatenated feature map is processed by 1×1 convolution, batch normalization, and ReLU function to obtain the adaptive receptive field calibration feature map.

[0025] like Figure 1 As shown, the present invention also provides a model for performing the medical image segmentation method based on dynamic layer-view-scale fusion as described in any one of claims 1-5, comprising: The initial feature extraction module is used to extract initial features from the input image; The dynamic dense fusion coding module is used to perform multiple rounds of coding iteration on the initial feature map sequentially; The cross-scale attention gate module is used to weight the output after dynamic dense fusion and pooling upsampling in each decoding iteration; The hole feature extraction module is used to perform view arbitration-based adaptive scale calibration on the encoded feature map output from the last round of encoding iteration; The dynamic dense fusion decoding module is used to perform multiple rounds of decoding iteration on the feature map after adaptive receptive field calibration. The prediction head module is used to perform 1×1 convolution on the decoded feature map output from the last round of decoding iteration.

[0026] The state-dense fusion coding module includes multiple dynamic dense fusion modules arranged in a preset order and electrically connected in sequence, and a two-dimensional max pooling layer. The dynamic dense fusion modules are used to perform dynamic dense fusion processing on the input feature maps, and the two-dimensional max pooling layer is used to perform max pooling downsampling processing on the output of the dynamic dense fusion modules.

[0027] like Figure 2 The dynamic dense fusion module performs dynamic dense fusion processing on the input feature map. Specifically, the output of the previous convolutional layer is concatenated with the input of all subsequent layers. Finally, the output of each dynamic dense fusion module is the result of concatenating its input with the output of all its constituent layers. This method ensures that a skip connection is established between the input and output of each layer within the block.

[0028] Its pseudocode is as follows:

[0029] like Figure 3 The specific steps of the cross-scale attention gate module for weighting the output after dynamic dense fusion and pooling upsampling are as follows: First, the encoded features at each scale are concatenated with the output of the corresponding decoding layer after dynamic dense fusion and pooling upsampling. Then, after 1×1 convolution, they are fused by adding pixels one by one. Subsequently, they are processed by the ReLU activation function and the sigmoid function to obtain the pixel-by-pixel attention coefficients. Finally, the pixel-by-pixel attention coefficients are multiplied element-by-element by the output of the corresponding decoding layer after dynamic dense fusion and pooling upsampling to complete the weighting, thereby realizing pixel-level dynamic routing to enhance effective detail regions and suppress redundant noise.

[0030] like Figure 4 As shown, the hole feature extraction module includes a first, second, third, and fourth dilated convolutional layer and a global pooling layer arranged in parallel. Their outputs flow into the corresponding global-local fusion module. The outputs of all global-local fusion modules flow into the stitching layer, and then pass through a 1×1 convolutional layer, a normalization layer, and a ReLU activation function layer in sequence. The outputs are obtained by passing through the first, second, third, and fourth dilated convolutional layers and the global pooling layer to obtain five branch feature maps. The five branch feature maps are reweighted by the global-local fusion module to achieve adaptive receptive field calibration, which can more accurately adapt to anatomical structures and lesions of different scales and contrasts, significantly reduce the missed detection of small lesions and the missegmentation of large structures, and show stronger robustness on variable multicenter and heterogeneous data.

[0031] The global-local fusion module includes a global feature extraction submodule for global pooling and upsampling of the input feature map, a fusion submodule for concatenating the outputs of the local feature extraction submodule and the global feature extraction submodule, and then sequentially performing channel doubling expansion, channel halving compression and Sigmoid activation to generate attention weights for view arbitration, and a modulation submodule for combining the attention weights for view arbitration with the input feature map.

[0032] The dynamic dense fusion decoding module includes a dynamic dense fusion module for performing dynamic dense fusion processing on the input feature map, and a scale projector for upsampling the output of the dynamic dense fusion module. The multiple dynamic dense fusion modules and scale projectors are arranged in a preset order and electrically connected in sequence.

[0033] like Figure 5 As shown, in order to comprehensively evaluate the performance and generalization ability of the model proposed in this invention, the Parse2022 dataset was used for validation. The experiment shows that the average Dice coefficient is improved by 3-5% compared with the baseline U-Net in the three types of fluid segmentation: IRF, SRF and PED.

[0034] like Figure 6 As shown in the figure, the segmentation results of the model of this invention and other models are compared when a chest CT cross-sectional image is input. The segmentation results of the model of this invention are highly consistent with the gold standard Image+label at the bottom in terms of detail and morphology, with more complete branches and less noise. Other comparison methods (such as FABR, FANN, etc.) have cases of missing branches, breaks, or missegmentation.

[0035] As is known from common technical knowledge, this invention can be implemented through other embodiments that do not depart from its spirit or essential characteristics. Therefore, the disclosed embodiments described above are merely illustrative and not exhaustive. All modifications within the scope of this invention or its equivalents are included in this invention.

Claims

1. A medical image segmentation method based on dynamic layer-view-scale fusion, characterized in that, include: Step 1: Perform initial feature extraction on the input image to obtain an initial feature map; Step 2: The initial feature map is processed through multiple rounds of encoding iteration. In each round of encoding iteration, dynamic dense fusion and pooling downsampling are performed on the current input feature map to obtain encoded feature maps at various scales. Step 3: Perform view arbitration-based adaptive scale calibration on the encoded feature maps output from the last round of encoding iterations at each scale to obtain the feature maps after adaptive receptive field calibration; Step 4: Perform multiple rounds of decoding iteration on the feature map after adaptive receptive field calibration. In each round of decoding iteration, perform dynamic dense fusion processing and pooling upsampling processing on the current input feature map. Then, weight the processed output through cross-scale attention gates to obtain decoded feature maps at each scale. Step 5: Input the decoded feature map output from the last decoding iteration of the decoded feature maps at each scale into the prediction head for convolution to obtain the segmentation result.

2. The medical image segmentation method based on dynamic layer-view-scale fusion according to claim 1, characterized in that: Step 3 specifically includes the following steps: Step 31: Perform dilated convolution and global pooling operations of 1×1, 1×3, 3×1, and 3×3 on the encoded feature map output from the last round of encoding iteration to obtain five branch feature maps; Step 32: Perform view arbitration on the five branch feature maps respectively to obtain five weighted feature maps; Step 33: Concatenate the five weighted feature maps to obtain the concatenated feature map; Step 34: The concatenated feature maps are processed sequentially by 1×1 convolution, batch normalization, and ReLU function to obtain the feature maps after adaptive receptive field calibration.

3. The medical image segmentation method based on dynamic layer-view-scale fusion according to claim 2, characterized in that: The specific steps for view arbitration of each branch feature map in step 33 include: Step 331: Perform 3×3 grouped convolution and global pooling on the branch feature maps respectively to obtain local feature maps and global feature maps; Step 332: Concatenate the local feature map and the global feature map, and then perform expansion and compression sequentially to generate attention weights for view arbitration; Step 333: Multiply the attention weights used for view arbitration with the branch feature map to obtain the weighted feature map.

4. The medical image segmentation method based on dynamic layer-view-scale fusion according to claim 1, characterized in that: Step 2 involves at least 3 rounds of encoding iteration, and step 4 involves at least 3 rounds of decoding iteration.

5. A model for performing the medical image segmentation method based on dynamic layer-view-scale fusion as described in any one of claims 1-5, characterized in that, include: The initial feature extraction module is used to extract initial features from the input image; The dynamic dense fusion coding module is used to perform multiple rounds of coding iteration on the initial feature map sequentially; The cross-scale attention gate module is used to weight the output after dynamic dense fusion and pooling upsampling in each decoding iteration; The hole feature extraction module is used to perform view arbitration-based adaptive scale calibration on the encoded feature map output from the last round of encoding iteration; The dynamic dense fusion decoding module is used to perform multiple rounds of decoding iteration on the feature map after adaptive receptive field calibration. The prediction head module is used to perform 1×1 convolution on the decoded feature map output from the last round of decoding iteration.

6. The model according to claim 5, characterized in that, The dynamic dense fusion coding module specifically includes: multiple dynamic dense fusion modules and a two-dimensional max pooling layer; The dynamic dense fusion module is used to perform dynamic dense fusion processing on the input feature maps; A two-dimensional max-pooling layer is used to perform max-pooling downsampling on the output of the dynamic dense fusion module; Among them, multiple dynamic dense fusion modules and two-dimensional maximum pooling layers are arranged in a preset order and electrically connected in sequence.

7. The model according to claim 5, characterized in that, The hole feature extraction module includes: The first dilated convolutional layer is used to perform a 1×1 dilated convolution operation with a dilation rate of 1 on the input feature map; The second dilated convolutional layer is used to perform a 1×3 dilated convolution operation with a dilation rate of 2 on the input feature map; The third dilated convolutional layer is used to perform a 3×1 dilated convolution operation with a dilation rate of 3 on the input feature map; The fourth dilated convolutional layer is used to perform a 3×3 dilated convolution operation with a dilation rate of 4 on the input feature map; The global pooling layer is used to perform global average pooling or global max pooling operations on the input feature map. Multiple global-local fusion modules are used to perform view arbitration operations on the outputs of the first dilated convolutional layer, the second dilated convolutional layer, the third dilated convolutional layer, the fourth dilated convolutional layer, and the global pooling layer, respectively. The splicing layer is used to perform splicing operations on the outputs of multiple global and local fusion modules; A 1×1 convolutional layer is used to perform a 1×1 convolution operation on the output of the splicing layer; The normalization layer is used to perform batch normalization on the output of the 1×1 convolutional layer. The ReLU activation function layer is used to perform an element-wise nonlinear transformation on the output of the normalization layer; The outputs of the first dilated convolutional layer, the second dilated convolutional layer, the third dilated convolutional layer, the fourth dilated convolutional layer, and the global pooling layer flow into the corresponding global-local fusion modules. The outputs of all global-local fusion modules flow into the splicing layer, and then pass through the 1×1 convolutional layer, the normalization layer, and the ReLU activation function layer in sequence before being output.

8. The model according to claim 7, characterized in that, The global-local fusion module includes: The local feature extraction submodule is used to extract the local spatial context of the input feature map using 3×3 grouped convolution; The global feature extraction submodule is used to perform global pooling and upsampling on the input feature map; The fusion submodule is used to concatenate the output of the local feature extraction submodule with the output of the global feature extraction submodule, and then sequentially perform channel doubling expansion, channel halving compression and Sigmoid activation to generate attention weights for view arbitration; The modulation submodule is used to combine the attention weights used for view arbitration with the input feature map.

9. The model according to claim 5, characterized in that, The dynamic dense fusion decoding module includes: multiple dynamic dense fusion modules and a scale-based projector; The dynamic dense fusion module is used to perform dynamic dense fusion processing on the input feature maps; A scale-up projector is used to upsample the output of the dynamic dense fusion module; Among them, multiple dynamic dense fusion modules and scale projectors are arranged in a preset order and electrically connected in sequence.

Images