A Medical Image Segmentation Method Based on Multi-Scale Bridging Pre-decoding Skip Correction

By forming a unified bridging representation in the multi-scale bridging stage between the encoder and decoder, and by using channel gating coefficients to correct the original skip features, the problem of lack of cross-scale semantic correction before decoding is solved, thus improving the accuracy and efficiency of medical image segmentation.

CN123118675APending Publication Date: 2026-11-17JILIN UNIVERSITY
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202611603665.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-10-16
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing medical image segmentation methods lack cross-scale semantic correction before decoding, which leads to increased decoder burden due to background-dominant response and semantically inconsistent information, thus affecting segmentation accuracy.

Method used

A multi-scale bridging stage is set up between the encoder and decoder to form a unified bridging representation. Correction candidate features are generated through lightweight projection and progressive fusion, and the original skip features are adjusted using channel gating coefficients to reduce the processing burden on the decoder.

Benefits of technology

It improves the accuracy of multi-scale feature fusion, reduces the processing burden of the decoder, enhances the semantic consistency and spatial detail preservation of the segmentation results, and reduces model complexity and computational cost.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123118675A_ABST
    Figure CN123118675A_ABST
Patent Text Reader

Abstract

This invention discloses a medical image segmentation method based on multi-scale bridging pre-decoding skip correction, belonging to the field of medical image processing. The method includes acquiring the medical image to be segmented and extracting multi-scale encoded features; performing lightweight projection on the encoded features at each scale to obtain compact projection features; progressively upsampling, stitching, and fusing from deep to shallow layers to form a unified bridging representation; adjusting the unified bridging representation to each scale to generate candidate features and channel gates, performing residual correction, and outputting skip features; sequentially performing efficient upsampling, bidirectional gated fusion, and attention-based multi-scale refinement to complete progressive decoding; and outputting pixel-level segmentation results of target tissues or lesions in the medical image. This invention aims to solve the problem that the original encoded skip features lack cross-scale semantic correction before entering the decoder, leading to increased decoding burden due to background-dominant responses and semantically inconsistent information.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention pertains to medical image processing, specifically to a medical image segmentation method based on multi-scale bridging pre-decoding skip correction. Background Technology

[0002] Medical image segmentation aims to determine the pixel-level boundaries of organs, tissues, or lesion regions from medical images, and is a fundamental technology in medical image analysis and computer-aided diagnosis. Existing medical image segmentation methods typically employ an encoder-decoder structure: the encoder extracts image features at different scales through progressive downsampling, and the decoder gradually restores the spatial resolution of the features through upsampling, using skip connections to pass features from the encoding stage to the corresponding decoding stage to compensate for the spatial details lost during downsampling.

[0003] Features from different coding layers differ in spatial resolution, channel composition, and semantic level. Shallow coding features contain more local information such as edges and textures, but may also contain background interference; deep coding features have strong semantic expressive power, but relatively insufficient spatial detail. If the original coding features are directly fed into the decoder through skip connections, or if only features of different scales are concatenated, added, and attention-weighted, features with inconsistent semantic levels may be directly mixed during the decoding stage, affecting target boundary recovery and small-scale target recognition.

[0004] Chinese patent document CN114066904A (publication date: February 18, 2022) discloses a method, device, and storage medium for skin lesion image segmentation based on deep learning. This scheme uses a multi-scale bridging module to interact with multi-scale encoded features between the encoder and decoder, and outputs bridging features of multiple scales for the decoder to fuse. Chinese patent document CN120182612A (publication date: June 20, 2025) discloses an image semantic segmentation method, device, and medium based on multi-scale feature fusion. This scheme introduces a multi-scale feature fusion module at the bridging point between the encoder and decoder, fuses features of different scales based on the semantic information output by the encoder, outputs multiple fused feature maps of different scales, and then the decoder performs feature fusion on these fused feature maps. The technical solutions disclosed in the above documents all complete the interaction or fusion of multi-scale features between the encoder and decoder, and provide the resulting multiple scale features to the decoder respectively. Their focus is on enhancing the information interaction between features of different scales and providing multi-scale input to the decoder. However, none of the above schemes disclose how to perform scale-specific correction of the original skip features using a common cross-scale semantic reference before decoding and fusion, nor do they disclose the establishment of a unified bridging representation to perform gated residual correction of the original skip features before decoding. As a result, the decoder needs to perform semantic alignment, background interference suppression, and spatial reconstruction simultaneously during the decoding stage.

[0005] Therefore, how to improve the accuracy of multi-scale feature fusion while correcting the original skip features before decoding and reducing the multi-processing burden of the decoder is a technical problem that needs to be solved in the existing technology. Summary of the Invention

[0006] The purpose of this invention is to provide a medical image segmentation method that corrects multi-scale encoded skip features before decoding and fusion, and outputs pixel-level segmentation results for organs, tissues, or lesions. This addresses the problem that the original encoded skip features lack cross-scale semantic correction before entering the decoder, leading to increased decoding burden due to background-dominant responses and semantically inconsistent information. By setting a multi-scale bridging stage between the encoder and decoder, a unified bridging representation is formed, and correction candidates and channel gates are generated according to scale. This injects cross-scale semantic references while preserving original structural details, providing corrected skip features for a lightweight decoder.

[0007] The objective of this invention is achieved through the following technical solution: a medical image segmentation method based on multi-scale bridging pre-decoding skip correction, the method comprising the following steps: S1: Acquire the medical image to be segmented, resize and normalize the image, and input it into a layer encoder to obtain images sequentially from light to dark. Original encoded features at each scale ,in, This indicates the number of feature scale levels output by the hierarchical encoder, where each feature scale level corresponds one-to-one with the correction skip feature and the decoding feature. It is an integer greater than or equal to 2; S2: Original encoded features for each scale. Lightweight projection is performed to obtain projection features that correspond one-to-one with each scale. , Indicates scale index. , Indicates the shallowest scale. This represents the deepest scale; the lightweight projection is used to adjust the channel dimensions of features at each scale. S3: Projecting features at the deepest scale Starting with the first layer, a top-down progressive fusion is performed in order from deepest to shallowest, specifically including: for the first... Scale projection features The spatial dimensions of the already fused adjacent deeper-scale fused features are adjusted to match the projected features. The spatial dimensions are matched, and the two are spliced ​​and merged along the channel dimension to obtain the first... Scale-progressive fusion features Repeat the above spatial size adjustment, splicing, and fusion processes in order from deep to shallow until the shallowest scale progressive fusion feature is obtained and identified as a unified bridging representation. ; S4: Targeting the Each scale will unify the bridging representation. Adjust to the original encoded features at this scale Spatial dimensions are used to obtain scale-fit bridging characterization. Characterized by bridging based on the same scale Generate features corresponding to the original encoded features at that scale. Bridged aligned candidate features with consistent channel count and channel gating coefficients with values ​​between 0 and 1 ; using original encoded features Using the residual basis, the channel gating coefficients are utilized. Adjustment by the bridging alignment candidate features The determined candidate correction values ​​are used to obtain the correction jump characteristics. ; S5: Input the scale-corrected jump features into the decoder, and perform progressive upsampling, feature fusion, and multi-scale refinement in order from deep to shallow to obtain the decoded features at each scale. ; S6: Input the shallowest layer decoded features into the segmentation head and adjust them to the spatial size of the input image to obtain the pixel category probability. Based on the maximum probability or a preset threshold, obtain the pixel-level segmentation results of the organ, tissue or lesion region. Wherein, the unified bridging representation It is not directly used as input to the decoder, but rather as a common correction reference for the original jump features at each scale before they enter the decoding fusion process.

[0008] Furthermore, in step S2, the lightweight projection employs depth-separable convolutional blocks that maintain a constant spatial size; each depth-separable convolutional block sequentially includes a depth convolutional layer, a first batch of normalized layers, a first ReLU activation layer, a pointwise convolutional layer, a second batch of normalized layers, and a second ReLU activation layer, wherein the kernel size of the depth convolutional layer is 3×3, the stride is 1, the padding is 1, and the number of groups is equal to the number of input channels; the kernel size of the pointwise convolutional layer is 1×1, and the stride is 1.

[0009] Furthermore, in step S3, during the progressive fusion, the deeper fusion features are adjusted to the spatial size of the current layer's projected features using bilinear interpolation; the fusion process employs depthwise separable convolutional blocks that maintain the same spatial size. These depthwise separable convolutional blocks sequentially include a depthwise convolutional layer, a first batch of normalized layers, a first ReLU activation layer, a pointwise convolutional layer, a second batch of normalized layers, and a second ReLU activation layer. The depthwise convolutional layer has a kernel size of 3×3, a stride of 1, padding of 1, and the number of groups equals the number of input channels; the pointwise convolutional layer has a kernel size of 1×1 and a stride of 1.

[0010] Furthermore, in step S4, the bridging alignment candidate features Characterized by the scale adaptation bridging The candidate feature generation branch is obtained by sequentially including a convolutional layer with a kernel size of 3×3, a stride of 1, and padding of 1, a batch normalization layer, and a ReLU activation layer, and the output is the same as the original encoded feature. Candidate features with consistent channel count; the channel gating coefficient Characterized by the scale adaptation bridging The channel gate generation branch is obtained by sequentially including a global average pooling layer, a first 1×1 pointwise convolutional layer, a ReLU activation layer, a second 1×1 pointwise convolutional layer, and a Sigmoid activation layer, and the output is the same as the original encoded features. Channel gating coefficients with consistent channel count; the corrected skip feature Determined according to the following formula: ; in, This represents element-wise multiplication. Here, represents the channel gating factor, indicating the correction strength of each channel; when When the value approaches 0, the original encoded features are preserved. ,when Increase the number of bridging alignment candidate features Its function.

[0011] Furthermore, in step S5, the decoder includes an efficient upconvolutional block, a bidirectional gated fusion block, and an attention multi-scale aggregator; the recursive relationship of the decoder is as follows: ; ; in, This represents the deepest-scale decoded features, and the jump features are corrected based on the deepest-scale features. After the first Scale attention multiscale aggregation operation Processed; Indicates the relationship with the first Decoding features at deeper scales that are adjacent in scale; Indicates the first Scale-based decoding features; , and They represent the first Efficient upconvolution at various scales, bidirectional gated fusion, and attention-based multi-scale aggregation operations; The efficient upconvolutional block receives decoding features from deeper scales adjacent to the current scale, first performs 2x bilinear upsampling, then sequentially performs 3×3 depth convolution, batch normalization and ReLU activation, and finally adjusts the number of channels through 1×1 convolution. The bidirectional gated fusion block performs 3×3 grouped convolution and batch normalization on the upsampled decoding features and current scale-corrected skip features output by the efficient upper convolution block, respectively. The two results are added together and activated by ReLU to obtain a shared context. Then, skip branch gate coefficients and decoding branch gate coefficients are generated by two independent 1×1 pointwise convolutions and Sigmoid activation, respectively. The two input features are multiplied element-wise with the corresponding gate coefficients and then added together to obtain the fused features. The attention multi-scale aggregator extracts different receptive field information through parallel deep convolution branches with kernel sizes of 1×1, 3×3, and 5×5. Each branch sequentially performs deep convolution, batch normalization, and ReLU6 activation, and then generates channel weights through global average pooling, a first 1×1 convolution, ReLU activation, a second 1×1 convolution, and Sigmoid activation. The channel weights are then multiplied channel by channel with the corresponding branch features. After the outputs of each branch are added together, they are sequentially processed through 1×1 pointwise convolution, batch normalization, and ReLU6 activation to obtain the current stage decoding features.

[0012] further, The hierarchical encoder described in step S1 employs a visual Transformer network or convolutional neural network capable of outputting features at four scales. The original encoded features at the four scales are denoted as follows: .

[0013] Furthermore, the layered encoder adopts either PVTv2-B0 or ​​PVTv2-B2; when PVTv2-B0 is adopted, the number of output channels in the four stages is 32, 64, 160 and 256, corresponding to the number of projection channels of 32, 64, 80 and 128; when PVTv2-B2 is adopted, the number of output channels in the four stages is 64, 128, 320 and 512, corresponding to the number of projection channels of 64, 128, 160 and 256.

[0014] Furthermore, in step S6, multi-class segmentation obtains the pixel probability of each class through Softmax and takes the class with the highest probability; binary segmentation obtains the foreground probability through Sigmoid and performs binarization according to a preset threshold.

[0015] Furthermore, during the training phase, the method employs 1×1 convolutions at the segmentation heads corresponding to the decoded features at each scale to generate prediction maps. And each prediction map is adjusted to the actual labeled spatial size before being used in loss calculation; For multi-class medical image segmentation, Each non-empty combination of the prediction graph is summed element by element to obtain... Each combination of prediction results is calculated; for each combination of prediction results, the cross-entropy loss and Dice loss are calculated, and the losses corresponding to each combination are added together with equal weights of 1 to obtain the total training loss. For binary lesion segmentation, the segmentation loss, consisting of weighted binary cross-entropy loss and weighted intersection-union ratio loss, is calculated for all predicted maps, and then... The element-wise sum of each prediction graph is used to calculate the same type of segmentation loss again. All losses are then summed with equal weights of 1 to obtain the total training loss.

[0016] Furthermore, in step S6, the inference stage only retains the prediction map generated by the segmentation head corresponding to the shallowest layer of decoded features, and outputs the pixel-level segmentation results of the organ, tissue or lesion region after Softmax or Sigmoid processing.

[0017] Compared with the prior art, the present invention can bring the following beneficial effects: First, the medical image segmentation method based on multi-scale bridging pre-decoding jump correction proposed in this invention reduces semantic inconsistencies between jump features at different scales by lightweightly projecting encoded features at different scales and fusing them step by step in a top-down manner. This results in a unified bridging representation that contains both deep semantic information and shallow structural information. Second, by using a unified bridging representation to generate bridging alignment candidate features for each scale before decoding and fusion, the original skip features can be semantically corrected across scales before entering the decoder, thereby reducing the processing burden of the decoder to simultaneously handle semantic alignment, interference suppression and spatial reconstruction. Third, by correcting channel gate residuals, the amount of bridging information injected is controlled according to the gate coefficient of each channel. While preserving the spatial details and boundary information in the original coding features, background response and channel response inconsistent with the decoding semantics are suppressed. Fourth, by separating jump feature correction from decoding reconstruction, the decoder receives the corrected jump features, thereby enabling a more compact progressive decoding structure and reducing the number of parameters and computational load required in the decoding process. Attached Figure Description

[0018] Figure 1 The flowchart illustrates the overall implementation of the medical image segmentation method based on multi-scale bridging pre-decoding jump correction provided in this embodiment of the invention. Figure 2 This is a flowchart of lightweight projection and progressive top-down fusion in an embodiment of the present invention; Figure 3 This is a flowchart of channel gating residual correction in an embodiment of the present invention; Figure 4 This is a flowchart of the lightweight multi-scale attention decoding in an embodiment of the present invention. Detailed Implementation

[0019] like Figures 1 to 4 As shown, this invention provides a medical image segmentation method based on multi-scale bridging pre-decoding skip correction. For ease of description, the multi-scale bridging network constructed using the method of this invention is denoted as MSBN (Multi-Scale Bridge Network), and the multi-scale bridging module set between the encoder and decoder for constructing a unified bridging representation and implementing pre-decoding skip feature correction is denoted as MSBM (Multi-Scale Bridge Module). The MSBN takes the medical image to be segmented as input, uses the encoder to extract multi-scale original encoded features, forms a unified bridging representation for use by all scales through the MSBM, performs channel-gated residual correction on the original skip features before decoding fusion, and finally uses the decoder to restore the spatial resolution step by step and output pixel-level segmentation results. The method includes the following steps: S1: Acquire the medical image to be segmented, resize and normalize it, and input it into the hierarchical encoder to obtain the original encoded features at four scales from shallow to deep. , , and .

[0020] S2: Encode the original features at each scale respectively. Lightweight projection is performed to obtain projection features. This is to adjust the channel dimensions of features at various scales and reduce channel redundancy during the bridging construction process; ; in, This indicates a lightweight projection operation.

[0021] S3: Projecting features from the deepest layer The process begins with a top-down progressive fusion, proceeding from deepest to shallowest layers: Deeper fusion features are adjusted to the spatial dimensions of the current layer's projected features, stitched together with the current layer's projected features along the channel direction, and then fused. This process is repeated sequentially to obtain... , , and ; with the shallowest fusion features As a unified bridging representation Unified bridging representation It is not directly used as the input to the decoder, but as a common correction reference for the original jump features at each scale; ; ; ; in This indicates a cross-scale fusion operation. and These represent upsampling and splicing operations along the channel direction, respectively. Indicates the first Scale projection characteristics; Indicates the first Scale fusion features Indicates the relationship with the first Deeper scale fusion features with adjacent scales.

[0022] S4: Targeting the Each scale will adjust the unified bridging representation. To the original coding features Spatial dimensions are used to obtain scale-fit bridging characterization. Characterized by bridging based on the same scale Generate features respectively compared to the original encoded features Bridged aligned candidate features with consistent channel count and channel gating coefficients with values ​​between 0 and 1 ; ; ; ; in, and These represent the original encoded features. The space's height and width; This indicates a size adjustment operation; Indicates batch normalization; This indicates a 3×3 depthwise convolution with a kernel size of 3. Represents a linear rectified function; This represents the Sigmoid activation function; This represents the first layer of pointwise convolution; This indicates the second layer of pointwise convolution; Indicates global average pooling; With original encoded features Using the residual basis, channel gating coefficients are employed. Adjustment by bridging alignment candidate features The determined candidate correction values ​​are used to obtain the correction jump characteristics. ; Indicates the correction intensity of each channel; when When the value approaches 0, the original encoded features are preserved. ,when Increase the number of bridging alignment candidate features This allows for adaptive correction rather than direct replacement; .

[0023] S5: Input the scale-corrected jump features into the decoder, and perform progressive upsampling, feature fusion, and multi-scale refinement in order from deep to shallow to obtain the decoded features at each scale. As a preferred implementation, the decoder adopts a lightweight multi-scale attention decoder, denoted as LMAD (Lightweight Multi-Scale Attention Decoder); the LMAD includes an efficient upconvolutional block, a bidirectional gated fusion block, and an attention multi-scale aggregator.

[0024] S6: Input the shallowest layer decoded features into the segmentation head and adjust them to the spatial size of the input image; obtain the pixel category probability through Softmax or Sigmoid, and obtain the pixel-level segmentation result of the organ, tissue or lesion region based on the maximum probability or a preset threshold.

[0025] Figure 1 The overall flowchart of the medical image segmentation method of the present invention is shown. Steps S1 to S6 in the figure represent, in turn, multi-scale feature acquisition, lightweight projection, unified bridging representation construction, bridging-guided residual correction, progressive decoding consisting of efficient upsampling, bidirectional gating fusion and attention-based multi-scale refinement, and pixel-level segmentation result output. Figure 2 The flowchart of lightweight projection and progressive top-down fusion is shown, along with the original encoded features at each scale. Projection features are obtained through light projection. The deepest feature satisfies , , and Upsampling respectively ( ) to the adjacent shallow layer, and the corresponding , and The concatenation and fusion using depthwise separable convolutions ultimately yield a unified bridging representation. , . Figure 3 The flowchart of channel gating residual correction is shown, with a unified bridging characterization for any scale. Scale-adaptive bridging characterization was obtained through size adjustment. The candidate features are then fed into the candidate feature generation branch and the channel gate generation branch, respectively; the resulting bridged and aligned candidate features are then processed. Channel gating coefficient and original coding features The error is fed into the channel-gated residual correction unit, which outputs the correction jump characteristic. . Figure 4 The flowchart of the lightweight multi-scale attention decoding is shown. The deepest correction features are refined at multiple scales and then upsampled layer by layer, and selectively fused with the corresponding shallow correction features. Finally, the segmentation head generates pixel-level segmentation results.

[0026] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, this invention is not limited to the following embodiments, and specific implementation methods can be determined according to the technical solutions of this invention and actual circumstances. To avoid obscuring the essence of this invention, well-known methods, processes, and procedures are not described in detail.

[0027] This embodiment uses multi-organ segmentation of abdominal CT images as an example. Abdominal CT images and their pixel-level annotations are acquired, the input image is adjusted to 224×224 pixels, and data augmentation is performed during the training phase using random rotation and random flipping.

[0028] The processed medical image is input into a hierarchical encoder capable of outputting features at four scales. This embodiment uses ImageNet pre-trained PVTv2-B0 or ​​PVTv2-B2. For the Synapse dataset implementation (the Synapse multi-organ dataset is a high-quality 3D image dataset designed specifically for medical image segmentation research and widely used in the development and evaluation of multi-organ segmentation algorithms), the downsampling ratio r in the first stage is set to 2, and the subsequent three stages are downsampled by a factor of 2. The number of output channels for the four stages of PVTv2-B0 is 32, 64, 160, and 256, corresponding to the number of projection channels of 32, 64, 80, and 128; the number of output channels for the four stages of PVTv2-B2 is 64, 128, 320, and 512, corresponding to the number of projection channels of 64, 128, 160, and 256. Depending on the encoder and corresponding channel configuration used, this embodiment presents two network configurations: a lightweight configuration using a PVTv2-B0 encoder, denoted as MSBN-T, and a standard configuration using a PVTv2-B2 encoder, denoted as MSBN-S. Both configurations employ the same MSBM and pre-decoding skip correction process; the main difference lies in the number of output channels and projection channels at each scale of the encoder. The encoder can also be replaced with a convolutional neural network or other visual Transformer networks capable of generating multi-scale hierarchical features.

[0029] Table 1 Network structure parameter configuration in this embodiment ; Table 1 lists the spatial dimensions and channel configurations of the four scale features of MSBN-T and MSBN-S. Table 1 , , and These represent the original coding features, projection features, corrected skip features, and decoding features at the first level, respectively; within the same level... , and The number of channels remains the same. The appropriate number of projection channels is used; in this embodiment .

[0030] According to step S2, lightweight projection employs a depthwise separable convolutional block that maintains a constant spatial dimension. The depthwise separable convolutional block sequentially includes a depthwise convolutional layer with a 3×3 kernel, a stride of 1, padding of 1, and a group number equal to the number of input channels; a first batch of normalization layers; a first ReLU activation layer; a pointwise convolutional layer with a 1×1 kernel and a stride of 1; a second batch of normalization layers; and a second ReLU activation layer. The depthwise convolutional layer extracts spatial information within each input channel, and the pointwise convolutional layer performs channel blending and adjusts the output to the preset number of projection channels.

[0031] Following step S3, progressively fuse features projected from the deepest layer. Begin. For deeper fused features, bilinear interpolation is used to ensure their spatial dimensions match the projected features of the current layer. The two are then stitched together along the channel direction, and further processed using a depth-separable fusion block that maintains the same spatial dimensions. This fusion block sequentially includes a deep convolutional layer with a 3×3 kernel, a stride of 1, padding of 1, and a group number equal to the number of input channels; a first batch of normalization layers; a first ReLU activation layer; a pointwise convolutional layer with a 1×1 kernel and a stride of 1; a second batch of normalization layers; and a second ReLU activation layer, until the shallowest fused feature is obtained. and will As a unified bridging representation .

[0032] Following step S4, bilinear interpolation is used for each scale to achieve a unified bridging representation. Adjust to original coding features The spatial dimensions. Candidate feature branches are sequentially subjected to convolution with a 3×3 kernel, a stride of 1, and padding of 1, followed by batch normalization and ReLU activation. The output is the same as the original encoded features. Bridged aligned candidate features with consistent channel count The gated branches sequentially employ global average pooling, a first 1×1 pointwise convolution, ReLU activation, a second 1×1 pointwise convolution, and Sigmoid activation, outputting the same features as the original encoded features. Channel gating coefficient with consistent channel number Through relational expressions The calculation yielded corrected jump features at four scales. to .

[0033] As a preferred implementation, LMAD includes an efficient upconvolutional block (EUCB), a bidirectional gated fusion block (BDGF), and an attention-based multi-scale aggregator (AMSA). The EUCB receives the decoding features from the previous stage, performs a 2x bilinear upsampling, then sequentially executes 3×3 depthwise convolution, batch normalization, and ReLU activation, and finally adjusts the number of channels through 1×1 convolution. The recursive relationship of the decoding process is as follows: ; ; In this embodiment, the deepest-scale decoding feature Jump characteristics corrected by the deepest scale After 4th-scale attention multi-scale aggregation operation Processed to obtain; decoding recursion sequentially in Execution time, among which In order to be consistent with the current number Decoding features at deeper scales that are adjacent in scale. For the current scale Correcting the jump characteristics; , and They represent the first Efficient upconvolution at various scales, bidirectional gated fusion, and attention-based multi-scale aggregation operations; Indicates the first Decoding features of scale.

[0034] The bidirectional gated fusion block BDGF performs 3×3 grouped convolution and batch normalization on the upsampled decoding features and current scale-corrected skip features output by the efficient upper convolution block EUCB, respectively. The two results are added together and activated by ReLU to obtain a shared context. Then, two independent 1×1 pointwise convolutions and Sigmoid activations are used to generate skip branch gating coefficients and decoding branch gating coefficients, respectively. The two input features are multiplied element-wise with the corresponding gating coefficients and then added together to obtain the fused features.

[0035] The Attention Multi-Scale Aggregator (AMSA) extracts information from different receptive fields through parallel depthwise convolutional branches with kernel sizes of 1×1, 3×3, and 5×5. Each branch sequentially performs depthwise convolution, batch normalization, and ReLU6 activation. Then, it generates channel weights through global average pooling, a first 1×1 convolution, ReLU activation, a second 1×1 convolution, and sigmoid activation, multiplying each channel weight by the corresponding branch feature channel by channel. The outputs of each branch are summed, and then sequentially processed through 1×1 pointwise convolution, batch normalization, and ReLU6 activation to obtain the current stage's decoded features. Here, ReLU6 represents a linear rectified activation function with output values ​​limited to the range of 0 to 6.

[0036] The aforementioned efficient upconvolutional block EUCB, bidirectional gated fusion block BDGF, and attention multi-scale aggregator AMSA are preferred implementation structures and are not the only limitation on the scope of protection of this invention; other decoding structures that can receive and correct skip features and complete stepwise spatial resolution recovery, feature fusion, and multi-scale refinement can also be used in conjunction with the unified bridging pre-decoding skip correction process of this invention.

[0037] The shallowest decoding feature is obtained through step-by-step decoding. Subsequently, a segmentation head consisting of 1×1 convolutions is used to generate prediction channels, which are then adjusted to the input image size using bilinear interpolation. Multi-class segmentation obtains the pixel probabilities of each class through Softmax and selects the class with the highest probability; binary segmentation obtains the foreground probability through Sigmoid and performs binarization based on a preset threshold, ultimately outputting a pixel-level segmentation mask for the organ, tissue, or lesion region.

[0038] The following training method is used to illustrate a specific training implementation of the model of this invention, and is not a necessary limitation on the unified bridging representation construction, channel gating residual correction, and pre-decoding skip feature correction process. In the decoder... , , and Each segmentation head is set at a specific location, and each segmentation head uses a 1×1 convolution to obtain the prediction image. , , and Each prediction map is then adjusted to the actual labeled spatial dimensions before being used in loss calculation.

[0039] For multi-class medical image segmentation, all non-empty combinations of the four prediction images are summed element-wise to obtain a total of 15 combination prediction results. Cross-entropy loss and Dice loss are calculated for each combination prediction result, and the losses corresponding to the 15 combinations are summed with equal weights of 1 to obtain the total training loss.

[0040] For the segmentation of second-class lesions, , , and Calculate the segmentation loss consisting of the weighted binary cross-entropy loss and the weighted intersection-union ratio loss separately, and then recalculate the same type of segmentation loss for each element-wise sum of the four prediction graphs. All losses are summed with equal weights of 1. The total training loss is determined according to the following formula: ; in, Indicates the total training loss; This represents the segmentation loss corresponding to the predicted graph. ; This represents the segmentation loss corresponding to the element-wise summation of the four prediction graphs.

[0041] This embodiment uses the AdamW optimizer, with an initial learning rate and weight decay coefficient of 1. The model parameters are updated through backpropagation. The training parameters can be adjusted according to the type of medical image, the size of the data, and the computing device.

[0042] During the model usage phase, the medical image to be segmented is input into the trained model, which sequentially performs multi-scale feature extraction, unified bridging representation construction, skip feature residual correction, and progressive decoding. The inference phase only retains the data generated by the model. Prediction map generated by the corresponding segmentation head After processing with Softmax or Sigmoid, the output is pixel-level segmentation results of organs, tissues or lesion regions.

[0043] To verify the feasibility of the technical solution of this invention, experiments were conducted on the publicly available Synapse abdominal multi-organ segmentation dataset (this dataset was obtained from the Synapse open-source data sharing and analysis platform, project identifier (Synapse ID) syn3193805), and the segmentation performance was evaluated using the average Dice similarity coefficient and the 95% Hausdorff distance (HD95). A higher average Dice similarity coefficient indicates a greater overlap between the predicted and real regions; a lower HD95 indicates that the predicted boundary is closer to the real boundary.

[0044] The experimental environment and basic settings are as follows: The model is implemented using PyTorch 1.11.0 and trained on an NVIDIA RTX 4090 graphics processor with 24GB of video memory; MSBN-T and MSBN-S use ImageNet pre-trained PVTv2-B0 and PVTv2-B2 encoders, respectively; the first-stage downsampling ratio r of the Synapse embodiment is set to 2, and the kernel set of the parallel depthwise convolution branch of the attention multi-scale aggregator AMSA is fixed at {1, 3, 5}.

[0045] MSBN-T and MSBN-S use the aforementioned encoder and channel configuration, and employ the same MSBM and progressive decoding methods. The Synapse experiment used an input image size of 224×224 pixels, a batch size of 6, and trained for 300 epochs. The model with the best performance was selected for testing based on the Dice similarity coefficient on the validation set. The comparative method metrics listed in Table 2 are summarized according to publicly available statistical standards. Except for Sim-MPNet, which uses dual encoders of 224×224 pixels and 256×256 pixels as input, the computational cost of the other methods is calculated based on a 224×224 pixel input. Therefore, comparisons of Sim-MPNet's computational cost should be considered in conjunction with this input difference.

[0046] Table 2 Comparison of Synapse dataset splitting performance and complexity ; It should be noted that U-Net, TransUNet, SwinUNet, TransDeepLab, TransCASCADE, PVT-EMCAD-B2, and Sim-MPNet are all publicly available prior art. As shown in Table 2, the MSBN-S method of this invention achieves an average Dice similarity coefficient of 85.23% and an HD95 of 14.07 (HD95 represents 95% Hausdorff distance), achieving the highest average Dice similarity coefficient and the lowest HD95 among the listed methods. Compared to Sim-MPNet, MSBN-S improves the average Dice similarity coefficient by 0.89 percentage points, reduces HD95 by 0.78, decreases the number of parameters from 91.30M to 27.04M, and reduces the computational cost from 43.74G to 17.75G, demonstrating that this invention can improve segmentation accuracy and boundary quality while reducing model complexity.

[0047] Among the methods listed in Table 2, the lightweight model MSBN-T of this invention has the lowest number of parameters and computational cost, but its average Dice similarity coefficient still reaches 83.98%, which is higher than U-Net, TransUNet, SwinUNet, TransDeepLab and TransCASCADE, and its HD95 is 14.69. This shows that this invention can still balance segmentation accuracy and computational efficiency when using a lightweight configuration.

[0048] To further distinguish the roles of multi-scale bridging pre-decoding correction and optimized decoding components, component ablation experiments were conducted under the same Synapse dataset and MSBN-S configuration. The results are shown in Table 3.

[0049] Table 3 Ablation results of each component of MSBN on the Synapse dataset. ; In Table 3, MSBM represents the aforementioned multi-scale bridging module, including lightweight projection, progressive top-down fusion, and channel-gated residual correction; AMSA represents the attention multi-scale aggregator, and BDGF represents the bidirectional gated fusion block. "√" indicates that the corresponding module is enabled, and "—" indicates that the corresponding module is disabled. The average Dice of the base model is 81.68%; after adding MSBM alone, it increases to 83.63%, an improvement of 1.95 percentage points, indicating that pre-decoding skip correction itself can improve segmentation performance without introducing multi-scale refinement and bidirectional gated fusion at the decoding end. With both AMSA and BDGF present, adding MSBM increases the average Dice from 84.18% to 85.23%, further demonstrating that multi-scale bridging correction and decoding-end reconstruction have complementary effects. Removing AMSA or BDGF from the complete model reduces the average Dice by 1.21 and 0.70 percentage points, respectively, indicating that the two decoding components can further improve reconstruction performance based on correcting skip features.

[0050] To verify whether the pre-decoding skip correction of the present invention depends on a specific decoder, the decoder was replaced while keeping the encoder, MSBM and training method unchanged. The results are shown in Table 4.

[0051] Table 4 Comparison of different decoders when MSBM is enabled ; Table 4 shows that LMAD is the preferred decoder used in this embodiment, while CASCADE and EMCAD are comparative structures used for decoder replacement experiments. As shown in Table 4, when all receive the correction skip features generated by MSBM, the average Dice of CASCADE, EMCAD, and LMAD are 85.14%, 85.17%, and 85.23%, respectively, with a maximum difference of 0.09 percentage points. This indicates that the pre-decoding skip correction driven by the unified bridging representation can be used in conjunction with different decoding structures and does not depend on a single decoder. Among them, LMAD achieves the highest average Dice while having the lowest number of decoder parameters and computational cost, indicating that it can be considered a preferred decoding structure that balances accuracy and computational efficiency.

[0052] In summary, the overall comparison results in Table 2 show that the present invention can balance medical image segmentation accuracy, boundary quality, and model complexity; the component ablation results in Table 3 show that the pre-decoding jump correction implemented by the multi-scale bridging module can independently improve segmentation performance and complement bidirectional gated fusion and attention-based multi-scale refinement; the decoder replacement results in Table 4 further demonstrate that the jump correction driven by the unified bridging representation does not depend on a single decoder structure. These results collectively verify the feasibility of the technical solution of the present invention and its main technical effects.

Claims

1. A medical image segmentation method based on multi-scale bridging pre-decoding skip correction, characterized in that, The method includes the following steps: S1: Acquire the medical image to be segmented, resize and normalize the image, and input it into a layer encoder to obtain images sequentially from light to dark. Original encoded features at each scale ,in, This indicates the number of feature scale levels output by the hierarchical encoder, where each feature scale level corresponds one-to-one with the correction skip feature and the decoding feature. It is an integer greater than or equal to 2; S2: Encode the original features for each scale separately. Lightweight projection is performed to obtain projection features that correspond one-to-one with each scale. , Indicates scale index. , Indicates the shallowest scale. This represents the deepest scale; the lightweight projection is used to adjust the channel dimensions of features at each scale. S3: Projecting features at the deepest scale Starting with the first layer, a top-down progressive fusion is performed in order from deepest to shallowest, specifically including: for the first... Scale projection features The spatial dimensions of the already fused adjacent deeper-scale fused features are adjusted to match the projected features. The spatial dimensions are matched, and the two are spliced ​​and merged along the channel dimension to obtain the first... Scale-progressive fusion features Repeat the above spatial size adjustment, splicing, and fusion processes in order from deep to shallow until the shallowest scale progressive fusion feature is obtained and identified as a unified bridging representation. ; S4: Targeting the Each scale will unify the bridging representation. Adjust to the original encoded features at this scale Spatial dimensions are used to obtain scale-fit bridging characterization. Characterized by bridging based on the same scale Generate features corresponding to the original encoded features at that scale. Bridged aligned candidate features with consistent channel count and channel gating coefficients with values ​​between 0 and 1 ; using original encoded features Using the residual basis, the channel gating coefficients are utilized. Adjustment by the bridging alignment candidate features The determined candidate correction values ​​are used to obtain the correction jump characteristics. ; S5: Input the scale-corrected jump features into the decoder, and perform progressive upsampling, feature fusion, and multi-scale refinement in order from deep to shallow to obtain the decoded features at each scale. ; S6: Input the shallowest layer decoded features into the segmentation head and adjust them to the spatial size of the input image to obtain the pixel category probability. Based on the maximum probability or a preset threshold, obtain the pixel-level segmentation results of the organ, tissue or lesion region. Wherein, the unified bridging representation It is not directly used as input to the decoder, but rather as a common correction reference for the original jump features at each scale before they enter the decoding fusion process.

2. The medical image segmentation method based on multi-scale bridging pre-decoding jump correction according to claim 1, characterized in that, In step S2, the lightweight projection uses depth-separable convolutional blocks that maintain a constant spatial size; each depth-separable convolutional block sequentially includes a depth convolutional layer, a first batch normalization layer, a first ReLU activation layer, a pointwise convolutional layer, a second batch normalization layer, and a second ReLU activation layer, wherein the kernel size of the depth convolutional layer is 3×3, the stride is 1, the padding is 1, and the number of groups is equal to the number of input channels; the kernel size of the pointwise convolutional layer is 1×1, and the stride is 1.

3. The medical image segmentation method based on multi-scale bridging pre-decoding jump correction according to claim 1, characterized in that, In step S3, during the progressive fusion, the deeper fusion features are adjusted to the spatial size of the current layer's projected features using bilinear interpolation. The fusion process employs depthwise separable convolutional blocks that maintain the same spatial size. These depthwise separable convolutional blocks sequentially include a depthwise convolutional layer, a first batch of normalized layers, a first ReLU activation layer, a pointwise convolutional layer, a second batch of normalized layers, and a second ReLU activation layer. The depthwise convolutional layer has a kernel size of 3×3, a stride of 1, padding of 1, and the number of groups equals the number of input channels. The pointwise convolutional layer has a kernel size of 1×1 and a stride of 1.

4. The medical image segmentation method based on multi-scale bridging pre-decoding jump correction according to claim 1, characterized in that, In step S4, the bridging alignment candidate features Characterized by the scale adaptation bridging The candidate feature generation branch is obtained by sequentially including a convolutional layer with a kernel size of 3×3, a stride of 1, and padding of 1, a batch normalization layer, and a ReLU activation layer, and the output is the same as the original encoded feature. Candidate features with the same number of channels; The channel gating coefficient Characterized by the scale adaptation bridging The channel gate generation branch is obtained by sequentially including a global average pooling layer, a first 1×1 pointwise convolutional layer, a ReLU activation layer, a second 1×1 pointwise convolutional layer, and a Sigmoid activation layer, and the output is the same as the original encoded features. Channel gating coefficients with consistent channel count; the corrected skip feature Determined according to the following formula: ; in, This represents element-wise multiplication. Here, represents the channel gating factor, indicating the correction strength of each channel; when When the value approaches 0, the original encoded features are preserved. ,when Increase the number of bridging alignment candidate features Its function.

5. The medical image segmentation method based on multi-scale bridging pre-decoding jump correction according to claim 1, characterized in that, In step S5, the decoder includes an efficient upconvolutional block, a bidirectional gated fusion block, and an attention multi-scale aggregator; the recursive relationship of the decoder is as follows: ; ; in, This represents the deepest-scale decoding feature, and the jump feature is corrected based on the deepest-scale feature. After the first Scale attention multiscale aggregation operation Processed; Indicates the relationship with the first Decoding features at deeper scales that are adjacent in scale; Indicates the first Scale-based decoding features; , and They represent the first Efficient upconvolution at various scales, bidirectional gated fusion, and attention-based multi-scale aggregation operations; The efficient upconvolutional block receives decoding features from deeper scales adjacent to the current scale, first performs 2x bilinear upsampling, then sequentially performs 3×3 depth convolution, batch normalization and ReLU activation, and finally adjusts the number of channels through 1×1 convolution. The bidirectional gated fusion block performs 3×3 grouped convolution and batch normalization on the upsampled decoding features and current scale-corrected skip features output by the efficient upper convolution block, respectively. The two results are added together and activated by ReLU to obtain a shared context. Then, skip branch gate coefficients and decoding branch gate coefficients are generated by two independent 1×1 pointwise convolutions and Sigmoid activation, respectively. The two input features are multiplied element-wise with the corresponding gate coefficients and then added together to obtain the fused features. The attention multi-scale aggregator extracts different receptive field information through parallel deep convolution branches with kernel sizes of 1×1, 3×3, and 5×5. Each branch sequentially performs deep convolution, batch normalization, and ReLU6 activation, and then generates channel weights through global average pooling, a first 1×1 convolution, ReLU activation, a second 1×1 convolution, and Sigmoid activation. The channel weights are then multiplied channel by channel with the corresponding branch features. After the outputs of each branch are added together, they are sequentially processed through 1×1 pointwise convolution, batch normalization, and ReLU6 activation to obtain the current stage decoding features.

6. The medical image segmentation method based on multi-scale bridging pre-decoding jump correction according to claim 1, characterized in that, The hierarchical encoder described in step S1 employs a visual Transformer network or convolutional neural network capable of outputting features at four scales. The original encoded features at the four scales are denoted as follows: .

7. The medical image segmentation method based on multi-scale bridging pre-decoding jump correction according to claim 1, characterized in that, The layered encoder uses either PVTv2-B0 or ​​PVTv2-B2. When using PVTv2-B0, the number of output channels for the four stages is 32, 64, 160, and 256, corresponding to the number of projection channels of 32, 64, 80, and 128. When using PVTv2-B2, the number of output channels for the four stages is 64, 128, 320, and 512, corresponding to the number of projection channels of 64, 128, 160, and 256.

8. The medical image segmentation method based on multi-scale bridging pre-decoding jump correction according to claim 1, characterized in that, In step S6, multi-class segmentation obtains the pixel probability of each class through Softmax and takes the class with the highest probability; binary segmentation obtains the foreground probability through Sigmoid and performs binarization according to a preset threshold.

9. The medical image segmentation method based on multi-scale bridging pre-decoding jump correction according to claim 1, characterized in that, During the training phase, the method employs 1×1 convolutions at the segmentation heads corresponding to the decoded features at each scale to generate prediction maps. And each prediction map is adjusted to the actual labeled spatial size before being used in loss calculation; For multi-class medical image segmentation, Each non-empty combination of the prediction graph is summed element by element to obtain... Each combination of prediction results is calculated; for each combination of prediction results, the cross-entropy loss and Dice loss are calculated, and the losses corresponding to each combination are added together with equal weights of 1 to obtain the total training loss. For binary lesion segmentation, the segmentation loss, consisting of weighted binary cross-entropy loss and weighted intersection-union ratio loss, is calculated for all predicted maps, and then... The element-wise sum of each prediction graph is used to calculate the same type of segmentation loss again. All losses are then summed with equal weights of 1 to obtain the total training loss.

10. The medical image segmentation method based on multi-scale bridging pre-decoding jump correction according to claim 1, characterized in that, In step S6, the inference stage only retains the prediction map generated by the segmentation head corresponding to the shallowest layer of decoded features. After processing by Softmax or Sigmoid, the pixel-level segmentation results of organs, tissues or lesion regions are output.

Citation Information

Patent Citations

  • Deep learning-based skin lesion image segmentation method and device, and storage medium

    CN114066904A

  • Image semantic segmentation method and device based on multi-scale feature fusion and medium

    CN120182612A