A medical image segmentation method, apparatus, device and medium

Abstract

The application discloses a medical image segmentation method, device, equipment and medium, the method comprises the following steps: acquiring a medical image to be segmented; inputting the medical image into a double-flow encoder for feature extraction, the double-flow encoder comprising a first encoder branch based on a pre-trained visual base model and a second encoder branch based on a visual state space model, the first encoder branch encoding the medical image to obtain first features, and the second encoder branch encoding the medical image to obtain second features; performing domain self-adaptive optimization processing on the first features to obtain optimized first features, and a dynamic feature fusion optimizer is used to bridge the domain difference between natural images and medical images; performing cross-architecture feature fusion on the optimized first features and the second features to obtain fused features; and inputting the fused features into a decoder for decoding processing to obtain a segmentation result of the medical image. The application significantly enhances the clarity of the segmentation boundary and the continuity of the anatomical structure.

Description

Technical Field

[0001] This application relates to the field of medical image processing technology, specifically to a medical image segmentation method, apparatus, device, and medium. Background Technology

[0002] Cardiac magnetic resonance imaging (CMR) is a crucial imaging technique for the clinical diagnosis of cardiovascular diseases. Its segmentation results directly determine the accuracy of quantitative analysis of key structures such as the left ventricle, right ventricle, and myocardium, making it a core prerequisite for robot-assisted cardiac surgery planning and navigation. Traditional segmentation methods based on convolutional neural networks (CNNs) are limited by local receptive fields, making it difficult to model the global long-range dependencies of cardiac structures. While Transformer-based methods possess global modeling capabilities, their computational complexity increases quadratically with image resolution, resulting in high memory overhead and slow inference speeds in high-resolution medical image scenarios.

[0003] In recent years, general-purpose vision-based models, such as the Segment Anything Model (SAM), have demonstrated excellent generalization ability in natural image segmentation tasks, providing new technical approaches for medical image segmentation. However, directly applying SAM to cardiac MRI segmentation still faces significant bottlenecks: on the one hand, the feature distribution obtained by SAM pre-training on natural images differs significantly from that in medical images, leading to a substantial decrease in segmentation accuracy on fine anatomical structures such as the thin wall of the right ventricle; on the other hand, SAM's encoder lacks the ability to specifically model the global semantic context unique to medical images, and the segmentation paradigm that relies solely on prompts is insufficient to meet the needs of automated batch processing in clinical settings.

[0004] To alleviate these problems, existing research has attempted to introduce state-space models such as Mamba to construct U-shaped segmentation networks like Mamba-UNet, leveraging their linear complexity advantage to capture long sequence dependencies. However, existing solutions often employ simple two-branch parallel structures or rely solely on conventional methods such as weighted summation and ordinary cross-attention during the fusion stage. These approaches fail to effectively calibrate and collaboratively enhance the heterogeneous characteristics of SAM features, which are prone to losing pixel-level spatial semantics, and Mamba features, which weaken local salient patterns. Consequently, the fused features still suffer from semantic misalignment and loss of detail, particularly exhibiting insufficient robustness in segmentation at complex boundaries, such as those in the heart.

[0005] Therefore, how to effectively bridge the differences between the natural and medical image domains while preserving the general representation capabilities of SAM pre-training, and how to achieve deep synergy between SAM features and state space model features through targeted cross-architecture attention mechanisms, has become a key technical issue for improving cardiac MRI segmentation accuracy and meeting the clinical application needs of medical robots. Summary of the Invention

[0006] To address the aforementioned technical problems, embodiments of this application provide a medical image segmentation method, apparatus, device, and medium.

[0007] In a first aspect, embodiments of this application provide a medical image segmentation method, the method comprising: Acquire the medical image to be segmented; The medical image is input into a dual-stream encoder for feature extraction. The dual-stream encoder includes a first encoder branch based on a pre-trained visual basic model and a second encoder branch based on a visual state space model. The first encoder branch encodes the medical image to obtain a first feature, and the second encoder branch encodes the medical image to obtain a second feature. The first feature is optimized by performing domain adaptive optimization on the first feature through a dynamic feature fusion optimizer. The dynamic feature fusion optimizer is used to bridge the inter-domain differences between natural images and medical images. The optimized first feature and the second feature are fused across architectures using a heterogeneous omnidirectional attention convergence module to obtain a fused feature. The fused features are input into the decoder for decoding to obtain the segmentation result of the medical image.

[0008] In some embodiments of this application, the first encoder branch based on the pre-trained visual base model is an encoder branch based on the second-generation segmentation model, and the second encoder branch based on the visual state space model is an encoder branch based on the visual state space model. The encoder branch based on the second-generation segmentation model uses a frozen layered image encoder to encode the medical image, and adaptively adjusts the multi-scale features of the layered image encoder through a multilayer perceptron adapter. The encoder branch based on the visual state space model uses a visual state space module to encode the medical image in order to capture global semantic context features.

[0009] In some embodiments of this application, the step of performing domain adaptive optimization processing on the first feature using a dynamic feature fusion optimizer to obtain the optimized first feature includes: Perform dimensional rearrangement on the first feature; The first feature after dimensional rearrangement is subjected to adaptive max pooling and adaptive average pooling respectively. The adaptive max pooling is used to preserve global salient semantic information, and the adaptive average pooling is used to suppress high-frequency noise interference. The features obtained by adaptive max pooling and adaptive average pooling are concatenated along the channel dimension to obtain the concatenated features. The optimized first feature is obtained based on the first feature and the splicing feature.

[0010] In some embodiments of this application, obtaining the optimized first feature based on the first feature and the splicing feature includes: The first feature is processed sequentially through the first convolutional layer and the first activation function, and then through the transposed convolutional layer and the second activation function to obtain the spatial feature; The splicing features and the spatial features are fused through residual connections to obtain residual fused features; The residual fusion features are subjected to dimensional rearrangement and layer normalization to obtain the optimized first feature.

[0011] In some embodiments of this application, the heterogeneous omnidirectional attention convergence module includes a bi-branch selective emphasis attention submodule and an omniscient context attention submodule; The dual-branch selective emphasis attention submodule is used to dynamically enhance the second feature using both channel-space dual-path enhancement to obtain the enhanced second feature. The omniscient context attention submodule is used to perform global semantic enhancement on the optimized first feature to obtain the enhanced first feature; The heterogeneous omnidirectional attention convergence module is also used to hierarchically integrate the enhanced second feature, the enhanced first feature, and the historical fusion semantics through a cross-attention fusion mechanism to obtain the fused feature.

[0012] In some embodiments of this application, the dynamic enhancement of the second feature via a dual channel-space path includes: The second feature is subjected to spatial average pooling and spatial max pooling respectively. The spatial average pooling is used to capture long-distance dependencies across regions, and the spatial max pooling is used to preserve local salient patterns. The features obtained by spatial average pooling and spatial max pooling are respectively input into an adaptive self-attention mechanism with shared weights for processing to obtain dual-path attention features.

[0013] In some embodiments of this application, the processing steps of the adaptive self-attention mechanism with shared weights include: The pooled features are convolutionally processed and flattened using three independent 1×1 convolutional layers to generate a query matrix, a key matrix, and a value matrix. Calculate the similarity matrix between the query matrix and the key matrix, and normalize the similarity matrix using a normalization function to obtain the spatial relationship weight matrix; The spatial relationship weight matrix and the value matrix are multiplied by matrix multiplication to dynamically aggregate context information and obtain the dual-path attention features; The dual-path attention features are fused by weighted summation and normalized to generate a spatial attention map. The second feature is then enhanced by element-wise multiplication to obtain the enhanced second feature.

[0014] In some embodiments of this application, the global semantic enhancement of the optimized first feature includes: The optimized first feature is subjected to spatial max pooling and spatial average pooling along the channel dimension to extract salient features and capture global context information, respectively. The features obtained by spatial max pooling and spatial average pooling are concatenated along the channel dimension to obtain channel compressed features.

[0015] In some embodiments of this application, the global semantic enhancement of the optimized first feature further includes: Spatial average pooling is performed on the channel compressed features to construct a global feature representation. The global feature representation is then multiplied element-wise with the channel compressed features to achieve channel-level contrast enhancement, resulting in enhanced global features. The enhanced global features are convolved using a 7×7 convolution kernel, and then gated spatial attention is generated using an activation function. The gated spatial attention is processed by basic convolution and activation functions to generate spatial attention weights, and the optimized first feature is adaptively recalibrated by element-wise multiplication to obtain the enhanced first feature.

[0016] In some embodiments of this application, the decoder is a visual state space model decoder, which decodes the fused features using a progressive upsampling framework; The progressive upsampling framework achieves hierarchical feature fusion through successive transposed convolutions and generates a high-resolution segmentation mask as the segmentation result through a prediction head with shared parameters.

[0017] In some embodiments of this application, the medical image is a cardiac magnetic resonance image, and the segmentation result is used in a medical robot system configured to perform at least one of preoperative planning, intraoperative navigation, and postoperative screening based on the segmentation result.

[0018] A second aspect of this application provides a medical image segmentation apparatus, the apparatus comprising: The acquisition module is used to acquire the medical image to be segmented; The input module is used to input the medical image into a dual-stream encoder for feature extraction. The dual-stream encoder includes a first encoder branch based on a pre-trained visual basic model and a second encoder branch based on a visual state space model. The first encoder branch encodes the medical image to obtain a first feature, and the second encoder branch encodes the medical image to obtain a second feature. An optimization module is used to perform domain adaptive optimization processing on the first feature through a dynamic feature fusion optimizer to obtain an optimized first feature. The dynamic feature fusion optimizer is used to bridge the inter-domain differences between natural images and medical images. The fusion module is used to perform cross-architecture feature fusion on the optimized first feature and the second feature through the heterogeneous omnidirectional attention convergence module to obtain the fused feature; The segmentation module is used to input the fused features into the decoder for decoding processing to obtain the segmentation result of the medical image.

[0019] A third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the medical image segmentation method described in the embodiments of this application.

[0020] The fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, characterized in that the computer program is executed by a processor to implement the medical image segmentation method described in the embodiments of this application.

[0021] This application provides a medical image segmentation method, which involves acquiring a medical image to be segmented; inputting the medical image into a two-stream encoder for feature extraction; the two-stream encoder includes a first encoder branch based on a pre-trained visual basic model and a second encoder branch based on a visual state space model; the first encoder branch encodes the medical image to obtain a first feature, and the second encoder branch encodes the medical image to obtain a second feature; this method effectively preserves the general visual prior knowledge learned in the natural image pre-training stage, avoiding the risk of overfitting under small sample medical data; while the parallel visual state space model branch captures the global long-range dependence of the heart structure with linear computational complexity, making up for the shortcomings of the SAM2 window attention mechanism in global topology modeling, and the two complement each other. The first feature is optimized by performing domain adaptive optimization on the first feature through a dynamic feature fusion optimizer. The dynamic feature fusion optimizer is used to bridge the inter-domain differences between natural images and medical images. The optimizer uses a dual-path adaptive pooling mechanism to retain the semantic information of significant anatomical structures such as the thin wall of the right ventricle in the SAM2 feature through adaptive max pooling. At the same time, it uses adaptive average pooling to suppress noise interference and artifacts unique to medical images. Combined with residual connection and layer normalization processing, it significantly improves the adaptability of pre-trained features to the domain distribution of medical images and effectively solves the problem of segmentation accuracy degradation caused by domain offset mentioned in the background technology. A heterogeneous omnidirectional attention convergence module performs cross-architecture feature fusion on the optimized first and second features to obtain fused features. This module uses a bi-branch selective emphasis attention submodule to capture long-distance cross-regional dependencies of VMamba features using spatial average pooling and spatial max pooling to enhance local salient patterns, preventing the loss of key textures caused by sequential scanning. Simultaneously, an omniscient context attention submodule performs channel-level contrast enhancement and gated spatial attention recalibration on the optimized SAM2 features, achieving deep collaboration between local details and global semantics, overcoming the semantic misalignment and detail blurring problems caused by simple bi-branch feature fusion in existing technologies. Finally, the fused features are input into the decoder for decoding to obtain the segmentation result of the medical image. Then, through a progressive upsampling framework and continuous transposed convolution operations, high-resolution spatial details are restored while maintaining consistency with the encoder architecture. A high-precision segmentation mask is output through a prediction head with shared parameters.

[0022] In summary, the medical image segmentation method provided in this application significantly enhances the clarity of segmentation boundaries and the continuity of anatomical structures, providing a more reliable quantitative basis for anatomical structures in preoperative planning, intraoperative navigation, and postoperative screening of medical robot systems.

[0023] The above description is merely an overview of the technical solutions of the embodiments of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the embodiments of this application more obvious and understandable, specific implementation methods of this application are described below. Attached Figure Description

[0024] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiments below. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 This is a schematic diagram illustrating the steps of a medical image segmentation method provided in some embodiments of this application; Figure 2 This is a schematic diagram illustrating the workflow of a medical robot control system provided in some embodiments of this application; Figure 3 A schematic diagram illustrating the architecture of a medical image segmentation method provided in some embodiments of this application; Figure 4 This is a schematic diagram of the architecture of the dynamic feature fusion optimizer provided in some embodiments of this application; Figure 5 This is a schematic diagram of the architecture of a heterogeneous omnidirectional attention convergence module provided in some embodiments of this application; Figure 6 A schematic diagram showing the segmentation performance comparison of different models provided in some embodiments of this application on a cardiac magnetic resonance imaging dataset; Figure 7 This is a schematic diagram of the structure of a medical image segmentation device provided in an embodiment of this application; Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0025] The embodiments of the technical solution of this application will now be described in detail with reference to the accompanying drawings. These embodiments are only used to more clearly illustrate the technical solution of this application and are therefore merely examples, and should not be used to limit the scope of protection of this application.

[0026] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which embodiments of this application pertain; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the embodiments of this application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings of this application are intended to cover non-exclusive inclusion.

[0027] In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0028] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0029] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0030] In the clinical application of cardiac magnetic resonance imaging (CMR), high-precision automatic segmentation based on deep learning is a key prerequisite for realizing preoperative planning and intraoperative navigation of medical robots. Existing solutions mainly face three technical bottlenecks: First, although general-purpose visual basic models such as the SAM series have strong natural image representation capabilities, direct transfer due to the significant difference in domain distribution between pre-training data and medical images can lead to a serious degradation in the segmentation accuracy of delicate anatomical structures such as the thin wall of the right ventricle. Second, although visual state space models (VMamba / Mamba) can capture long-range dependencies with linear complexity, they are prone to weakening local significant texture features during sequential scanning, resulting in blurred boundary localization. Third, existing dual-branch fusion strategies mostly adopt simple weighted summation or conventional cross-attention mechanisms, failing to collaboratively optimize the heterogeneous characteristics of SAM features, which are prone to losing spatial details, and Mamba features, which lack semantic calibration, resulting in semantic misalignment of fused features.

[0031] Therefore, this application provides a medical image segmentation method that can effectively bridge inter-domain differences and achieve a segmentation scheme that deeply coordinates dual-stream heterogeneous features, thereby meeting the practical needs of high-precision segmentation in clinical practice. Figure 1 As shown, the method includes S1-S5: S1. Obtain the medical image to be segmented.

[0032] Specifically, the medical images are cardiac magnetic resonance (CMR) images, such as short-axis slices from the ACDC dataset. These medical images, used as input data, are uniformly preprocessed to a fixed resolution (e.g., 256×256) and normalized to the [0,1] interval for input into a dual-stream encoder for parallel feature extraction.

[0033] In this embodiment, the medical image is a cardiac magnetic resonance image, and the segmentation result is used by a medical robot system, which is configured to perform at least one of preoperative planning, intraoperative navigation, and postoperative screening based on the segmentation result. Figure 2 This is a schematic diagram illustrating the workflow of the medical robot control system provided in an embodiment of this application. Figure 2 As shown, the medical robot control system includes a robot-assisted cardiac magnetic resonance imaging (MRI) scanning stage on the left, an image processing stage in the middle, and a three-stage intelligent robot control system on the right. The image processing stage includes a visual state space model branch for global semantic feature extraction and a segmentation model branch for spatial location semantic feature extraction. The three-stage intelligent robot control system includes preoperative planning, intraoperative navigation, and postoperative screening. With the above configuration, the medical image segmentation method provided in this embodiment can provide high-precision anatomical structure segmentation results for the medical robot system, helping to achieve a seamless clinical closed loop from preoperative planning to intraoperative navigation.

[0034] Specifically, for example Figure 2 As shown in the figure, the "Acquire Image Data" label on the left corresponds to the step of acquiring the medical image to be segmented; "Assist Medical Diagnosis" indicates that the segmentation results are used to assist medical diagnosis. In the intermediate image processing stage, "Global Semantic Feature Extraction" corresponds to the processing process of the VMamba branch of the visual state space model, and "Spatial Location Semantic Feature Extraction" corresponds to the processing process of the SAM2 branch based on the second-generation segmentation model; "Generate Segmentation Mask" indicates that a high-resolution segmentation mask is generated through the decoder; "Display Heart Location and Morphology" indicates that the segmentation results are presented in a three-dimensional visualization of the heart's anatomical structure; "Multi-Slice Data Input" indicates that multi-slice cardiac magnetic resonance image data is input into the intelligent robot control system. In the three-stage intelligent robot control system on the right, "Surgical Planning and Simulation" corresponds to the preoperative planning stage, "Robot-Assisted Surgery" and "Augmented Reality / Mixed Reality" correspond to the intraoperative navigation stage, and "Screening Complications" corresponds to the postoperative screening stage.

[0035] S2. Input the medical image into a dual-stream encoder for feature extraction. The dual-stream encoder includes a first encoder branch based on a pre-trained visual basic model and a second encoder branch based on a visual state space model. The first encoder branch encodes the medical image to obtain a first feature, and the second encoder branch encodes the medical image to obtain a second feature.

[0036] In one specific implementation, the first encoder branch based on the pre-trained visual basic model is an encoder branch based on the second-generation segmentation model, namely SAM2, and the second encoder branch based on the visual state space model is an encoder branch based on the visual state space model.

[0037] For the first encoder branch, the encoder branch based on the second-generation segmentation model encodes the medical image using a frozen layered image encoder, and adaptively adjusts the multi-scale features of the layered image encoder through a multilayer perceptron adapter.

[0038] Specifically, this application employs the Hiera-Large architecture of SAM2 as the backbone network of the first encoder branch. During the training phase, all weight parameters of the Hiera encoder are frozen to preserve its general visual representation capabilities learned during the large-scale natural image pre-training phase. To address the domain distribution differences between medical images and natural images, multilayer perceptron (MLP) adapters are introduced at different levels of the Hiera encoder. These MLP adapters perform nonlinear transformations and adaptive adjustments on the multi-scale feature maps output by the encoder, thereby effectively bridging the inter-domain differences between natural and medical images without compromising the pre-trained representations, and outputting the first feature.

[0039] For the second encoder branch, the encoder branch based on the visual state space model uses a visual state space module to encode the medical image in order to capture global semantic context features.

[0040] Specifically, this application employs the VMamba (Visual Mamba) network as the second encoder branch. VMamba, based on the Selective State Space Model (S6), uses a 2D Selective Scan (SS2D) mechanism to sequentially model the input image. Unlike the quadratic complexity self-attention mechanism of the Transformer, VMamba can capture long-range dependencies between distant pixels in cardiac MRI with linear computational complexity. It is particularly suitable for modeling the overall spatial topology of anatomical structures such as the left ventricle, right ventricle, and myocardium, outputting the second feature.

[0041] Figure 3 This is a schematic diagram of the architecture of a medical image segmentation method. Figure 3 As shown, the architecture is a U-shaped architecture, comprising a dual-stream encoder and a single visual state-space model decoder. In the encoder branch based on the second-generation segmentation-everything model, a dual-adaptor architecture with a dynamic feature fusion optimizer is employed to perform domain-adaptive optimization of the attention output, combined with parallel multilayer perceptron adapter operations to enhance nonlinear mapping capabilities. In the encoder branch based on the visual state-space model, a visual state-space module is used to capture global semantic context, addressing the pixel-level spatial semantic loss problem caused by the windowed positional encoding mechanism in the encoder branch based on the second-generation segmentation-everything model.

[0042] Specifically, for example Figure 3 As shown in the figure, the “SAM2 encoder” labeled in the figure represents the hierarchical structure of the encoder branch based on the second-generation segmentation model; “HOACM” represents the heterogeneous omnidirectional attention convergence module, which is inserted between the layers of the encoder for cross-architecture feature fusion. The “VSS block” labeled in the decoder section refers to the visual state space block, corresponding to the visual state space module; “Patch merging” represents the block merging operation, used for multi-scale feature aggregation in the downsampling stage; “Block augmentation” represents the feature augmentation operation in the upsampling stage, corresponding to the transposed convolution in the progressive upsampling framework; “Linear embedding” represents the linear embedding layer, used to map input features to a high-dimensional space. In the SAM2 Hiera block section of the figure, the “optimizer” corresponds to the dynamic feature fusion optimizer, and the “fine-tuner” corresponds to the multilayer perceptron adapter; both are marked as learnable parameters (identified by flame icons in the figure), while the SAM2 encoder backbone parameters are marked as frozen parameters (identified by snowflake icons in the figure). The "Depth-aware Convolution" in the legend refers to depthwise separable convolution, used for feature transfer between layers of the encoder; "Residual Connection" refers to skip connections, used to preserve multi-scale information.

[0043] S3. The first feature is subjected to domain adaptive optimization processing by a dynamic feature fusion optimizer to obtain the optimized first feature. The dynamic feature fusion optimizer is used to bridge the inter-domain differences between natural images and medical images.

[0044] In one specific implementation, the first feature is optimized by a dynamic feature fusion optimizer through domain adaptive optimization, specifically including the following steps: First, the first feature is rearranged in dimensions. Since the multi-scale features output by the SAM2 Hiera encoder are typically organized as a tensor format of (Batch, Heads, Length, Channels), while the expected input format for subsequent convolution operations is (Batch, Channels, Height, Width), it is necessary to first rearrange the dimensions to adjust it into a four-dimensional feature tensor suitable for two-dimensional convolution processing. Subsequently, adaptive max pooling and adaptive average pooling are performed on the rearranged first feature, respectively. The adaptive max pooling is used to preserve globally significant semantic information, and the adaptive average pooling is used to suppress high-frequency noise interference.

[0045] The features obtained after adaptive max pooling and adaptive average pooling are concatenated along the channel dimension to obtain the concatenated features. Specifically, the first features after dimensional rearrangement are subjected to adaptive average pooling and adaptive max pooling respectively, and then concatenated along the channel dimension after dimensional rearrangement, as shown in the following formula:

[0046] The optimized first feature is obtained based on the first feature and the concatenated feature. Specifically, the first feature is processed sequentially through a first convolutional layer and a first activation function, and then through a transposed convolutional layer and a second activation function to obtain the spatial feature. As shown in the following formula:

[0047] In formula (1), X represents the input feature, i.e., the attention output of the SAM2 encoder, X∈R (B×H×W×C) B represents the batch size; H and W represent the height and width of the feature map, respectively; C represents the number of channels; Permute(·) represents the dimension rearrangement operation, corresponding to the dimension rearrangement step described above; AdaptiveAvgPool2d(·) represents adaptive average pooling, used to suppress high-frequency noise interference; AdaptiveMaxPool2d(·) represents adaptive max pooling, used to preserve globally significant semantic information; Concat(·) represents the concatenation operation along the channel dimension, corresponding to the step of concatenating the features after adaptive max pooling and the features after adaptive average pooling along the channel dimension; X cat This indicates splicing characteristics.

[0048] In formula (2), X attX represents the attention output feature, i.e., the input feature X; Conv2D(·) represents the convolution operation; K_d and K_u represent the learnable convolution kernel parameters, respectively; ReLU(·) represents the ReLU activation function; DeConv2D(·) represents the transposed convolution operation; X_sp represents the spatial feature, corresponding to the spatial feature obtained by processing the first feature sequentially through the first convolutional layer and the first activation function, and then through the transposed convolutional layer and the second activation function.

[0049] The concatenated features and the spatial features are fused using residual connections to obtain residual fused features. Residual connections fuse the refined features with the original input to preserve multi-scale information, while layer normalization is applied to stabilize gradient propagation and ensure numerical stability during training. The residual fused features are then subjected to dimensionality rearrangement and layer normalization to obtain the optimized first feature, as shown in the following formula:

[0050] Among them, X sp Represents spatial features; "+" indicates residual join, such as element-wise addition, corresponding to the step of fusing the spliced ​​features and spatial features through residual join; Permute(·) indicates dimensional rearrangement operation; LayerNorm(·) indicates layer normalization processing, corresponding to the steps of performing dimensional rearrangement and layer normalization processing on the residual fused features; X out This represents the first feature after optimization.

[0051] Figure 4 This is a schematic diagram of the architecture of the dynamic feature fusion optimizer provided in an embodiment of this application. Figure 4 As shown, the dynamic feature fusion optimizer uses the attention output as the input feature and employs dual adaptive pooling operations: adaptive max pooling preserves globally salient semantics, while adaptive average pooling suppresses high-frequency noise interference. Through this configuration, the dynamic feature fusion optimizer can effectively bridge the inter-domain differences between natural and medical images, significantly improve the adaptability of pre-trained features to the medical image domain distribution, and effectively solve the segmentation accuracy degradation problem caused by domain shift.

[0052] Specifically, for example Figure 4As shown in the figure, the "Input" marked in the figure corresponds to the attention output, i.e., the first feature; "Max Pooling" and "Average Pooling" correspond to the adaptive max pooling and adaptive average pooling, respectively; the concatenated features pass through the "Channel Attention" module, which includes a "Linear Layer," a "GELU" activation function, and a "Sigmoid" activation function, used to perform channel-dimensional attention weighting on the concatenated features; "2D Convolution" corresponds to the first convolutional layer, processed with the ReLU activation function; "Transposed Convolution" corresponds to the transposed convolutional layer, also processed with the ReLU activation function; "Output" corresponds to the optimized first feature. The feedback arrow in the figure represents the residual connection, that is, the fusion of the concatenated features and spatial features by element-wise addition.

[0053] S4. The optimized first feature and the second feature are fused across architectures using a heterogeneous omnidirectional attention convergence module to obtain a fused feature.

[0054] In this embodiment, the heterogeneous omnidirectional attention convergence module includes a bi-branch selective emphasis attention submodule and an omniscient context attention submodule. The bi-branch selective emphasis attention submodule is used to dynamically enhance the second feature using a channel-space dual-path approach to obtain the enhanced second feature. The omniscient context attention submodule is used to perform global semantic enhancement on the optimized first feature to obtain the enhanced first feature. The heterogeneous omnidirectional attention convergence module is also used to hierarchically integrate the enhanced second feature, the enhanced first feature, and historical fusion semantics through a cross-attention fusion mechanism to obtain the fused feature. Figure 5 This is a schematic diagram of the architecture of the heterogeneous omnidirectional attention convergence module provided in an embodiment of this application. Figure 5 As shown, the heterogeneous omnidirectional attention convergence module includes three key components: omniscient contextual attention, bi-branch selective emphasis attention, and cross-attention fusion mechanism.

[0055] like Figure 5As shown, the heterogeneous omnidirectional attention convergence module includes a bi-branch selective emphasis attention submodule and an omniscient context attention submodule. The bi-branch selective emphasis attention submodule receives the second feature output by the visual state space model encoder and extracts it through both a spatial average pooling path and a spatial max pooling path. The spatial average pooling path is used to capture long-distance dependencies across regions, while the spatial max pooling path is used to preserve local salient patterns. The features from both paths are input into a shared-weight adaptive self-attention mechanism, generating a query matrix Q, a key matrix K, and a value matrix V through a 1×1 convolutional layer. After similarity matrix calculation and normalization, a spatial relationship weight matrix M is obtained, which is then multiplied with the value matrix V to obtain the bi-path attention features. Finally, a spatial attention map is generated by weighted summation and normalization, and the second feature is enhanced element-wise to obtain the enhanced second feature. The omniscient context attention submodule receives the optimized first feature, performs spatial max pooling and spatial average pooling along the channel dimension, concatenates the two to obtain channel compressed features, then constructs a global feature representation through spatial average pooling and performs channel-level contrast enhancement. Subsequently, it uses a 7×7 convolutional kernel to model long-distance spatial dependencies to generate gated spatial attention, and finally generates spatial attention weights through basic convolution and sigmoid activation to adaptively recalibrate the first feature, obtaining the enhanced first feature. The heterogeneous omnidirectional attention convergence module also integrates the enhanced second feature, the enhanced first feature, and historical fusion semantics hierarchically through a cross-attention fusion mechanism to obtain the fused feature.

[0056] Specifically, for example Figure 5 As shown in the figure, the "Bi-branch Selective Emphasis Attention" submodule is the Bi-branch Selective Emphasis Attention submodule, where "Adaptive Attention" corresponds to the adaptive self-attention mechanism with shared weights. The labels "Shared" and "Weight" indicate that the two paths share the same set of weight parameters. The "Omniscient Context Attention" submodule is the Omniscient Context Attention (OCA) submodule; "Global Context-Aware Attention (GCAA)" represents the Global Context-Aware Attention component within the OCA submodule, where "Average Pooling" corresponds to the spatial average pooling used to construct global feature representations, "7×7 Convolution (pad=3)" corresponds to the 7×7 convolution kernel, "GSA" is Global Spatial Attention, corresponding to the gated spatial attention, "Basic Convolution" corresponds to the basic convolution, and "Global Spatial Attention" represents the spatial attention weights generated through the basic convolution and activation function. The "Inverted Residual Attention" submodule corresponds to the cross-attention fusion mechanism, used to hierarchically integrate the enhanced second feature, the enhanced first feature, and historical fusion semantics. "SortMax" is the Softmax normalization function, used to normalize the attention weights into a probability distribution.

[0057] In one specific implementation, the channel-spatial dual-path dynamic enhancement of the second feature includes: performing spatial average pooling and spatial max pooling on the second feature, respectively. Spatial average pooling is used to capture long-distance dependencies across regions, and spatial max pooling is used to preserve local salient patterns. Specifically, given the features output layer-by-layer by the visual state space model encoder, a parallel spatial max pooling path is designed to retain the inherent advantage of the visual state space model in capturing local salient patterns. This design enhances sensitivity to key segmentation boundaries by emphasizing the extreme features of regional responses, as shown in the following formula:

[0058]

[0059] Among them, X mamba The second feature, X, represents the feature output layer by layer by the encoder of the visual state space model. mamba ∈R (B×C×H×W) AvgPool(·) represents spatial average pooling, used to capture long-range dependencies across regions, corresponding to the aforementioned spatial average pooling; MaxPool(·) represents spatial max pooling, used to preserve locally salient patterns, corresponding to the aforementioned spatial max pooling; X avg Represents the features after spatial average pooling; X max This represents the features after spatial max pooling. In some embodiments of this application, the features after spatial average pooling and the features after spatial max pooling are respectively input into a shared-weight adaptive self-attention mechanism for processing to obtain dual-path attention features. The processing of the shared-weight adaptive self-attention mechanism includes: performing convolution processing on the pooled features through three independent 1×1 convolutional layers and flattening them to generate a query matrix, a key matrix, and a value matrix. Specifically, the three independent 1×1 convolutional layers generate query, key, and value triples, as shown in the following formula:

[0060] Among them, Conv2D 1×1 (·) represents a 1×1 convolution operation; W q W k and W v represents the learnable convolutional kernel parameters for generating the query matrix, key matrix, and value matrix, respectively, corresponding to the three independent 1×1 convolutional layers; Flatten(·) represents the flattening operation; Q represents the query matrix; K represents the key matrix; V represents the value matrix; Q, K, V∈R (B×C×N)Where B represents the batch size, C represents the number of channels, and N = H × W represents the flattened spatial dimension. The convolution kernel is a learnable parameter used to generate the corresponding query matrix, key matrix, and value matrix, where the flattened spatial dimension is the product of height and width. Subsequently, the similarity matrix between the query matrix and the key matrix is ​​calculated, and the similarity matrix is ​​normalized using a normalization function to obtain the spatial relationship weight matrix. As shown in the following formula:

[0061] Where Q represents the query matrix; K T Represents the transpose of the bond matrix; Q×K T The similarity matrix between the query matrix and the key matrix is ​​represented by Softmax(·), which corresponds to the normalization function mentioned above. dim=-1 indicates normalization along the last dimension. M represents the spatial relation weight matrix, i.e., the normalized attention weight matrix. The spatial relation weight matrix and the value matrix are multiplied by matrix multiplication to dynamically aggregate contextual information, thus obtaining the dual-path attention features. As shown in the following formula:

[0062] Where M represents the spatial relation weight matrix; V represents the value matrix; M×V represents the matrix multiplication of the attention weight matrix and the value matrix, used to dynamically aggregate contextual information, corresponding to the step of performing matrix multiplication of the spatial relation weight matrix and the value matrix; the subscript "avg / max" of X' indicates that the formula is applicable to both the spatial average pooling path and the spatial max pooling path, that is, the same matrix multiplication M×V is applied independently to the two paths, resulting in two independent dual-path attention features: X' avg X' represents the attention feature obtained after the spatial average pooling path is processed by an adaptive self-attention mechanism with shared weights. max This represents the attention feature obtained after the spatial max-pooling path is processed by an adaptive self-attention mechanism with shared weights. The dual-path attention features are fused through weighted summation and normalized to generate a spatial attention map. The enhanced second feature is then obtained by element-wise multiplication, as shown in the following formula:

[0063] Where X' avg The dual-path attention feature representing the spatial average pooling path; X' maxThe first part represents the dual-path attention features of the spatial max-pooling path; Softmax(·) represents the normalization function, used to normalize the fused dual-path attention features into a probability distribution form, corresponding to the normalization process described above; "+" represents element-wise addition, used to fuse the attention features of the spatial average pooling path and the spatial max-pooling path, corresponding to the weighted summation process described above; ⊙ represents element-wise multiplication, used to enhance the spatial attention map and the original features element-wise; X mamba The second feature represents the layer-by-layer output of the visual state space model encoder. Through the above operations, the fused and normalized spatial attention map is multiplied element-by-element by the second feature to achieve dynamic enhancement of the second feature through both channel and spatial paths, resulting in the enhanced second feature.

[0064] In one specific implementation, global semantic enhancement of the optimized first feature includes: performing spatial max pooling and spatial average pooling along the channel dimension to extract salient features and capture global contextual information, respectively. The features obtained from spatial max pooling and spatial average pooling are then concatenated along the channel dimension to obtain channel-compressed features. Specifically, the omniscient context attention mechanism initially performs dual-channel compression: spatial max pooling along the channel dimension to extract salient features, and spatial average pooling to capture global contextual information, to achieve multi-granularity feature representation. As shown in the following formula:

[0065]

[0066] Among them, X sam The feature represented by X, which is the optimized first feature, is the feature output layer by layer by the SAM2 encoder. sam ∈R (B ×C×H×W) max{c∈C}(·) represents spatial max pooling along the channel dimension, used to extract salient features, corresponding to the aforementioned spatial max pooling; (1 / C)Σ(·) represents spatial average pooling along the channel dimension, used to capture global context information, corresponding to the aforementioned spatial average pooling; C represents the number of channels; X_Msam represents the salient features; X Asam Represents global context features; Concat(·) represents concatenation operation along the channel dimension; X cat X represents the channel compression feature. cat ∈R (B ×2×H×W) , where 2 represents the number of channels after splicing.

[0067] In another specific implementation, global semantic enhancement of the optimized first feature further includes: performing spatial average pooling on the channel compressed feature to construct a global feature representation, and multiplying the global feature representation element-wise with the channel compressed feature to achieve channel-level contrast enhancement, resulting in the enhanced global feature. Specifically, a global feature representation is constructed through spatial average pooling, and channel-level contrast enhancement is performed using the channel compressed feature, as shown in the following formula:

[0068] AvgPool(·) represents spatial average pooling, used to construct a global feature representation, corresponding to the step of constructing a global feature representation by performing spatial average pooling on the channel compressed features; ⊙ represents element-wise multiplication, used to achieve channel-level contrast enhancement, corresponding to the step of performing element-wise multiplication of the global feature representation with the channel compressed features to achieve channel-level contrast enhancement; X global This represents the enhanced global features.

[0069] The enhanced global features are convolved using a 7×7 convolution kernel, and then gated spatial attention is generated using an activation function, as shown in the following formula:

[0070] Among them, Conv2D 7×7, padding=3 (·) represents a convolution operation with a kernel size of 7×7 and padding of 3, used to model long-range spatial dependencies, corresponding to the step of convolution processing the enhanced global features using a 7×7 convolution kernel; σ(·) represents the Sigmoid activation function, corresponding to the step of generating gated spatial attention through the activation function; ⊙ represents element-wise multiplication; X gsa This represents the gated spatial attention feature. The gated spatial attention is processed using basic convolution and activation functions to generate spatial attention weights. The optimized first feature is then adaptively recalibrated by element-wise multiplication to obtain the enhanced first feature. As shown in the following formula:

[0071] Among them, BasicConv 7×7 (·) represents the basic 7×7 convolution operation, corresponding to the step of processing the gated spatial attention through basic convolution; σ(·) represents the Sigmoid activation function, corresponding to the step of generating spatial attention weights through the activation function; ⊙ represents element-wise multiplication for adaptive recalibration, corresponding to the step of adaptively recalibrating the optimized first feature through element-wise multiplication; X' sam This represents the enhanced first feature.

[0072] With the above configuration, the heterogeneous omnidirectional attention convergence module uses a dual-branch selective emphasis attention submodule to capture the cross-regional long-distance dependencies of visual state space model features using spatial average pooling, and uses spatial max pooling to enhance local salient patterns, preventing the loss of key textures caused by sequential scanning. At the same time, the omniscient context attention submodule performs channel-level contrast enhancement and gated spatial attention recalibration on the optimized first feature, realizing deep collaboration between local details and global semantics, and overcoming the semantic misalignment and detail blurring problems caused by simple fusion of dual-branch features.

[0073] S5. Input the fused features into the decoder for decoding to obtain the segmentation result of the medical image.

[0074] In this embodiment, the decoder is a visual state space model decoder, which decodes the fused features using a progressive upsampling framework. This progressive upsampling framework achieves hierarchical feature fusion through successive transposed convolutions and generates a high-resolution segmentation mask as the segmentation result using a prediction head with shared parameters. Specifically, the decoder design maintains architectural consistency with the visual state space model U-shaped network by implementing a progressive upsampling framework. This multi-stage feature aggregation mechanism achieves hierarchical feature fusion through successive transposed convolutions and ultimately generates a high-resolution segmentation mask using a prediction head with shared parameters. Through this configuration, high-resolution spatial details are restored while maintaining consistency with the encoder architecture, and a high-precision segmentation mask is output through a prediction head with shared parameters.

[0075] In one specific implementation, the medical image is a cardiac magnetic resonance image, and the segmentation result is used by a medical robot system configured to perform at least one of preoperative planning, intraoperative navigation, and postoperative screening based on the segmentation result. Figure 6 A schematic diagram showing the segmentation performance comparison of different models provided in the embodiments of this application on a cardiac magnetic resonance dataset. Figure 6The columns from left to right show a comparison of segmentation results for Image (original image), Ground Truth (ground truth annotation), UNet, Mamba-UNet, and SAMba-UNet (this application). The red boxes highlight key comparison areas to visually demonstrate the differences in segmentation accuracy at cardiac structure boundaries. The medical image segmentation method provided in this application achieves an average Dice coefficient of 0.9103 and a boundary error of 1.0859 mm on a cardiac magnetic resonance imaging dataset, achieving state-of-the-art performance and significantly improving the boundary localization accuracy of complex structures such as the right ventricle. In summary, the medical image segmentation method provided in this application significantly enhances the clarity of segmentation boundaries and the continuity of anatomical structures, providing a more reliable quantitative basis for anatomical structures in preoperative planning, intraoperative navigation, and postoperative screening of medical robot systems.

[0076] To verify the effectiveness of the medical image segmentation method provided in the embodiments of this application, experimental verification was conducted on a publicly available cardiac magnetic resonance dataset, as detailed below.

[0077] The experiment used the Automated Cardiac Diagnosis Challenge (ACDC) dataset, which contains cardiac magnetic resonance imaging sequences from 100 patients, covering normal cases and four pathological conditions: dilated cardiomyopathy (DCM), hypertrophic cardiomyopathy (HCM), myocarditis (MINF), and right ventricular abnormalities (RV-abnormal). Following the ACDC protocol, the end-diastolic (ED) and end-systolic (ES) phases were evaluated, and the average performance for both phases was reported. Region-based metrics (including Dice coefficient, IoU, accuracy, precision, sensitivity, and specificity) and boundary-aware metrics (including HD95 and ASD) were used to evaluate segmentation performance. The performance of different models on the ACDC dataset is shown in Table 1. Table 1 Performance of different models on the ACDC dataset

[0078] Table 1 shows the average performance of the ED and ES stages. The SAMba-UNet provided in this application achieves state-of-the-art performance on the ACDC dataset with an mDice score of 0.9103, 0.71% higher than the second-ranked LeViT-UNet-384. It demonstrates significant improvement in complex structures, achieving a right ventricular segmentation accuracy of 0.9039, a 0.94% improvement over the second-best model; and a myocardial MYO segmentation accuracy of 0.8935, a 0.84% ​​improvement over the second-best model. In the left ventricular (LV) segmentation task, this application's embodiment maintains competitiveness with models such as UNet++ (0.9507), R50 Attn-UNet (0.9347), and SwinUNet (0.9449). These results validate that this application, by fusing SAM2 and a state-space model, has significant advantages in boundary localization and global consistency.

[0079] To further verify the effectiveness of each module, ablation experiments were conducted. Table 2 shows the effectiveness of different components in the model architecture, and Table 3 shows the effectiveness of different adapters and their components.

[0080] Table 2. Validity of different components in the model architecture

[0081] As shown in Table 2, the complete configuration achieved the best performance across all metrics (mDice = 0.9103, mHD95 = 1.0859, ASD = 0.2611). Removing the OCA (Omniscient Context Attention) module decreased mDice to 0.9037 and increased ASD to 0.3501, indicating that the OCA module is crucial for boundary localization. Removing the BSEA (Bi-branch Selective Emphasis Attention) module decreased mDice to 0.9064 and increased mHD95 to 1.1441, indicating that the BSEA module plays a significant role in preserving local salient patterns. Removing adaptive attention significantly deteriorated mHD95 to 1.8263 and increased ASD to 0.5154, indicating that the adaptive attention mechanism contributes the most to boundary perception accuracy. These results validate the functional complementarity of the modules.

[0082] Table 3. Effectiveness of Different Adapters and Their Components

[0083] As shown in Table 3, in the adapter ablation experiment, after removing the refining module, mDice decreased to 0.9028, mHD95 deteriorated significantly to 1.9504, and ASD increased to 0.4936, indicating that the dynamic feature fusion optimizer made the most critical contribution to the boundary localization accuracy. After removing the channel attention, mDice decreased to 0.9043, and after removing the MLP adapter, mDice decreased to 0.9054, indicating that the channel attention and MLP adapter were mainly used to improve the accuracy of fine-grained features. The complete configuration achieved optimal results on all metrics, verifying the synergistic enhancement effect of each adapter component.

[0084] Based on the above experimental results, the medical image segmentation method provided in this application represents a systemic breakthrough. Firstly, by using a dynamic feature fusion optimizer to perform domain-adaptive optimization on the first feature output by the SAM2 encoder, the inter-domain differences between natural and medical images are effectively bridged. This solves the problem of segmentation accuracy degradation caused by mismatched domain distribution of pre-trained data when a general visual basic model is directly applied to cardiac MRI segmentation. In the ablation experiment, after removing the refinement module, mHD95 significantly deteriorated to 1.9504 mm and ASD increased to 0.4936 mm, verifying the module's crucial contribution to boundary localization accuracy. Secondly, the dual-branch selective emphasis attention submodule in the heterogeneous omnidirectional attention convergence module captures the cross-regional long-distance dependencies of VMamba features using spatial average pooling and strengthens local salient patterns using spatial max pooling, overcoming the key texture loss problem caused by sequential scanning of the state space model. At the same time, the omniscient context attention submodule performs channel-level contrast enhancement and gated spatial attention recalibration on the optimized first feature, solving the pixel-level spatial position semantic loss problem caused by the SAM windowing position encoding mechanism. Furthermore, the cross-attention fusion mechanism achieves deep collaboration of dual-stream heterogeneous features, overcoming the semantic misalignment and detail blurring problems caused by simple fusion strategies in existing schemes. In the ablation experiment, the ASD increased to 0.3501 after removing OCA and the mHD95 increased to 1.1441mm after removing BSEA, verifying the functional complementarity of each module. Third, the progressive upsampling framework of the VMamba decoder maintains architectural consistency while restoring high-resolution spatial details. Ultimately, it achieves state-of-the-art performance on the ACDC dataset with mDice of 0.9103, mHD95 of 1.0859mm, and ASD of 0.2611mm. This significantly enhances the clarity of segmentation boundaries and the continuity of anatomical structures, providing high-precision quantitative data on anatomical structures for preoperative planning, intraoperative navigation, and postoperative screening of medical robot systems.

[0085] In some embodiments of this application, a medical image segmentation apparatus is also provided, such as... Figure 7 As shown, the device includes: The acquisition module 701 is used to acquire the medical image to be segmented; The input module 702 is used to input the medical image into a dual-stream encoder for feature extraction. The dual-stream encoder includes a first encoder branch based on a pre-trained visual basic model and a second encoder branch based on a visual state space model. The first encoder branch encodes the medical image to obtain a first feature, and the second encoder branch encodes the medical image to obtain a second feature. The optimization module 703 is used to perform domain adaptive optimization processing on the first feature through a dynamic feature fusion optimizer to obtain the optimized first feature. The dynamic feature fusion optimizer is used to bridge the inter-domain differences between natural images and medical images. The fusion module 704 is used to perform cross-architecture feature fusion on the optimized first feature and the second feature through the heterogeneous omnidirectional attention convergence module to obtain the fused feature; The segmentation module 705 is used to input the fused features into the decoder for decoding processing to obtain the segmentation result of the medical image.

[0086] The medical image segmentation device provided in this embodiment is used to perform the above-described location selection method, and therefore can at least achieve the beneficial effects that can be achieved by each implementation of the above method. Moreover, each implementation of the above method is also applicable to this device, and will not be described again here.

[0087] In some embodiments of this application, an electronic device is also provided to perform the above-described medical image segmentation method. Please refer to... Figure 8 This illustrates a schematic diagram of an electronic device provided by some embodiments of this application. For example... Figure 8 As shown, the electronic device 9 includes: a processor 901, a memory 902, a bus 903, and a communication interface 904. The processor 901, the communication interface 904, and the memory 902 are connected via the bus 903. The memory 902 stores a computer program that can run on the processor 901. When the processor 901 runs the computer program, it executes the medical image segmentation method provided in any of the foregoing embodiments of this application.

[0088] The memory 902 may include high-speed random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Communication between this device network element and at least one other network element is achieved through at least one communication interface 904 (which can be wired or wireless), such as the Internet, wide area network, local area network, metropolitan area network, etc.

[0089] Bus 903 can be an ISA bus, PCI bus, or EISA bus, etc. Buses can be divided into address buses, data buses, control buses, etc. Memory 902 is used to store programs. After receiving execution instructions, processor 901 executes the programs. The medical image segmentation method disclosed in any of the aforementioned embodiments of this application can be applied to processor 901, or implemented by processor 901.

[0090] The processor 901 may be an integrated circuit chip with signal processing capabilities. In implementation, each step of the above method can be completed by the integrated logic circuitry in the hardware of the processor 901 or by instructions in software form. The processor 901 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 902. Processor 901 reads the information in memory 902 and, in conjunction with its hardware, completes the steps of the above method.

[0091] The electronic device provided in this application embodiment and the medical image segmentation method provided in this application embodiment are based on the same inventive concept and have the same beneficial effects as the methods they employ, operate or implement.

[0092] This application also provides a computer-readable storage medium corresponding to the medical image segmentation method provided in the foregoing embodiments, which stores a computer program (i.e., a program product) thereon. When the computer program is run by a processor, it executes the medical image segmentation method provided in any of the foregoing embodiments.

[0093] It should be noted that computer-readable storage media may include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, optical disc or other optical and magnetic storage media, etc., which will not be elaborated here.

[0094] The computer-readable storage medium provided in this application embodiment and the medical image segmentation method provided in this application embodiment are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented by the application stored therein.

[0095] This application also provides a computer program product corresponding to the medical image segmentation method provided in the foregoing embodiments, including a computer program that is executed by a processor to implement the above-described medical image segmentation method.

[0096] The computer program product provided in this application embodiment is based on the same inventive concept as the medical image segmentation method provided in this application embodiment, and has the same beneficial effects as the method implemented by the computer program being executed by a processor.

[0097] It is understood that the description of the various embodiments above tends to emphasize the differences between the various embodiments, and the similarities or similarities can be referred to each other. For the sake of brevity, these will not be repeated here.

[0098] Those skilled in the art will understand that, in the methods described above in specific embodiments, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined based on its function and possible internal logic.

[0099] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and not to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. These modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application, and they should all be covered within the scope of the claims and specification of this application. In particular, as long as there is no structural conflict, the various technical features mentioned in the embodiments can be combined in any way. This application is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A medical image segmentation method, characterized in that, The method includes: Acquire the medical image to be segmented; The medical image is input into a dual-stream encoder for feature extraction. The dual-stream encoder includes a first encoder branch based on a pre-trained visual basic model and a second encoder branch based on a visual state space model. The first encoder branch encodes the medical image to obtain a first feature, and the second encoder branch encodes the medical image to obtain a second feature. The first feature is optimized by performing domain adaptive optimization on the first feature through a dynamic feature fusion optimizer. The dynamic feature fusion optimizer is used to bridge the inter-domain differences between natural images and medical images. The optimized first feature and the second feature are fused across architectures using a heterogeneous omnidirectional attention convergence module to obtain a fused feature. The fused features are input into the decoder for decoding to obtain the segmentation result of the medical image.

2. The method as described in claim 1, characterized in that, The first encoder branch based on the pre-trained visual basic model is an encoder branch based on the second-generation segmentation model, and the second encoder branch based on the visual state space model is an encoder branch based on the visual state space model. The encoder branch based on the second-generation segmentation model uses a frozen layered image encoder to encode the medical image, and adaptively adjusts the multi-scale features of the layered image encoder through a multilayer perceptron adapter. The encoder branch based on the visual state space model uses a visual state space module to encode the medical image in order to capture global semantic context features.

3. The method as described in claim 1, characterized in that, The step of performing domain-adaptive optimization on the first feature using a dynamic feature fusion optimizer to obtain the optimized first feature includes: Perform dimensional rearrangement on the first feature; The first feature after dimensional rearrangement is subjected to adaptive max pooling and adaptive average pooling respectively. The adaptive max pooling is used to preserve global salient semantic information, and the adaptive average pooling is used to suppress high-frequency noise interference. The features obtained by adaptive max pooling and adaptive average pooling are concatenated along the channel dimension to obtain the concatenated features. The optimized first feature is obtained based on the first feature and the splicing feature.

4. The method as described in claim 3, characterized in that, The optimized first feature obtained based on the first feature and the spliced ​​feature includes: The first feature is processed sequentially through the first convolutional layer and the first activation function, and then through the transposed convolutional layer and the second activation function to obtain the spatial feature; The splicing features and the spatial features are fused through residual connections to obtain residual fused features; The residual fusion features are subjected to dimensional rearrangement and layer normalization to obtain the optimized first feature.

5. The method as described in claim 1, characterized in that, The heterogeneous omnidirectional attention convergence module includes a dual-branch selective emphasis attention submodule and an omniscient context attention submodule; The dual-branch selective emphasis attention submodule is used to dynamically enhance the second feature using both channel-space dual-path enhancement to obtain the enhanced second feature. The omniscient context attention submodule is used to perform global semantic enhancement on the optimized first feature to obtain the enhanced first feature; The heterogeneous omnidirectional attention convergence module is also used to hierarchically integrate the enhanced second feature, the enhanced first feature, and the historical fusion semantics through a cross-attention fusion mechanism to obtain the fused feature.

6. The method as described in claim 5, characterized in that, The dynamic enhancement of the second feature using a dual-path channel-space approach includes: The second feature is subjected to spatial average pooling and spatial max pooling respectively. The spatial average pooling is used to capture long-distance dependencies across regions, and the spatial max pooling is used to preserve local salient patterns. The features obtained by spatial average pooling and spatial max pooling are respectively input into an adaptive self-attention mechanism with shared weights for processing to obtain dual-path attention features.

7. The method as described in claim 6, characterized in that, The processing steps of the adaptive self-attention mechanism with shared weights include: The pooled features are convolutionally processed and flattened using three independent 1×1 convolutional layers to generate a query matrix, a key matrix, and a value matrix. Calculate the similarity matrix between the query matrix and the key matrix, and normalize the similarity matrix using a normalization function to obtain the spatial relationship weight matrix; The spatial relationship weight matrix and the value matrix are multiplied by matrix multiplication to dynamically aggregate context information and obtain the dual-path attention features; The dual-path attention features are fused by weighted summation and normalized to generate a spatial attention map. The second feature is then enhanced by element-wise multiplication to obtain the enhanced second feature.

8. The method as described in claim 5, characterized in that, The global semantic enhancement of the optimized first feature includes: The optimized first feature is subjected to spatial max pooling and spatial average pooling along the channel dimension to extract salient features and capture global context information, respectively. The features obtained by spatial max pooling and spatial average pooling are concatenated along the channel dimension to obtain channel compressed features.

9. The method as described in claim 8, characterized in that, The global semantic enhancement of the optimized first feature further includes: Spatial average pooling is performed on the channel compressed features to construct a global feature representation. The global feature representation is then multiplied element-wise with the channel compressed features to achieve channel-level contrast enhancement, resulting in enhanced global features. The enhanced global features are convolved using a 7×7 convolution kernel, and then gated spatial attention is generated using an activation function. The gated spatial attention is processed by basic convolution and activation functions to generate spatial attention weights, and the optimized first feature is adaptively recalibrated by element-wise multiplication to obtain the enhanced first feature.

10. The method as described in claim 1, characterized in that, The decoder is a visual state space model decoder, which decodes the fused features through a progressive upsampling framework. The progressive upsampling framework achieves hierarchical feature fusion through successive transposed convolutions and generates a high-resolution segmentation mask as the segmentation result through a prediction head with shared parameters.

11. The method as described in claim 1, characterized in that, The medical image is a cardiac magnetic resonance image, and the segmentation result is used in a medical robot system configured to perform at least one of preoperative planning, intraoperative navigation, and postoperative screening based on the segmentation result.

12. A medical image segmentation device, characterized in that, The device includes: The acquisition module is used to acquire the medical image to be segmented; The input module is used to input the medical image into a dual-stream encoder for feature extraction. The dual-stream encoder includes a first encoder branch based on a pre-trained visual basic model and a second encoder branch based on a visual state space model. The first encoder branch encodes the medical image to obtain a first feature, and the second encoder branch encodes the medical image to obtain a second feature. An optimization module is used to perform domain adaptive optimization processing on the first feature through a dynamic feature fusion optimizer to obtain an optimized first feature. The dynamic feature fusion optimizer is used to bridge the inter-domain differences between natural images and medical images. The fusion module is used to perform cross-architecture feature fusion on the optimized first feature and the second feature through the heterogeneous omnidirectional attention convergence module to obtain the fused feature; The segmentation module is used to input the fused features into the decoder for decoding processing to obtain the segmentation result of the medical image.

13. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor runs the computer program to implement the medical image segmentation method as described in any one of claims 1-11.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer program is executed by a processor to implement the medical image segmentation method as described in any one of claims 1-11.

Images