Medical image multi-target segmentation method, system and device based on convolution and Mama framework and medium

Abstract

The invention discloses a medical image multi-target segmentation method, system and device based on convolution and Mama frameworks and a medium, and belongs to the technical field of medical image recognition, and the method comprises the steps: obtaining an original three-dimensional medical image, and carrying out the preprocessing of the original three-dimensional medical image, and generating a standardized input tensor; performing feature extraction to generate a coding feature sequence; carrying out long-range dependence modeling and applying a MonteCarlo attention mechanism according to the coding feature sequence, and generating a bottleneck feature and a jump connection feature of global context enhancement; splicing the bottleneck feature and the jump connection feature to generate a segmented feature map; mapping the segmentation feature map into a three-dimensional segmentation mask, and performing post-processing and spatial restoration to generate a three-dimensional multi-target segmentation result; according to the invention, through constructing a three-dimensional segmentation architecture fusing convolution local modeling and Mama state space long-range modeling, the defects of small target segmentation, cross-layer dependence modeling and reasoning efficiency in the prior art are overcome.

Description

Technical Field

[0001] This invention relates to the field of medical image recognition technology, specifically to a method, system, device, and medium for multi-target segmentation of medical images based on convolution and the Mamba framework. Background Technology

[0002] Currently, automatic multi-organ segmentation technology in medical images has been widely applied in clinical scenarios such as radiotherapy planning, lesion detection, and preoperative assessment. Existing related research is mainly based on convolutional neural networks (CNN) or Transformer structures (such as SwinUNETR), but in the field of multi-target segmentation of 3D medical images (such as multi-organ, tumor, and lesion identification), some problems still exist, including insufficient segmentation accuracy of small-volume or irregular targets, limited ability to model cross-layer spatial dependencies, large number of model parameters, and low inference efficiency.

[0003] Regarding the segmentation accuracy of small or irregular targets, traditional convolutional networks are limited by their local receptive fields. When faced with small structures such as the optic nerve and optic chiasm in head and neck CT scans, or small lesions in abdominal CT scans, their spatial dependencies are insufficiently captured, leading to blurred boundaries and missed detections. While existing methods can improve accuracy by increasing network depth or using pyramid feature fusion, these methods inevitably lead to a surge in computational costs. In terms of limited ability to model cross-layer spatial dependencies and a large number of model parameters, 3D medical images inherently possess strong inter-layer (cross-slice) spatial correlations. Standard CNNs struggle to establish effective long-range cross-layer dependencies, resulting in low information utilization in the Z-axis direction. Even though Transformer-based models such as TransBTS and SwinUNETR introduce global self-attention mechanisms to address this issue, the computational complexity of this approach increases quadratically with image resolution. When processing high-resolution 3D volumetric data, the high computational cost makes efficient inference extremely difficult. Regarding inference efficiency, existing medical image segmentation networks based on Transformers or deep CNNs typically contain tens of millions of parameters, resulting in high memory consumption and severe inference latency, which is detrimental to deployment and use in real-time clinical environments. Summary of the Invention

[0004] In view of the above-mentioned problems, the present invention is proposed.

[0005] Therefore, the technical problem solved by this invention is that existing multi-organ automatic segmentation technology is mainly based on convolutional neural networks or Transformer structures. In the field of multi-object segmentation, the use of convolutional neural networks or Transformer structures has always had problems such as insufficient segmentation accuracy for small or irregular objects, limited ability to model cross-layer spatial dependencies, large number of model parameters, and low inference efficiency.

[0006] To address the aforementioned technical problems, this invention provides the following technical solution: a multi-target segmentation method for medical images based on convolution and the Mamba framework, comprising the following steps: Acquire raw 3D medical images and preprocess them to generate standardized input tensors; Feature extraction is performed on the standardized input tensor to generate an encoded feature sequence; Three-dimensional Mamba long-range dependency modeling is performed on the deepest encoded features in the encoded feature sequence to generate bottleneck features with global context enhancement; A Monte Carlo attention mechanism is applied to the non-deepest encoded features in the encoded feature sequence to generate skip connection features; The bottleneck features are input into the decoder, upsampled, and concatenated with the skip connection features to generate a segmentation feature map. The segmentation feature map is mapped to a 3D segmentation mask, and post-processing and spatial reconstruction are performed to generate 3D multi-object segmentation results.

[0007] As a preferred embodiment of the medical image multi-target segmentation method based on convolution and the Mamba framework described in this invention, the steps of acquiring the original three-dimensional medical image and performing preprocessing to generate a standardized input tensor include: The original 3D medical images are obtained from the medical image archiving system, and the original 3D medical images are preprocessed to obtain normalized images. Extract the ROI volume block from the normalized image to obtain the cropped medical image; To add channel dimensions to crop medical images, a standardized input tensor is generated.

[0008] As a preferred embodiment of the medical image multi-target segmentation method based on convolution and the Mamba framework described in this invention, the step of extracting features from the standardized input tensor to generate an encoded feature sequence includes: Initialize the encoding layer parameters and construct SE-ResBlock convolutional blocks level by level; The SE-ResBlock convolutional block is used to extract features from the normalized input tensor to generate an encoded feature sequence.

[0009] As a preferred embodiment of the medical image multi-target segmentation method based on convolution and Mamba framework described in this invention, the step of performing three-dimensional Mamba long-range dependency modeling on the deepest encoded features in the encoded feature sequence to generate bottleneck features with global context enhancement includes: The deepest encoded feature in the encoded feature sequence is input into the dilated convolution module to perform dilated convolution and generate a single feature map. A single feature map is input into a gated convolutional module for integration and purification, generating a purified feature map. The purified feature map is unfolded into independent sequences along three orthogonal anatomical directions, and global long-range dependency modeling is performed to generate enhanced feature maps. The enhanced feature map and the purified feature map are subjected to a first residual connection to obtain the fused feature map; The fused feature map is input into a multilayer perceptron for channel feature transformation and nonlinear mapping to output a feedforward feature map. The feedforward feature map and the fused feature map are second residually concatenated to generate bottleneck features with global context enhancement.

[0010] The beneficial effects of this preferred technical solution are as follows: by adopting the three-dimensional Mamba module (TD-Mamba), the deepest encoded features are expanded along the three anatomically orthogonal directions of axis, coronal and sagittal. The selective state space model is used to model global long-range dependencies with linear computational complexity, capturing cross-slice spatial correlations, thus overcoming the modeling bottlenecks caused by the limited receptive field of traditional CNNs and the quadratic complexity of Transformers.

[0011] As a preferred embodiment of the medical image multi-target segmentation method based on convolution and the Mamba framework described in this invention, the step of applying a Monte Carlo attention mechanism to the non-deepest encoded features in the encoded feature sequence to generate skip connection features includes: Channel adjustment is performed on the non-deepest encoded features in the encoded feature sequence to generate channel-adjusted features; Feature extraction is performed on the channel adjustment features to generate local enhancement features; Local enhancement features are randomly sampled using Monte Carlo methods and then weighted and fused to generate an attention map; Based on the attention map, attention-weighted local enhancement features are applied and channel dimension fine-tuning is performed to generate attention-enhanced features; Attention-enhancing features and channel-adjusting features are residually connected to generate skip connection features.

[0012] The beneficial effects of this preferred technical solution are as follows: by introducing the Monte Carlo attention mechanism (a method that integrates Monte Carlo sampling into attention weight calculation) into the skip connection path, multi-scale random sampling and attention reweighting are performed on shallow high-resolution features, which enhances the feature response of small-volume or weak-contrast targets such as optic nerves and optic chiasms, suppresses background noise, and thus improves boundary continuity and small target recall.

[0013] As a preferred embodiment of the medical image multi-target segmentation method based on convolution and the Mamba framework described in this invention, the steps of inputting bottleneck features into the decoder, performing upsampling, and concatenating them with skip connection features to generate a segmentation feature map include: Initialize the decoder layer parameters and perform trilinear interpolation upsampling on the bottleneck features to generate upsampled bottleneck features; The upsampled bottleneck features are depthwise convolved and then activated by ReLU to generate convolutional bottleneck features. The bottleneck features of the convolution are adjusted by pointwise convolution to adjust the number of channels, and the upsampled output features of the current level are output. The upsampled output features of the current level are concatenated with the skip connection features and processed by ResDconv to generate a segmentation feature map.

[0014] The advantages of this preferred technical solution are as follows: the overall network adopts the design of convolutional blocks and efficient upsampling convolutional blocks, avoiding the use of heavy attention mechanisms or deep stacked structures, reducing the number of parameters and memory usage while ensuring segmentation accuracy, improving inference speed, and meeting the needs of real-time clinical deployment.

[0015] As a preferred embodiment of the medical image multi-object segmentation method based on convolution and the Mamba framework described in this invention, the steps of mapping the segmentation feature map to a 3D segmentation mask and performing post-processing and spatial reconstruction to generate 3D multi-object segmentation results include: The segmentation feature map is input into a 1×1×1 convolutional layer to map the number of channels to a multi-channel feature map of the number of medical target categories. Activation functions are applied to the multi-channel feature maps and voxel-level class probability distributions are calculated; Generate an initial segmentation mask based on the category probability distribution; The initial segmentation mask is corrected by three-dimensional morphological closing operation and connected component analysis to generate a corrected segmentation mask; The modified segmentation mask is sampled into the original CT spatial coordinate system using the affine transformation matrix of the original 3D medical image, resulting in a 3D multi-object segmentation result.

[0016] This invention provides a multi-target segmentation system for medical images based on convolution and the Mamba framework.

[0017] To address the aforementioned technical problems, the present invention further provides the following technical solution: a multi-object segmentation system for medical images based on convolution and the Mamba framework, comprising: Data acquisition module: Acquires raw 3D medical images and performs preprocessing to generate standardized input tensors; Feature extraction module: Extracts features from the standardized input tensor and generates a sequence of encoded features; Long-range dependency modeling module: Performs 3D Mamba long-range dependency modeling on the deepest encoded features in the encoded feature sequence to generate bottleneck features with global context enhancement; Attention module: Apply Monte Carlo attention mechanism to non-deepest encoded features in the encoded feature sequence to generate skip connection features; Feature decoding module: Inputs bottleneck features into the decoder, performs upsampling, and concatenates them with skip connection features to generate a segmentation feature map; Spatial Restoration Module: Maps the segmentation feature map to a 3D segmentation mask, and performs post-processing and spatial restoration to generate 3D multi-object segmentation results.

[0018] The present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, characterized in that the processor executes the computer program to implement the steps of the aforementioned medical image multi-target segmentation method based on convolution and the Mamba framework.

[0019] The present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that, when the computer program is executed by a processor, it implements the steps of the aforementioned medical image multi-target segmentation method based on convolution and the Mamba framework.

[0020] The beneficial effects of this invention are as follows: By introducing a Monte Carlo attention mechanism (a method that integrates Monte Carlo sampling into attention weight calculation) into the skip connection path, this invention performs multi-scale random sampling and attention reweighting on shallow high-resolution features, enhancing the feature response of small-volume or weak-contrast targets such as the optic nerve and optic chiasm, suppressing background noise, and thus improving boundary continuity and small target recall. By employing a three-dimensional Mamba module (TD-Mamba), the deepest layer encoded features are expanded along three anatomically orthogonal directions: axial, coronal, and sagittal. A selective state-space model is used to model global long-range dependencies with linear computational complexity, capturing cross-slice spatial correlations and overcoming the modeling bottlenecks caused by the limited receptive field of traditional CNNs and the quadratic complexity of Transformers. The overall network adopts a design of convolutional blocks and efficient upsampling convolutional blocks, avoiding the use of heavy attention mechanisms or deep stacked structures. While ensuring segmentation accuracy, it reduces the number of parameters and memory usage, improves inference speed, and meets the needs of real-time clinical deployment. This invention solves the shortcomings of existing technologies in small target segmentation, cross-layer dependency modeling, and inference efficiency by constructing a three-dimensional segmentation architecture that integrates convolutional local modeling and Mamba state-space long-range modeling. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart illustrating the overall process of a multi-target segmentation method for medical images based on convolution and the Mamba framework, as provided in one embodiment of the present invention. Detailed Implementation

[0023] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0024] Example 1, referring to Figure 1 This is the first embodiment of the present invention, which provides a multi-object segmentation method for medical images based on convolution and the Mamba framework, including: S100: Acquire raw 3D medical images and preprocess them to generate standardized input tensors; S200: Extract features from the standardized input tensor to generate a coded feature sequence; S300: Performs 3D Mamba long-range dependency modeling on the deepest encoded features in the encoded feature sequence to generate bottleneck features with global context enhancement; S400: Apply Monte Carlo attention mechanism to non-deepest encoded features in the encoded feature sequence to generate skip connection features; S500: Input the bottleneck features into the decoder, perform upsampling, and concatenate them with the skip connection features to generate a segmentation feature map; S600: Maps the segmentation feature map to a 3D segmentation mask, and performs post-processing and spatial reconstruction to generate 3D multi-object segmentation results.

[0025] It should be noted that existing multi-organ automatic segmentation technologies are mainly based on convolutional neural networks or Transformer structures. However, the use of convolutional neural networks or Transformer structures in the field of multi-object segmentation has always suffered from problems such as insufficient segmentation accuracy for small or irregular objects, limited ability to model cross-layer spatial dependencies, large number of model parameters, and low inference efficiency. Therefore, it is very important to propose a three-dimensional segmentation architecture that integrates convolutional local modeling and Mamba state space long-range modeling.

[0026] Therefore, to address the aforementioned problems in multi-target segmentation of medical images, a multi-target segmentation method based on convolution and the Mamba framework is constructed through steps S100-S600. By introducing a Monte Carlo attention mechanism (a method that integrates Monte Carlo sampling into attention weight calculation) into the skip connection path, multi-scale random sampling and attention reweighting are performed on shallow high-resolution features to enhance the feature response of small-volume or weak-contrast targets such as the optic nerve and optic chiasm, suppressing background noise, thereby improving boundary continuity and small target recall. By employing a three-dimensional Mamba module (TD-Mamba), the deepest encoded features are expanded along the three anatomically orthogonal directions of axis, coronal, and sagittal. A selective state-space model is used to model global long-range dependencies with linear computational complexity, capturing cross-slice spatial correlations and overcoming the modeling bottlenecks caused by the limited receptive field of traditional CNNs and the quadratic complexity of Transformers. The overall network adopts a design of convolutional blocks and efficient upsampling convolutional blocks, avoiding the use of heavy attention mechanisms or deep stacked structures. While ensuring segmentation accuracy, the number of parameters and memory usage are reduced, and inference speed is improved to meet the needs of real-time clinical deployment.

[0027] Example 2, refer to Figure 1 This is the second embodiment of the present invention, which provides a multi-target segmentation method for medical images based on convolution and the Mamba framework.

[0028] In this embodiment of the application, the step S100 of acquiring the original three-dimensional medical image and performing preprocessing to generate a standardized input tensor includes the following steps A1-A3: A1: Obtain the original 3D medical images from the medical image archiving system, preprocess the original 3D medical images to obtain normalized images; In this embodiment, head and neck enhanced CT images are read from the hospital's medical image archive and communication system as the original three-dimensional medical images. The HU value of the original three-dimensional medical images is truncated to the range of [-1000, 1000] according to clinical standards. The original three-dimensional medical images after HU value truncation are then standardized to generate normalized images. A2: Extract the ROI volume block from the normalized image to obtain the cropped medical image; In this embodiment, the multi-target region, including targets such as the eyeball, optic nerve, optic chiasm, parotid gland, and tumor, annotated by radiologists, is used as multi-target annotation data. The smallest cuboid region that can completely contain all target voxels in the multi-target annotation data is determined, and this smallest cuboid region is used as the ROI (Region of Interest). The ROI volume block is extracted from the normalized image according to the ROI range to obtain the cropped medical image. By extracting the ROI volume block, background interference is reduced and video memory usage is decreased. In one alternative implementation, the ROI volume block extracted from the normalized image can also be achieved through key anatomical point construction. A 3D keypoint detection network is invoked, and the output of the 3D keypoint detection network is the coordinates of several fixed anatomical points in the head and neck, such as the center of the eyes, the angle of the mandible, and the center of the C2 vertebral body of the cervical spine. These points can be pre-trained using synthetic data or weak supervision, without relying on case-by-case annotation, to calculate the minimum bounding box of all keypoint coordinates, and dynamically adjust the boundary based on clinical prior knowledge. The normalized image is then cropped using this minimum bounding box to obtain the ROI volume block.

[0029] In another alternative implementation, the extraction of ROI volume blocks from the normalized image can also be achieved by generating a response heatmap using a sliding window classifier. A 64×64×64 window is slid across the normalized image with a fixed step size. Each window is input to a pre-trained binary classifier that determines whether the target structure is present. The output probabilities of the binary classifiers form a three-dimensional response heatmap. The heatmap is thresholded and morphological closing operations are performed to obtain the foreground region. The minimum bounding box of the foreground region is calculated and extended by 15mm as the ROI clipping range to obtain the ROI volume block.

[0030] A3: Generates standardized input tensors by adding channel dimensions to crop medical images.

[0031] In this embodiment of the application, the cropped medical image is a three-dimensional tensor. Add a channel dimension to the foreground dimension of the cropped medical image to form a four-dimensional tensor. Generate a standardized input tensor.

[0032] In this embodiment of the application, step S200, which involves feature extraction from the standardized input tensor to generate an encoded feature sequence, includes the following steps B1-B2: B1: Initialize the encoding layer parameters and construct SE-ResBlock convolutional blocks level by level; In this embodiment, the encoding layer parameters are initialized, and the encoder is set to have 4 levels of downsampling. The number of output channels in each level is 32, 64, 128, and 256 respectively. The downsampling method uses 3D convolution with a stride of 2 to replace the traditional pooling operation, thereby preserving spatial information. Each level constructs an SE-ResBlock convolutional block. The structure of the SE-ResBlock convolutional block includes a 3D convolutional layer, batch normalization, SiLU activation function, and SE (Squeeze-and-Excitation) module. The 3D convolutional layer extracts local features such as edges, textures, and structures in three-dimensional space. Batch normalization standardizes the features, making training more stable and convergent faster. The SiLU activation function introduces non-linearity, enabling the learning of more complex patterns while preserving negative information. The SE module automatically determines which channels are more important, giving bonuses to useful channels and deweighting irrelevant channels, focusing on key targets, dynamically adjusting the weights of each channel, and enhancing the semantic response related to the target organ. B2: Use SE-ResBlock convolutional blocks to extract features from the standardized input tensor to generate encoded feature sequences.

[0033] In this embodiment, the normalized input tensor output in step A3 is sequentially input into the SE-ResBlock convolutional block constructed in step B1 for progressive feature extraction. At each level, while extracting local three-dimensional spatial features, spatial downsampling is achieved through 3D convolution with a stride of 2, gradually reducing the spatial resolution of the feature map and improving the semantic abstraction level. After multi-level encoding, a set of multi-scale feature maps with progressively halved resolution and progressively increased number of channels is output, generating an encoded feature sequence. In the encoded feature sequence, the deepest feature map (lowest resolution) preserves high-level semantic information and will be used for subsequent 3D Mamba long-range dependency modeling, while the remaining shallow feature maps (higher resolution) preserve spatial details and will be used for subsequent Monte Carlo attention skip connection enhancement, thereby providing contextual support with both semantic and detailed information for the decoding stage.

[0034] In this embodiment of the application, step S300, which involves performing three-dimensional Mamba long-range dependency modeling on the deepest encoded features in the encoded feature sequence to generate bottleneck features with global context enhancement, includes the following steps C1-C6: C1: Input the deepest encoded feature in the encoded feature sequence into the dilated convolution module to perform dilated convolution and generate a single feature map; In this embodiment of the application, the deepest encoded features in the encoded feature sequence obtained in step B2 are input into the multi-scale dilated convolution module. The dilated convolution module uses multiple parallel 3D dilated convolutions with dilation rates of 1, 2 and 3, respectively, to expand the receptive field without reducing the spatial resolution and capture multi-scale contextual information. The outputs of each branch are concatenated by channels and then fused into a single feature map by 1×1×1 convolution. C2: Input the single feature map into the gated convolution module for integration and purification to generate a purified feature map; In this embodiment of the application, the single feature map obtained in step C1 is input into the gated spatial convolution module. The gated spatial convolution module integrates spatial dimensions and suppresses redundancy through its gating mechanism, and outputs a more discriminative purified feature map for subsequent sequence modeling. The gated spatial convolution module structure includes two parallel paths. The main path performs a 3×3×3 standard convolution on the input features, followed by batch normalization and SiLU activation to extract local spatial details. The gated path performs a 1×1×1 convolution on the same input features and then Sigmoid activation to generate a spatial gating map, which is used to dynamically suppress redundant region responses. Multiplying the local spatial details element-wise with the spatial gating map yields the gated and modulated features, which are the purified feature maps. C3: Expand the purified feature map into independent sequences along three orthogonal anatomical directions, perform global long-range dependency modeling, and generate enhanced feature maps; In this embodiment, the purified feature map is unfolded into a one-dimensional sequence along the three anatomically orthogonal directions: axial, coronal, and sagittal. The sequence in each direction is normalized by layers, linearly projected, one-dimensional convolution, and SiLU activated before being input into a selective state space model. The selective state space model performs linear projection and local feature extraction on the input sequence and generates key parameters of the state space based on the input content, such as the discretization step size and the input-output projection matrix. This allows the modeling process to focus on the importance of different positions and updates the internal hidden state point by point through a recursive method, thereby achieving global information transmission of the entire sequence in linear time complexity. The modeling result is then gated and fused with the original input to output enhanced sequence features. The enhanced sequence features output from the three directions are weighted and fused to obtain the enhanced feature map. C4: Perform a first residual connection between the enhanced feature map and the purified feature map to obtain the fused feature map; In this embodiment of the application, the enhanced feature map and the purified feature map are added element by element to form the first residual connection result, and the fused feature map is obtained. C5: Input the fused feature map into the multilayer perceptron for channel feature transformation and nonlinear mapping, and output a feedforward feature map; In this embodiment, the fused feature map is layer normalized and input into a two-layer multilayer perceptron for channel feature transformation and nonlinear mapping to achieve nonlinear interaction and feature expression enhancement between channels, and output a feedforward feature map. It should be noted that in the two-layer multilayer perceptron, the first layer expands the channel dimension of the input features to four times the original dimension and introduces nonlinearity using the SiLU activation function; the second layer compresses the channel dimension back to its original size, does not use an activation function, and restores the channel structure consistent with the input. This design enables sufficient nonlinear interaction and feature recombination between channels, enhancing the expressive power of the model. In an alternative implementation, the fused feature map can also be output as a feedforward feature map via attention-guided feedforward. The fused feature map is layer normalized, and channel weights are generated through a small squeeze-and-excitation module. The process of generating channel weights includes: first, global average pooling to obtain channel statistics, and then generating weight vectors through two fully connected layers and a sigmoid function. The weight vectors are used to recalibrate the original features to obtain attention-enhanced intermediate features. The intermediate features are input into a single-layer fully connected network to output a feedforward feature map.

[0035] In another alternative implementation, the fused feature map can also be output as a feedforward feature map via Fourier domain feedforward. A fast Fourier transform is performed on the fused feature map along the channel dimension to convert the fused feature map into a frequency domain representation. In the frequency domain, a learnable complex weight matrix is ​​used to weight and modulate each frequency component. Then, an inverse fast Fourier transform is performed to restore the spatial domain features, which are then activated by GELU and output as a feedforward feature map.

[0036] C6: The feedforward feature map and the fused feature map are second residually concatenated to generate a bottleneck feature with global context enhancement.

[0037] In this embodiment, the feedforward feature map and the fused feature map are added element-wise to achieve a second residual connection, and the bottleneck feature with global context enhancement is output. The dimension of the bottleneck feature is consistent with the deepest encoded feature in the encoded feature sequence, but it contains cross-slice long-range dependency and channel nonlinear enhancement information.

[0038] In this embodiment of the application, step S400, which applies a Monte Carlo attention mechanism to the non-deepest encoded features in the encoded feature sequence to generate skip connection features, includes the following steps D1-D5: D1: Perform channel adjustment on the non-deepest encoded features in the encoded feature sequence to generate channel-adjusted features; In this embodiment of the application, the features of each level other than the deepest layer coding in the coding feature sequence obtained in step S200 are non-deepest layer coding features. Each level of non-deepest layer coding feature is input into a 1×1×1 convolutional layer, and the number of channels is adjusted to the dimension aligned with the corresponding level of the decoder, and the channel-adjusted features are output. D2: Extract features from the channel adjustment features to generate local enhancement features; In this embodiment, the channel adjustment features are input into a 3×3×3 standard convolutional layer to extract local spatial features, generate local enhancement features, and enhance the ability to express local spatial context. D3: Perform Monte Carlo random sampling and weighted fusion on the local enhancement features to generate an attention map; In this embodiment, Monte Carlo random sampling is performed on neighborhood windows at multiple preset local scales, such as 1×1×1, 2×2×2, and 3×3×3, for local enhancement features. For example, 4 voxels are randomly selected at each scale to obtain multi-scale local neighborhood feature values. The multi-scale local neighborhood feature values ​​are concatenated in the channel dimension, and then dimensionality is reduced by 1×1×1 convolution before being activated by the Sigmoid function to generate an attention map. D4: Perform attention weighting on the local enhancement features based on the attention map and fine-tune the channel dimension to generate attention-enhanced features; In this embodiment, the attention map and the local enhancement features are multiplied element by element to obtain the attention-weighted features, and then the attention-weighted features are output as attention enhancement features through a 1×1×1 convolution fine-tuning channel structure. D5: Generate skip connection features by performing residual connections between attention enhancement features and channel adjustment features.

[0039] In this embodiment of the application, the attention enhancement features are added element-wise to the channel adjustment features obtained in step D1 to achieve residual connections, and the output is a skip connection feature; It should be noted that applying the Monte Carlo attention mechanism to the non-deepest encoded features in the encoded feature sequence through steps D1-D5 can suppress the response of strong interference areas such as background muscles and bones in head and neck CT, while improving the feature signal-to-noise ratio of weak structures such as the optic nerve and optic chiasm. Thus, without increasing the network depth, it can improve the segmentation integrity and boundary continuity of small-volume targets.

[0040] In this embodiment of the application, step S500, which inputs the bottleneck features into the decoder, performs upsampling, and concatenates them with the skip connection features to generate a segmentation feature map, includes the following steps E1-E4: E1: Initialize decoder level parameters, perform trilinear interpolation upsampling on bottleneck features to generate upsampled bottleneck features; In this embodiment, the decoder layer parameters are initialized, the number of decoder layers is aligned with the number of shallow encoder layers, the upsampling factor for each layer is 2, the bottleneck features obtained in step S300 are used as the input of the first-level upsampled convolutional block of the decoder, and the spatial resolution is doubled by using trilinear interpolation to obtain the upsampled bottleneck features. E2: Perform depthwise convolution on the upsampled bottleneck features and pass them through the ReLU activation function to generate convolutional bottleneck features; In this embodiment, the upsampled bottleneck features are input into a 3×3×3 depth convolution to extract local spatial details. Then, after batch normalization and ReLU activation function processing, the convolutional bottleneck features are output; this reduces the number of parameters while efficiently extracting local spatial details. E3: Adjust the number of channels of the convolution bottleneck features through pointwise convolution and output the upsampled output features of the current layer; In this embodiment, the convolutional bottleneck features are input into a 1×1×1 pointwise convolution to fuse the channel information and adjust it to the target dimension, thereby generating the upsampled output features of the current level. E4: Concatenate the upsampled output features of the current layer with the skip connection features and process them through ResDconv to generate a segmentation feature map.

[0041] In this embodiment, the upsampled output features of the current level are concatenated with the skip connection features obtained in step S400 in the channel dimension to obtain a fusion tensor. The fusion tensor is then input into the ResDconv module containing residual convolutional structures for feature refinement, and the current level decoding features are output. The current level decoding features are used as the input to the next level of the decoder, and steps E1-E4 are repeated in the next level. Finally, a high-resolution segmentation feature map is output at the last level of the decoder. It should be noted that the ResDconv module is a deep convolutional module with residual structures. The ResDconv module contains a 3×3×3 deep convolutional module, batch normalization and ReLU activation function, and finally performs element-wise addition with the input features to achieve residual connections. While preserving the original information, it performs local adjustments on the stitched high-dimensional features to improve the reconstruction quality of small target boundaries, and the computational cost is lower than that of standard 3D convolution.

[0042] In one alternative implementation, concatenating the upsampled output features of the current level with the skip connection features can also be achieved using cross-attention feature alignment and fusion. The upsampled output features are treated as queries, and the skip connection features are treated as keys and values. An attention map is generated by calculating the similarity between the query and the key, and then the values ​​are weighted and summed to obtain the aligned skip features. Finally, the aligned skip features are added to the upsampled output features and concatenated.

[0043] In another alternative implementation, the upsampled output features of the current level can be concatenated with the skip connection features using adaptive convolution fusion. The upsampled output features and skip connection features are input into a dynamic convolution module. The dynamic convolution module generates local convolution kernel parameters based on the input content, and then performs spatial adaptive convolution to achieve feature concatenation.

[0044] In this embodiment of the application, step S600, which maps the segmentation feature map to a 3D segmentation mask and performs post-processing and spatial reconstruction to generate a 3D multi-object segmentation result, includes the following steps F1-F5: F1: Input the segmentation feature map into a 1×1×1 convolutional layer to map the number of channels to a multi-channel feature map of the number of medical target categories; In this embodiment of the application, the segmentation feature map obtained in step S500 is input into a 1×1×1 convolutional layer. The number of output channels of the 1×1×1 convolutional layer is equal to the total number of medical target categories to be segmented, such as eyeball, optic nerve, optic chiasm, parotid gland, and tumor, a total of 5 categories; after passing through the 1×1×1 convolutional layer, a multi-channel feature map is output. F2: Apply activation functions to the multi-channel feature maps and calculate the voxel-level class probability distribution; In this embodiment, the Softmax activation function is applied to the multi-channel feature map to calculate the normalized probability of each voxel belonging to each category, thereby obtaining the voxel-level category probability distribution, wherein the sum of the probabilities of all voxels is 1. F3: Generate an initial segmentation mask based on the category probability distribution; In this embodiment, category determination is performed based on the category probability distribution. For conventional large organs such as the eyeball and parotid gland, a maximum probability indexing operation is performed, and the category index with the highest probability value is used as the final label of the voxel. For small-volume or weakly boundaryed targets such as the optic nerve and small tumors, a probability threshold is set. When the voxel-level category probability in the voxel-level category probability distribution is greater than the probability threshold, it is determined to belong to the category corresponding to the voxel-level category probability. When the voxel-level category probability in the voxel-level category probability distribution is less than or equal to the probability threshold, it is determined to be background, thus suppressing false positive predictions caused by noise. The category probability distribution after category determination is output as the initial segmentation mask. F4: The initial segmentation mask is corrected by three-dimensional morphological closing operation and connected component analysis to generate a corrected segmentation mask; In this embodiment of the application, three-dimensional morphological closing operation and connected component analysis are performed on each type of target mask of the initial segmentation mask. The three-dimensional morphological closing operation can fill internal holes, such as the low-density area in the center of a tumor, and the connected component analysis can remove discrete noise areas with a voxel number less than a preset voxel threshold. F5: The modified segmentation mask is sampled into the original CT spatial coordinate system through the affine transformation matrix of the original 3D medical image, resulting in 3D multi-target segmentation.

[0045] In this embodiment of the application, the original three-dimensional medical image is usually in DICOM or NIfTI format. If the original three-dimensional medical image is in DICOM format, spatial information is extracted from the DICOM format file. The spatial information includes the position of the upper left corner of the slice in the patient coordinate system, the unit vectors in the row and column directions of the slice, the in-plane pixel spacing, and the slice thickness or layer spacing. The spatial information is used to construct a 4×4 affine transformation matrix according to the DICOM standard. If the original 3D medical image is in NIfTI format, the 4×4 affine matrix can be read from the sform_code field or qform_code field of the NIfTI format file to obtain the affine transformation matrix; Based on the affine transformation matrix, the modified segmentation mask is inversely transformed to the original voxel mesh through trilinear interpolation and nearest neighbor resampling, generating a three-dimensional multi-object segmentation result.

[0046] In one alternative implementation, the affine transformation matrix can also be obtained by reconstructing a set of anatomical landmarks that are manually or automatically labeled. A set of common anatomical landmarks, such as at least four non-coplanar points, such as the center of the eyeball, the angle of the mandible, and the center of the cervical vertebrae, are simultaneously labeled in the original medical image and the standardized image. These points can be manually labeled by radiologists or automatically extracted by a pre-trained key point detection model. The optimal affine transformation matrix is ​​solved by the least squares method using two sets of corresponding points, the original spatial coordinates and the standardized spatial coordinates.

[0047] In another alternative implementation, the affine transformation matrix can also be constructed using the physical size and orientation of voxels. The user is prompted to input or query information such as the in-plane pixel spacing, slice thickness and inter-slice spacing, scanning direction and patient orientation of the original image from the hospital information system. A 4×4 affine matrix is ​​constructed according to the DICOM standard coordinate system definition, which is the affine transformation matrix. The rotation part is determined by the scanning plane orientation, the scaling part is determined by the pixels or inter-slice spacing, and the translation part is estimated by the physical starting position of the first slice.

[0048] Example 3, referring to Figure 1 This is the third embodiment of the present invention, which provides a multi-object segmentation system for medical images based on convolution and the Mamba framework, comprising: Data acquisition module: Acquires raw 3D medical images and performs preprocessing to generate standardized input tensors; Feature extraction module: Extracts features from the standardized input tensor and generates a sequence of encoded features; Long-range dependency modeling module: Performs 3D Mamba long-range dependency modeling on the deepest encoded features in the encoded feature sequence to generate bottleneck features with global context enhancement; Attention module: Apply Monte Carlo attention mechanism to non-deepest encoded features in the encoded feature sequence to generate skip connection features; Feature decoding module: Inputs bottleneck features into the decoder, performs upsampling, and concatenates them with skip connection features to generate a segmentation feature map; Spatial Restoration Module: Maps the segmentation feature map to a 3D segmentation mask, and performs post-processing and spatial restoration to generate 3D multi-object segmentation results.

[0049] Example 4, the fourth embodiment of the present invention, differs from the previous three embodiments in that: if the function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0050] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-including system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0051] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, because the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0052] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0053] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A multi-object segmentation method for medical images based on convolution and the Mamba framework, characterized in that, include: Acquire raw 3D medical images and preprocess them to generate standardized input tensors; Feature extraction is performed on the standardized input tensor to generate an encoded feature sequence; Three-dimensional Mamba long-range dependency modeling is performed on the deepest encoded features in the encoded feature sequence to generate bottleneck features with global context enhancement; A Monte Carlo attention mechanism is applied to the non-deepest encoded features in the encoded feature sequence to generate skip connection features; The bottleneck features are input into the decoder, upsampled, and concatenated with the skip connection features to generate a segmentation feature map. The segmentation feature map is mapped to a 3D segmentation mask, and post-processing and spatial reconstruction are performed to generate 3D multi-object segmentation results.

2. The medical image multi-object segmentation method based on convolution and the Mamba framework as described in claim 1, characterized in that, The steps for acquiring and preprocessing raw 3D medical images to generate standardized input tensors include: The original 3D medical images are obtained from the medical image archiving system, and the original 3D medical images are preprocessed to obtain normalized images. Extract the ROI volume block from the normalized image to obtain the cropped medical image; To add channel dimensions to crop medical images, a standardized input tensor is generated.

3. The medical image multi-target segmentation method based on convolution and the Mamba framework as described in claim 2, characterized in that, The steps for extracting features from a standardized input tensor to generate an encoded feature sequence include: Initialize the encoding layer parameters and construct SE-ResBlock convolutional blocks level by level; The SE-ResBlock convolutional block is used to extract features from the normalized input tensor to generate an encoded feature sequence.

4. The medical image multi-object segmentation method based on convolution and the Mamba framework as described in claim 3, characterized in that, The steps for performing 3D Mamba long-range dependency modeling on the deepest encoded features in the encoded feature sequence to generate bottleneck features with global context enhancement include: The deepest encoded feature in the encoded feature sequence is input into the dilated convolution module to perform dilated convolution and generate a single feature map. A single feature map is input into a gated convolutional module for integration and purification, generating a purified feature map. The purified feature map is unfolded into independent sequences along three orthogonal anatomical directions, and global long-range dependency modeling is performed to generate enhanced feature maps. The enhanced feature map and the purified feature map are subjected to a first residual connection to obtain the fused feature map; The fused feature map is input into a multilayer perceptron for channel feature transformation and nonlinear mapping to output a feedforward feature map. The feedforward feature map and the fused feature map are second residually concatenated to generate bottleneck features with global context enhancement.

5. The medical image multi-object segmentation method based on convolution and the Mamba framework as described in claim 4, characterized in that, The steps for generating skip connection features by applying a Monte Carlo attention mechanism to the non-deepest encoded features in the encoded feature sequence include: Channel adjustment is performed on the non-deepest encoded features in the encoded feature sequence to generate channel-adjusted features; Feature extraction is performed on the channel adjustment features to generate local enhancement features; Local enhancement features are randomly sampled using Monte Carlo methods and then weighted and fused to generate an attention map; Based on the attention map, attention-weighted local enhancement features are applied and channel dimension fine-tuning is performed to generate attention-enhanced features; Attention-enhancing features and channel-adjusting features are residually connected to generate skip connection features.

6. The medical image multi-object segmentation method based on convolution and the Mamba framework as described in claim 5, characterized in that, The steps of inputting bottleneck features into the decoder, performing upsampling, and concatenating them with skip connection features to generate a segmentation feature map include: Initialize the decoder layer parameters and perform trilinear interpolation upsampling on the bottleneck features to generate upsampled bottleneck features; The upsampled bottleneck features are depthwise convolved and then activated by ReLU to generate convolutional bottleneck features. The bottleneck features of the convolution are adjusted by pointwise convolution to adjust the number of channels, and the upsampled output features of the current level are output. The upsampled output features of the current level are concatenated with the skip connection features and processed by ResDconv to generate a segmentation feature map.

7. The medical image multi-object segmentation method based on convolution and the Mamba framework as described in claim 6, characterized in that, The steps for mapping segmentation feature maps to 3D segmentation masks and performing post-processing and spatial reconstruction to generate 3D multi-object segmentation results include: The segmentation feature map is input into a 1×1×1 convolutional layer to map the number of channels to a multi-channel feature map of the number of medical target categories. Activation functions are applied to the multi-channel feature maps and voxel-level class probability distributions are calculated; Generate an initial segmentation mask based on the category probability distribution; The initial segmentation mask is corrected by three-dimensional morphological closing operation and connected component analysis to generate a corrected segmentation mask; The modified segmentation mask is sampled into the original CT spatial coordinate system using the affine transformation matrix of the original 3D medical image, resulting in a 3D multi-object segmentation result.

8. A multi-object segmentation system for medical images based on convolution and the Mamba framework, employing the multi-object segmentation method for medical images based on convolution and the Mamba framework as described in any one of claims 1 to 7, characterized in that, include: Data acquisition module: Acquires raw 3D medical images and performs preprocessing to generate standardized input tensors; Feature extraction module: Extracts features from the standardized input tensor and generates a sequence of encoded features; Long-range dependency modeling module: Performs 3D Mamba long-range dependency modeling on the deepest encoded features in the encoded feature sequence to generate bottleneck features with global context enhancement; Attention module: Apply Monte Carlo attention mechanism to non-deepest encoded features in the encoded feature sequence to generate skip connection features; Feature decoding module: Inputs bottleneck features into the decoder, performs upsampling, and concatenates them with skip connection features to generate a segmentation feature map; Spatial Restoration Module: Maps the segmentation feature map to a 3D segmentation mask, and performs post-processing and spatial restoration to generate 3D multi-object segmentation results.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the medical image multi-target segmentation method based on convolution and the Mamba framework as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the medical image multi-target segmentation method based on convolution and the Mamba framework as described in any one of claims 1 to 7.

Images