A lightweight left ventricular echocardiogram segmentation method and system based on Mamba

Abstract

A kind of Mamba-based lightweight left ventricular echocardiogram segmentation method and system, it is related to medical image analysis technical field, the present application is to solve the problems of insufficient segmentation accuracy, high computational complexity and large parameter quantity of existing medical image segmentation method proposed.The technical points are: to solve the challenge brought by the limitation of computing resources in real medical environment, a U-shaped lightweight segmentation method is constructed based on state space model Mamba, the original image is encoded using image feature encoding module, and residual visual Mamba module is constructed to scan and aggregate multi-angle features in multiple directions of image, adaptive feature enhancement module is also constructed to process echocardiogram data with size and shape differences between individuals, feature aggregation bridge module is used to integrate multi-party features and transfer high-dimensional layer features to low-dimensional layer layer by layer, mask generation module is constructed to output the segmentation result map of the current layer, and segmentation map output module is used to output the segmentation result processed by the whole model.The present application belongs to the field of medical image analysis.

Description

Technical Field

[0001] This invention belongs to the field of medical image analysis technology, specifically relating to a lightweight left ventricular echocardiography segmentation method and system based on Mamba. Background Technology

[0002] Left ventricular segmentation in echocardiography is a crucial step in calculating the left ventricular ejection fraction and thus assessing cardiac systolic function. Manual segmentation is time-consuming and often exhibits significant variability between and within observers; therefore, automated segmentation methods are highly needed in clinical practice. With the advent of deep learning, deep learning-based automated medical image segmentation models have been extensively explored. UNet, with its symmetrical U-shaped encoder-decoder structure and good segmentation performance, has laid the structural foundation for medical image segmentation models and has thus been widely incorporated into various segmentation tasks and methods. These methods are typically implemented using approaches represented by Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs). Convolutions have excellent local feature extraction capabilities but are insufficient in establishing correlations of long-range information. Transformer-based methods exhibit superior performance in global modeling but also incur significant computational burden. This is because the self-attention mechanism has a quadratic computational complexity closely related to the input image size, especially for tasks requiring dense prediction, such as medical image segmentation. This not only ignores the limitations of computational resources in real-world medical environments but also fails to meet the demands of mobile healthcare tasks for low-parameter and low-computational-cost models. Therefore, there is an urgent need to design a left ventricular echocardiographic segmentation model with low parameter count and low computational load.

[0003] The prior art, document number CN117197594B, discloses a cardiac shunt classification system based on a deep neural network, comprising: an echocardiogram preprocessing module for preprocessing raw echocardiograms; a clutter filtering module for removing noise from ultrasound images; an ultrasound microbubble localization module for locating microbubbles in ultrasound images; a segmentation training module for training a left ventricular chamber segmentation model using a U-Net network; a left ventricular chamber segmentation module for segmenting ventricular chambers from the preprocessed echocardiogram using the trained U-Net segmentation model; a microbubble number multivariate time series data generation module for generating microbubble number multivariate time series data by combining the results of the ultrasound microbubble localization module and the left ventricular chamber segmentation module; a classification training module for training a classification model using an LSTM-FCN network; and a classification module for classifying the microbubble number multivariate time series data generated from the raw echocardiogram using the trained LSTM-FCN classification model. This prior art improves the accuracy of cardiac shunt disease classification.

[0004] The prior art (CN118537317A) provides a deep learning-based algorithm for evaluating left ventricular ejection fraction (LVEF) in two-dimensional echocardiography. The algorithm includes: acquiring and preprocessing a publicly available echocardiography dataset (CAMUS); dividing the preprocessed dataset into training and testing sets according to a set ratio; establishing a cascaded echocardiography segmentation model based on deep learning, incorporating attention mechanisms and two-dimensional convolutions; iteratively training the segmentation model by sequentially inputting the processed echocardiography dataset training set; wherein the segmentation loss function is a combination of the Dice loss function and the cross-entropy loss function; loading the training model; inputting the test set of echocardiography images to be segmented into the training model to obtain the segmentation results of the left ventricle and left atrium images; and evaluating the ejection fraction based on the segmentation results. This proposed method can accurately and efficiently segment the target regions of the left ventricle and left atrium in echocardiography images and evaluate the ejection fraction based on the left ventricular region.

[0005] As can be seen from the above existing technologies, the existing technologies do not propose how to design a left ventricular echocardiography segmentation model with low parameter quantity and low computational load to achieve the lightweight segmentation task of left ventricular echocardiography.

[0006] Recently, a novel architecture based on the State-Space Model (SSM), namely Mamba, has opened a new avenue for visual understanding. Mamba uses variable parameters to selectively represent global dependencies and leverages hardware-aware parallel computing algorithms to optimize storage and computational efficiency. This makes Mamba not only adept at capturing long-range dependencies but also exhibit linear complexity related to input size, making it a strong competitor to CNNs and Transformers in the pursuit of lightweighting UNet. Methods based on Mamba were initially applied to sequence tasks and have recently been increasingly used in computer vision tasks. However, the U-shaped structure method combining Mamba has not yet been thoroughly explored in the lightweight segmentation task of left ventricular echocardiography. Summary of the Invention

[0007] The technical problem to be solved by this invention is:

[0008] To address the issues of insufficient segmentation accuracy, large number of parameters, and high computational complexity in current deep learning-based echocardiography segmentation models, which result in the consumption of substantial storage and computational resources, this invention proposes a lightweight U-shaped segmentation method based on Mamba.

[0009] The technical solution adopted by the present invention to solve the above-mentioned technical problems is as follows:

[0010] A lightweight left ventricular echocardiography segmentation method based on Mamba includes the following modules: an image feature encoding module, a residual visual Mamba module, a feature aggregation bridge module, an adaptive feature enhancement module, a mask generation module, and a segmentation map output module. It can be divided into two stages: an encoder stage and a decoder stage.

[0011] Encoder stage: The image feature encoding module is connected to the residual vision Mamba module, and the residual vision Mamba module is alternately connected to the adaptive feature enhancement module. The feature map processed by each layer is passed to the feature aggregation bridge module of the current layer.

[0012] In the decoder stage: the adaptive feature enhancement module and the residual visual Mamba module are alternately connected, and the residual visual Mamba module is connected to the segmentation map output module. The adaptive feature enhancement module, residual visual Mamba module, and segmentation map output module in the decoder stage are also used to receive and process the outputs from both the previous and current layer feature aggregation bridge modules and stitch them together to form a feature map. Both the adaptive feature enhancement module and the residual visual Mamba module in the decoder stage are connected to the mask generation module.

[0013] 1) The image feature encoding module (Patch Embedding) is used to initially encode the original image to prepare for the next step of image feature extraction; it is connected to the residual vision Mamba module to pass the encoded feature map to the residual vision Mamba module.

[0014] 2) The Residual Vision Mamba module (ResVMamba) is used to aggregate multi-angle features by scanning images from multiple directions; it is used to adaptively adjust the contribution ratio of the input, thereby enhancing the learning and expressive capabilities of the model; it is also used to significantly reduce the number of parameters while ensuring accurate segmentation results.

[0015] 3) Feature Aggregation Bridge (GAB) module, which receives three parts of input: low-dimensional feature map, high-dimensional feature map and a mask map; it is used to pass and merge high-dimensional features to low-dimensional layers layer by layer; it is used to integrate information at different scales; and it is also used to pass the aggregated features to the decoder.

[0016] 4) Adaptive Feature Enhancement Module (AFE): This module is used to process echocardiographic data in which there are differences in left ventricular size and morphology between individuals while maintaining a low number of parameters.

[0017] 5) The mask generation module is connected to the decoder and is used to receive the feature maps processed by the decoder; it is used to process and output the segmentation output mask of each layer; the generated mask is used to calculate the segmentation loss of each layer.

[0018] 6) Segmentation map output module (Final Conv), connected to the residual vision Mamba module, is used to receive the feature map from both the residual vision Mamba module and the feature aggregation bridge module and stitch them together, and then process it; it is also used to output the overall segmentation result processed by the model.

[0019] Furthermore, a lightweight left ventricular echocardiographic segmentation method based on Mamba includes the following steps:

[0020] Step S1: Obtain an echocardiogram video dataset with coordinate annotations of the left ventricular segmentation region, and unify the video format to AVI.

[0021] Step S2: Preprocess each video from step S1, cropping and masking text information outside the sectors, and standardizing the video size;

[0022] Step S3: Divide the dataset from step S2 into a training set, a validation set, and a test set according to a certain ratio;

[0023] Step S4: Slice the echocardiogram videos obtained in step S3 (training set, validation set, and test set) to form frame-by-frame images;

[0024] Step S5: Process the coordinate annotations of the segmented regions in the dataset from Step S1 to generate a segmentation label map GroundTruth.

[0025] Step S6: Construct an echocardiographic segmentation model, which includes the following parts:

[0026] Image Feature Encoding Module (Patch Embedding): This module performs initial encoding on the original image, preparing it for the next step of image feature extraction;

[0027] Residual Vision Mamba Module (ResVMamba): This module introduces the Mamba method, which aggregates multi-angle features by scanning the image from multiple directions and adaptively adjusts the contribution ratio of the input by introducing weighted residuals to enhance the model's learning and expressive capabilities.

[0028] Feature Aggregation Bridge (GAB) module: This module receives three inputs: a low-dimensional feature map, a high-dimensional feature map, and a mask map. It then performs grouping processing and uses dilated convolutions with different dilation rates for feature extraction.

[0029] Adaptive Feature Enhancement Module (AFE): This module uses deformable convolution and depthwise separable convolution to effectively process echocardiographic data where there are differences in the size and shape of the left ventricle between individuals;

[0030] The mask generation module receives the feature map processed by the decoder and further outputs the segmentation output mask for each layer. The mask is used to calculate the segmentation loss for each layer.

[0031] Segmentation Output Module (Final Conv): This module is used to output the segmentation results after the model has been processed as a whole;

[0032] Step S7: Prepare the computer and NVIDIA RTX 3090 GPU hardware, and set up the PyTorch 2.0.0 framework and Python 3.8 environment.

[0033] Step S8: Set the number of training epochs; set the batch size, which is the number of samples used in one forward propagation during training; set the loss function Loss; set the initial learning rate lr and weight decay; set the optimizer to calculate gradients and update parameters to minimize the loss function Loss; set the learning rate scheduler to automatically adjust the learner's learning rate; this step prepares the model for training.

[0034] Step S9: Use the training set from step S4 to train the segmentation model from step S6. The training settings are the same as in step S8. During the training process, the model segmentation output image and the segmentation label image from step S5 will be input into the loss function Loss from step S8 for loss calculation. The loss value will be gradually reduced through the optimizer and learning rate scheduler from step S8 until the number of training iterations reaches the preset epochs from step S8, at which point training will stop.

[0035] During the training process in steps S10 and S9, at the end of each epoch, the validation set in step S4 is used for validation, that is, relevant evaluation metrics are calculated for the segmentation model output map and the segmentation label map in step S5, and the best validation weights are saved for testing.

[0036] Step S11: Load the optimal validation weights from step S10 into the segmentation model from step 6, and test the model using the test set from step S4. If the expected results are not achieved, return to step S6, adjust the model, and repeat steps S7-S11 until the expected results are achieved.

[0037] Furthermore, the residual visual Mamba module in step S6 includes two main calculation steps: the VSS module and the weighted residual. The specific process is as follows:

[0038] Step S61: The input feature map is processed by a linear layer to obtain Linear(X) inThe feature map X1 is obtained by using depthwise separable convolution (DWConv) to extract important features and applying the activation function SiLU. Then, it is scanned in multiple directions by SS2D to collect multi-directional features. Finally, layer normalization is performed to obtain the feature map X1.

[0039] X1=LayerNorm(SS2D(SiLU(DWConv(Linear(X in )))))

[0040] Step S62: Input feature map X in Linear(X) is obtained after linear layer processing. in The feature map X2 is then obtained by processing it with the activation function SiLU.

[0041] X2=SiLU(Linear(X in ))

[0042] Step S63: Perform dot product and linear layer processing on the results obtained in steps S61 and S62 to obtain the output feature map X of the VSS module. out ;

[0043] X out =Linear(X1⊙X2)

[0044] Step S64: Process the input feature map X in Calculate the weighted residuals: weight×X in The feature map Y is obtained by adding the calculation results of the VSS module obtained in steps S61 to S63.

[0045] Y = VSSM(X) in )+weight×X in

[0046] Step S65: Perform layer normalization, linear layer, max pooling, and GELU activation function calculation on the result obtained in step S64 to obtain the final output feature map Y of the residual visual Mamba module in step S6. out ;

[0047] Y out =GELU(MaxPooling(Linear(LayerNorm(Y))))

[0048] Furthermore, the adaptive feature enhancement module (AFE) in step S6 is implemented as follows:

[0049] Step S66: The input feature map X is processed by deformable convolution to calculate DeformConv(OffsetsConv(X),X), and then layer normalization, max pooling and GELU activation function are performed to obtain feature map Y1;

[0050] Y1=GELU(MaxPooling(LayerNorm(DeformConv(OffsetsConv(X),X))))

[0051] Step S67: Input feature map X is processed by depthwise separable convolution (DWConv) and SiLU activation function to obtain feature map Y2;

[0052] Y2=SiLU(DWConv(X))

[0053] Step S68: Add the calculation result obtained in step S66 to the calculation result obtained in step S67 to obtain the output feature map Y of the adaptive feature enhancement module (AFE) in step S6. out ;

[0054] Y out =Y1+Y2

[0055] Furthermore, the grouping process mentioned in the Feature Aggregation Bridge (GAB) module in step S6 involves first unifying the size of the high-dimensional and low-dimensional feature maps, then dividing them into four equal parts according to the channel dimension. The corresponding high-dimensional and low-dimensional parts, along with the mask, are then concatenated to form four groups. Finally, dilated convolutions with different dilation rates are used to extract multi-scale features from each group of feature maps, such as... Figure 4 As shown.

[0056] Furthermore, the masks of different sizes generated by the mask generation module in step S6 are used for depth supervision, that is, the loss is calculated by comparing the segmentation output map mask and the segmentation label map GroundTruth of different layers of the model.

[0057] Furthermore, the loss function mentioned in step S8 is calculated by combining the segmentation output map mask and the segmentation label map GroundTruth for each layer of the model. i and with different weighting coefficients λ i Multiplication, the loss is calculated as follows:

[0058]

[0059] Where λ i These are the loss weights for different layers.

[0060] Furthermore, Loss i It is the sum of cross-entropy loss and Dice loss.

[0061] Loss i =L Bce (y,y pre [i])+L Dice (y,y pre [i])

[0062] Where L Bce and L Dice These represent the binary cross-entropy loss and the Dice loss, respectively, where y represents the Ground Truth. pre [i] represents the mask of the output of different layers.

[0063] Furthermore, the cross-entropy loss L Bce The specific calculation process is as follows:

[0064]

[0065] Where y i It's the GroundTruth tag. This is the probability value predicted by the model, and N is the total number of pixels.

[0066] Furthermore, Dice lost L Dice The specific calculation process is as follows:

[0067]

[0068] Where y i It's the GroundTruth tag. This is the probability value predicted by the model, and N is the total number of pixels.

[0069] The beneficial effects of this invention are:

[0070] This invention improves upon the traditional UNet architecture, addressing the issues of large parameter count and high computational load in current left ventricular echocardiography segmentation models. First, it employs Mamba as a lightweight optimization strategy. The residual visual Mamba module aggregates multi-angle features by scanning the image in multiple directions. By adaptively adjusting the contribution ratio of the input using weighted residuals, it enhances the model's learning and expressive capabilities, significantly reducing the number of parameters while maintaining more accurate segmentation. Second, it uses an adaptive feature enhancement module based on deformable and depthwise separable convolutions to effectively handle echocardiographic data with variations in left ventricular size and shape among individuals. Third, it utilizes a feature aggregation bridge module (GAB) as a skip connection between the encoder and decoder, effectively transferring high-dimensional features layer by layer to low-dimensional features and integrating encoder features into decoder features. Furthermore, it employs deep supervision, calculating the loss between the segmentation output of each layer and GroundTruth, multiplying it by the corresponding weights. Finally, the lightweight segmentation method provided by this invention is a novel framework with high practicality. It can solve the problems of large model parameters and high computational load in existing echocardiography segmentation methods in mobile medical applications, while maintaining strong segmentation performance and having strong generalization ability. It has the potential to be more widely used in clinical applications.

[0071] This invention addresses the problems of insufficient segmentation accuracy, high computational complexity, and large number of parameters in existing medical image segmentation methods. It effectively solves the challenges posed by storage and computing resource limitations in real medical environments. Based on the state-space model Mamba, this invention constructs a U-shaped lightweight segmentation method. It uses an image feature encoding module to encode the original image and constructs a residual visual Mamba module to aggregate multi-angle features from multiple directions of image scanning. It also constructs an adaptive feature enhancement module to handle echocardiographic data with differences in size and shape between individuals. A feature aggregation bridge module is used to integrate multiple features and pass high-dimensional features to low-dimensional layers layer by layer. A mask generation module is constructed to output the segmentation result map of the current layer. A segmentation map output module is used to output the overall segmentation result processed by the model.

[0072] This invention is applicable to the field of medical image analysis. Attached Figure Description

[0073] Figure 1 This is a schematic diagram of the overall structure of the method;

[0074] Figure 2 A schematic diagram of the ResVMamba structure for residual visual Mamba blocks;

[0075] Figure 3 This is a schematic diagram of the Adaptive Feature Enhancement (AFE) module structure.

[0076] Figure 4 This is a schematic diagram of the GAB structure of the feature aggregation bridge module. Detailed Implementation

[0077] Specific implementation method one, which uses the EchoNet-Dynamic dataset, includes the following steps:

[0078] Step S1: Load the EchoNet-Dynamic echocardiography video dataset and unify the videos into AVI format;

[0079] Step S2: Preprocess each video from step S1, cropping and masking text information outside the sector, and unifying the video size to 112*112;

[0080] Step S3: Divide the dataset from step S2 into a training set, a validation set, and a test set in a ratio of 8:1:1;

[0081] Step S4: Slice the echocardiogram videos obtained in step S3 (training set, validation set, and test set) to form frame-by-frame images;

[0082] Step S5: Process the coordinate annotations of the segmented regions in the dataset from Step S1 to generate a segmentation label map GroundTruth.

[0083] Step S6: Construct an echocardiographic segmentation model. The overall structure of the model is shown in the diagram below. Figure 1 As shown, the model comprises the following parts:

[0084] 1) Image Feature Encoding Module (Patch Embedding): This module performs initial encoding on the original image to prepare for the next step of image feature extraction;

[0085] 2) Residual Vision Mamba Module (ResVMamba): This module introduces the Mamba method, aggregating multi-angle features through multi-directional image scanning. It adaptively adjusts the contribution ratio of the input by introducing weighted residuals to enhance the model's learning and expressive capabilities. The module's structure diagram is shown below. Figure 2 As shown;

[0086] 3) Feature Aggregation Bridge Module (GAB): This module receives three inputs: a low-dimensional feature map, a high-dimensional feature map, and a mask map. It then performs grouping processing and uses dilated convolutions with different dilation rates for feature extraction. The module structure diagram is shown below. Figure 4 As shown;

[0087] 4) Adaptive Feature Enhancement Module (AFE): This module employs deformable convolution and depthwise separable convolution to effectively process echocardiographic data where there are differences in left ventricular size and morphology between individuals; the module's structure diagram is shown below. Figure 3 As shown;

[0088] 5) Mask generation module: This module receives the feature map processed by the decoder and further outputs the segmentation output result mask of each layer. The mask is used to calculate the segmentation loss of each layer.

[0089] 6) Segmentation Output Module (Final Conv): This module is used to output the segmentation results after the overall model processing;

[0090] Step S7: Prepare the computer and NVIDIA RTX 3090 GPU hardware, and set up the PyTorch 2.0.0 framework and Python 3.8 environment.

[0091] Step S8: Set the number of training epochs to 50; set the number of samples used in one forward propagation during training, i.e., the batch size to 4; set the initial learning rate to 0.001, the weight decay to 0.0001, set the optimizer AdamW to calculate gradients and update parameters to minimize the loss function, and set the learning rate scheduler CosineAnnealingLR to automatically adjust the optimizer's learning rate; this step prepares the model for training.

[0092] Step S9: Use the training set from Step S4 to train the segmentation model from Step S6. The training settings are the same as in Step S8. During training, the model segmentation output image and the segmentation label image from Step S5 are input into the loss function Loss from Step S8 for loss calculation. The loss value is gradually reduced through the AdamW optimizer and CosineAnnealingLR learning rate scheduler from Step S8 until the number of training iterations reaches the preset number of epochs in Step S8, at which point training stops.

[0093] During the training process in steps S10 and S9, at the end of each epoch, the validation set in step S4 is used for validation, that is, relevant evaluation metrics are calculated for the segmentation model output map and the segmentation label map in step S5, and the best validation weights are saved for testing.

[0094] Step S11: Load the optimal validation weights from step S10 into the segmentation model from step 6, and test the model using the test set from step S4. If the expected results are not achieved, return to step S6, adjust the model, and repeat steps S7-S11 until the expected results are achieved.

[0095] Step S6, the residual visual Mamba module, includes two main calculation steps: the VSS module and the weighted residual. The specific process is as follows:

[0096] Step S61: The input feature map is processed by a linear layer to obtain Linear(X) in The feature map X1 is obtained by using depthwise separable convolution (DWConv) to extract important features and applying the activation function SiLU. Then, it is scanned in multiple directions by SS2D to collect multi-directional features. Finally, layer normalization is performed to obtain the feature map X1.

[0097] X1=LayerNorm(SS2D(SiLU(DWConv(Linear(X in )))))

[0098] Step S62: The input feature map is processed by a linear layer to obtain Linear(X) in The feature map X2 is then obtained by processing it with the activation function SiLU.

[0099] X2=SiLU(Linear(X in ))

[0100] Step S63: Perform dot product and linear layer processing on the results obtained in steps S61 and S62 to obtain the output feature map X of the VSS module. out ;

[0101] X out =Linear(X1⊙X2)

[0102] Step S64: Calculate the weighted residual weight×X on the input feature map. in The feature map Y is obtained by adding the calculation results of the VSS module obtained in steps S61 to S63.

[0103] Y = VSSM(X) in )+weight×X in

[0104] Step S65: Perform layer normalization, linear layer, max pooling, and GELU activation function calculation on the result obtained in step S64 to obtain the final output feature map Y of the residual visual Mamba module in step S6. out ;

[0105] Y out =GELU(MaxPooling(Linear(LayerNorm(Y))))

[0106] The adaptive feature enhancement module in step S6 works as follows:

[0107] Step S66: The input feature map X is processed by deformable convolution to calculate DeformConv(OffsetsConv(X),X), and then layer normalization, max pooling and GELU activation function are performed to obtain feature map Y1;

[0108] Y1=GELU(MaxPooling(LayerNorm(DeformConv(OffsetsConv(X),X))))

[0109] Step S67: Input feature map X is processed by depthwise separable convolution (DWConv) and SiLU activation function to obtain feature map Y2;

[0110] Y2=SiLU(DWConv(X))

[0111] Step S68: Add the calculation result obtained in step S66 to the calculation result obtained in step S67 to obtain the output feature map Y of the adaptive feature enhancement module in step S6. out ;

[0112] Y out =Y1+Y2

[0113] The grouping process mentioned in the Feature Aggregation Bridge (GAB) module in step S6 involves first unifying the size of the high-dimensional and low-dimensional feature maps, then dividing each into four equal parts based on the channel dimension. Next, the corresponding high-dimensional and low-dimensional parts, along with a mask, are concatenated to form four groups. Finally, dilated convolutions with different dilation rates are used to extract multi-scale features from each group of feature maps. Figure 4 As shown.

[0114] In step S6, the mask generation module generates masks of different sizes for each layer, which are used for depth supervision. That is, the loss is calculated by comparing the segmentation output map mask and the segmentation label map GroundTruth for different layers of the model.

[0115] The loss function mentioned in step S7 is calculated by combining the segmentation output map mask and the segmentation label map GroundTruth of each layer of the model. i and with different weighting coefficients λ i Multiplication, the loss is calculated as follows:

[0116]

[0117] Where λ i These are the loss weights for different layers. In this implementation, λ0 to λ5 are set to 1, 0.5, 0.4, 0.3, 0.2, and 0.1.

[0118] Loss iIt is the sum of cross-entropy loss and Dice loss.

[0119] Loss i =L Bce (y,y pre [i])+L Dice (y,y pre [i])

[0120] Where L Bce and L Dice These represent the binary cross-entropy loss and Dice loss, respectively, where y represents Ground Truth. pre [i] represents the mask of the output of different layers.

[0121] Cross-entropy loss L Bce The specific calculation process is as follows:

[0122]

[0123] Where y i GroundTruth is a tag. This is the probability value predicted by the model, and N is the total number of pixels.

[0124] Dice loss L Dice The specific calculation process is as follows:

[0125]

[0126] Where y i It's the GroundTruth tag. This is the probability value predicted by the model, and N is the total number of pixels.

[0127] The second specific implementation scheme differs from the first implementation scheme in that it uses a different video dataset or a different image dataset.

[0128] For example, the EchoNet-Pediatric dataset. This dataset contains 7,643 echocardiographic videos of the apical four-chamber (A4C) and parasternal short-axis (PSAX) views collected at Stanford University's Lucile Packard Children's Hospital between 2014 and 2021. This dataset provides segmentation labels for the left ventricular region in both the apical four-chamber and parasternal short-axis views, allowing for the division of 6,114 (80%) samples for training, 765 (10%) for validation, and 764 (10%) for testing. The processing steps are the same as in Implementation Plan 1.

[0129] Verification or simulation experiments of the method of this invention:

[0130] To validate the segmentation performance of our method in left ventricular echocardiography segmentation, comparative and ablation experiments were conducted on the EchoNet-Dynamic dataset. This dataset comprises 10,030 apical four-chamber (A4C) echocardiographic videos collected at Stanford University Hospital between 2016 and 2018, each processed according to steps S2-S5. In this work, 7465 samples were used for training, 1288 for validation, and 1277 for testing.

[0131] Furthermore, all experiments were conducted on a single NVIDIA RTX 3090 GPU using the PyTorch 2.0.0 framework and Python 3.8. A combination of cross-entropy loss and Dice loss was employed to balance segmentation accuracy and robustness. Additionally, the AdamW optimizer was used with an initial learning rate of 0.001 and weight decay of 0.01. The CosineAnnealingLR learning rate scheduler was also applied. This method was trained for 50 epochs on the Echonet-Dynamic dataset with a batch size of 4.

[0132] Furthermore, we primarily use six performance evaluation metrics: (1) Dice similarity coefficient (DSC) to evaluate pixel overlap between ground truth and model predictions. (2) Mean intersection-to-union ratio (MIoU) to quantify pixel-level classification accuracy. (3) Hausdorff distance 95% (HD95) to evaluate local boundary segmentation bias between segmented region contours and annotated region contours. A smaller value is better. (4) Mean symmetric surface distance (ASSD) to quantify global average boundary bias. A smaller value is better. (5) Number of parameters to measure model memory usage and model complexity. (6) Number of FLOPs to measure model computational complexity and hardware computational resource consumption.

[0133] The comparative study in Table 1 demonstrates the effectiveness of the model of this invention, where “Params” represents the model size and “FLOPs” represents the computational cost of the model processing a 112×112 image. The “mIoU” and “DSC” metrics reflect segmentation accuracy, while “HD95” and “ASSD” measure boundary accuracy.

[0134] Specifically, compared to the baseline EchoNet, this method demonstrates superior segmentation capabilities, with a 1.04% improvement in Dice score, a 1.63% improvement in mIoU, a 0.22-unit improvement in HD95, and a 0.12-unit improvement in ASSD. CMU-Net, H2former, and Swin-UNet lag behind this method by 1.94%, 1.84%, and 1.83% in Dice, respectively; by 3.35%, 3.18%, and 3.15% in mIoU, respectively; by 0.63, 0.37, and 4.04 units in HD95 score, respectively; and by 0.23, 0.22, and 1.57 units in ASSD, respectively. UNet is closest to this method in Dice, differing by only 0.84%. The performance gaps in boundary metrics are even smaller, showing only a small change of 0.07 units in HD95 and a tiny difference of 0.07 units in ASSD value.

[0135] Table 1: Performance comparison of this method with previous methods on left ventricular segmentation based on the Echonet-Dynamic dataset.

[0136]

[0137] EchoNet refers to the document "Ouyang D, He B, Ghorbani A, et al. Video-based AI forbeat-to-beat assessment of cardiac function [J]. Nature, 2020,580(7802):252-256."; UNet refers to the document "Ronneberger O, Fischer P, Brox TU-net: Convolutional networks for biomedical image segmentation [C] / / Medical image computing andcomputer-assisted intervention–MICCAI 2015:18th international conference,Munich,Germany,October 5-9,2015,proceedings,part III 18.SpringerInternational Publishing,2015:234-241."; Swin-UNet refers to the document "Cao H, Wang Y, ChenJ, et al. Swin-unet: Unet-like pure transformer for medical image segmentation[C] / / European conference on computer vision.Cham:Springer Nature Switzerland, 2022:205-218.”; H2former refers to the literature “He A,Wang K,Li T,et al.H2Former:An efficient hierarchical hybrid transformer for medical image segmentation[J].IEEE Transactions on Medical Imaging,2023,42(9):2763-2775.”; CMU-Net refers to the literature “Tang F,Wang L,Ning C,et al.Cmu-net: a strong convmixer-based medical ultrasoundimage segmentation network[C] / / 2023IEEE 20th international symposium onbiomedical imaging(ISBI).IEEE,2023:1-5.".

[0138] The ablation studies in Table 2 quantify the contributions of our proposed components, demonstrating that integrating the new components into the baseline significantly improves performance. Conversely, removing them from the model results in performance degradation. "ED-DSC" represents the DSC metric evaluation of the model for segmenting end-diastolic frames, and "ES-DSC" represents the DSC metric evaluation of the model for segmenting end-systolic frames.

[0139] First, the baseline method is used as a benchmark, with an overall Dice of 91.55% (89.87% for the final contraction frame and 92.60% for the final diastole frame), and HD95 and ASSD values ​​of 3.61 and 1.42, respectively.

[0140] Secondly, we upgraded the baseline architecture by replacing all skip connections with GAB modules and introducing a deep supervision (DS) mechanism, thus establishing Baseline2. Baseline2 increases the number of parameters by 0.01M and the computational cost by 2.699M, while improving the overall Dice by 0.46% (0.28% improvement in end-diastolic frames and 0.76% improvement in end-contraction frames), HD95 by 0.34 units, and ASSD by 0.08 units.

[0141] Furthermore, when the ResVMamba module and standard convolutions replaced the encoder and decoder components in Baseline2, the overall Dice score of the model improved by 0.4%, HD95 by 0.24 units, and ASSD by 0.1 units. Specifically, the Dice score at the end of the diastolic phase improved by 0.33%, while the Dice score at the end of the contraction phase improved by 0.48%. These results demonstrate that the ResVMamba-Conv hybrid structure exhibits superior performance in small object segmentation, particularly with significantly better segmentation accuracy at the end of the contraction phase compared to the end of the diastolic phase.

[0142] Subsequently, when the AFE module replaced all standard convolutions, the model's overall Dice score improved by 0.63%, HD95 improved by 0.11 units, and ASSD improved by 0.04 units. Notably, a similar 0.63% improvement in Dice score was observed in the end-diastolic and end-contraction frames. This demonstrates the AFE module's excellent ability in multi-scale feature processing, effectively capturing features of both large structures and small targets.

[0143] Finally, when all modules are integrated, the proposed model achieves the best segmentation performance (Dice: 93.04%) and the best boundary accuracy (HD95: 2.92 mm), while maintaining excellent efficiency (0.323M parameters, 39.873M FLOPs). Compared to the baseline, Echo-Mamba increases the number of parameters and computational cost, but improves performance by 1.49% on Dice, 0.69 units on HD95, and 0.22 units on ASSD. These results demonstrate that integrating all modules significantly enhances the model's ability to capture both fine-grained and global features.

[0144] Table 2: Ablation experiments of this method on the Echonet-Dynamic dataset, quantifying the contribution of the proposed components.

[0145]

[0146] Matters not covered in this invention are common knowledge.

[0147] The above-described examples are merely illustrative of the technical concept and workflow of this invention, intended to enable those skilled in the art to understand and implement the invention, and should not be construed as limiting the scope of protection of this invention. For those skilled in the art, various variations or modifications can be made based on the above description; it is impossible to exhaustively list all possible implementations here. All equivalent changes or modifications made to the technical solution of this invention should be included within the scope of protection of this invention.

Claims

1. A lightweight left ventricular echocardiographic segmentation method based on Mamba, comprising the following steps: Step S1: Obtain an echocardiogram video dataset with coordinate annotations of the left ventricular segmentation region, and unify the video format to AVI. Step S2: Preprocess each video from step S1, cropping and masking text information outside the sectors, and standardizing the video size; Step S3: Divide the dataset from step S2 into a training set, a validation set, and a test set according to a certain ratio; Step S4: Slice the echocardiogram videos obtained in step S3 (training set, validation set, and test set) to form frame-by-frame images; Step S5: Process the coordinate annotations of the segmented regions in the dataset from Step S1 to generate a segmentation label map GroundTruth. Step S6: Construct an echocardiographic segmentation model. The overall structure of the model includes the following parts: Image feature encoding module: This module performs initial encoding on the original image to prepare for the next step of image feature extraction; Residual Vision Mamba Module: This module introduces the Mamba method, which aggregates multi-angle features by scanning the image from multiple directions and adaptively adjusts the contribution ratio of the input by introducing weighted residuals to enhance the model's learning and expressive capabilities. Feature aggregation bridge module: This module receives three inputs: a low-dimensional feature map, a high-dimensional feature map, and a mask map. It then performs grouping processing and uses dilated convolutions with different dilation rates for feature extraction. Adaptive Feature Enhancement Module: This module uses deformable convolution and depthwise separable convolution to effectively process echocardiographic data where there are differences in the size and shape of the left ventricle among individuals; Mask generation module: This module receives the feature map processed by the decoder and further outputs the segmentation output mask for each layer; the generated mask is used to calculate the segmentation loss for each layer; Segmentation output module: This module is used to output the segmentation results after the model has been processed as a whole; Step S7: Prepare the computer and NVIDIA RTX 3090 GPU hardware, and set up the PyTorch 2.0.0 framework and Python 3.8 environment; Step S8: Set the number of training epochs; set the batch size, which is the number of samples used in one forward propagation during training; set the loss function Loss. Set the initial learning rate (lr) and weight decay, set up the optimizer to calculate gradients and update parameters to minimize the loss function (Loss), and set up the learning rate scheduler to automatically adjust the optimizer's learning rate; this step prepares the model for training. Step S9: Use the training set from step S4 to train the segmentation model from step S6. The training settings are the same as in step S8. During the training process, the model segmentation output image and the segmentation label image from step S5 will be input into the loss function Loss from step S8 for loss calculation. The loss value will be gradually reduced through the optimizer and learning rate scheduler from step S8 until the number of training iterations reaches the preset epochs from step S8, at which point training will stop. During the training process in steps S10 and S9, at the end of each epoch, the validation set in step S4 is used for validation, that is, relevant evaluation metrics are calculated for the segmentation model output map and the segmentation label map in step S5, and the best validation weights are saved for testing. Step S11: Load the optimal validation weights from step S10 into the segmentation model from step S6, and test the model using the test set from step S4. If the expected results are not achieved, return to step S6, adjust the model, and repeat steps S7-S11 until the expected results are achieved.

2. The lightweight left ventricular echocardiography segmentation method based on Mamba according to claim 1, characterized in that... Step S6, the residual visual Mamba module, includes two main calculation steps: the VSS module and the weighted residual. The specific process is as follows: Step S61: The input feature map is processed by a linear layer to obtain... After depthwise separable convolution Extract important features and use activation functions Subsequently, SS2D is used for multi-directional scanning to collect multi-directional features, and finally, layer normalization is performed to obtain the feature map. ; Step S62: The input feature map is processed by a linear layer to obtain... Then use the activation function The feature map is obtained through processing. ; Step S63: Perform dot product and linear layer processing on the results obtained in steps S61 and S62 to obtain the output feature map of the VSS module. ; Step S64: Calculate the weighted residuals of the input feature map. The feature map is obtained by adding the calculation results of the VSS module obtained in steps S61 to S63. ; Step S65: Perform layer normalization, linear layering, max pooling, and... on the results obtained in step S64. The activation function is calculated to obtain the final output feature map of the residual visual Mamba module in step S6. ; 。 3. A lightweight left ventricular echocardiography segmentation method based on Mamba according to claim 1 or 2, characterized in that... The adaptive feature enhancement module in step S6 works as follows: Step S66: Input feature map After deformable convolution calculation Then, layer normalization, max pooling, and... Feature maps are obtained by activation function calculation ; Step S67: Input feature map After depthwise separable convolution and Feature maps are obtained by activation function calculation ; Step S68: Add the calculation result obtained in step S66 to the calculation result obtained in step S67 to obtain the output feature map of the adaptive feature enhancement module in step S6. ; 。 4. A lightweight left ventricular echocardiography segmentation method based on Mamba according to claim 3, characterized in that... The grouping process mentioned in the feature aggregation bridge module in step S6 is as follows: First, the high-dimensional feature map and the low-dimensional feature map are unified in size. Then, they are divided into four equal parts according to the channel dimension. Next, the corresponding parts of the high-dimensional and low-dimensional features and the mask are concatenated to form four groups. Finally, dilated convolution with different dilation rates is used to extract multi-scale features from each group of feature maps.

5. A lightweight left ventricular echocardiography segmentation method based on Mamba according to claim 4, characterized in that... In step S6, the mask generation module generates masks of different sizes for each layer, which are used for depth supervision. That is, the loss is calculated by comparing the segmentation output map mask and the segmentation label map GroundTruth for different layers of the model.

6. A lightweight left ventricular echocardiography segmentation method based on Mamba according to claim 5, characterized in that... The loss function mentioned in step S8 is calculated by combining the output map mask and the segmentation label map GroundTruth for each layer. and with different weighting coefficients Multiplication, the loss is calculated as follows: in These are the loss weights for different layers.

7. A lightweight left ventricular echocardiography segmentation method based on Mamba according to claim 6, characterized in that... It is the sum of cross-entropy loss and Dice loss. in and These represent the binary cross-entropy loss and the Dice loss, respectively, and y represents Ground Truth. Masks representing the outputs of different layers.

8. A lightweight left ventricular echocardiography segmentation method based on Mamba according to claim 7, characterized in that... Cross-entropy loss The specific calculation process is as follows: Dice loss The specific calculation process is as follows: in It's the GroundTruth tag. It is the probability value predicted by the model. It represents the total number of pixels.

9. A lightweight left ventricular echocardiography segmentation system based on Mamba, characterized in that: The system has a program module corresponding to the steps of any one of claims 1-8, and executes the steps in the Mamba-based lightweight left ventricular echocardiography segmentation method when it is run.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program configured to, when invoked by a processor, implement the steps of the lightweight left ventricular echocardiography segmentation method based on Mamba as described in any one of claims 1-8.

Images