An echocardiography mitral regurgitation grading method, system and medium
By constructing a keyframe-guided dual-spatial attention network, the problem of failing to effectively focus on keyframes and regions in existing technologies is solved, thereby improving the accuracy and clinical reliability of mitral regurgitation grading and providing visual auxiliary support for the severity of mitral regurgitation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- FOURTH MILITARY MEDICAL UNIVERSITY
- Filing Date
- 2026-03-20
- Publication Date
- 2026-06-19
AI Technical Summary
Existing technologies fail to effectively focus on key frames and key regions when classifying mitral regurgitation based on echocardiography, resulting in inaccurate classification.
A keyframe-guided dual-space attention network is constructed, including a feature initialization module, a spatiotemporal encoder, and a classification fusion module. Adaptive attention weights are generated through a keyframe attention module, a color-guided attention module, a large selective kernel spatial attention module, and an adaptive attention fusion module, focusing on blood flow regions and keyframes to achieve accurate classification of the degree of mitral regurgitation.
It improves the accuracy and clinical reliability of mitral regurgitation grading, enhances the consistency between the model decision-making process and clinical diagnosis, and provides visual support for the severity of mitral regurgitation.
Smart Images
Figure CN122243954A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of mitral regurgitation grading technology, specifically to an echocardiogram... Figure 2 Methods, systems, and media for classifying cusp regurgitation. Background Technology
[0002] Color Doppler echocardiography is an important clinical tool for diagnosing mitral regurgitation, classifying it by comprehensively assessing parameters such as left atrial size, effective regurgitation orifice area, and regurgitation fraction. However, despite the increasing prevalence of ultrasound technology, accurate MRI assessment still requires experienced specialists for image acquisition and interpretation, resulting in significant inter-observer variability and poor reproducibility. Therefore, there is an urgent clinical need for more accurate, objective, and automated assessment tools. In recent years, research on echocardiographic analysis based on artificial intelligence has developed rapidly. Deep learning can not only automatically identify anatomical findings of the heart but also recognize imaging features that reflect disease severity but may not yet be apparent to clinicians. Currently available systems can assess a wide range of diseases, including left ventricular ejection fraction (LVEF), ventricular wall thickness measurement, and assist in the diagnosis of diseases such as aortic stenosis, hypertrophic cardiomyopathy, and cardiac amyloidosis.
[0003] However, research on deep learning for automatic MR severity grading remains limited, and its technical difficulty far exceeds that of previous breakthroughs. Due to the temporal sparseness of regurgitation signals, accurate assessment requires integrating multiple ultrasound sections and identifying key systolic regurgitation frames. Currently, some studies have attempted to apply deep learning to automatic MR grading. Vrudhula et al. used an R(2+1)D model to grade MR severity. Long et al. selected the three most severe regurgitation videos for each patient and used an R(2+1)D model for MR severity grading. Long et al. used a spatiotemporal convolutional neural network to grade the severity of mitral regurgitation, tricuspid regurgitation, and aortic stenosis. Zhang et al. used a Mask R-CNN model to extract image features for mitral regurgitation severity grading. Edwards et al. constructed two convolutional neural networks based on pediatric echocardiogram videos for section classification and mitral regurgitation severity grading, respectively. However, these deep learning models extract features uniformly from all frames in the video, becoming insensitive to sparse but crucial systolic frames, leading to inaccurate grading. Summary of the Invention
[0004] This invention provides an echocardiogram Figure 2 A mitral regurgitation grading method addresses the problem of inaccurate grading in existing techniques based on echocardiography due to the lack of focus on key frames and regions. The method includes the following steps:
[0005] Acquire datasets of multi-sectional echocardiographic video sequences from two-chamber, three-chamber, and four-chamber chambers. A keyframe-guided dual-space attention network is constructed and trained on a dataset to obtain a keyframe-guided dual-space attention network for mitral regurgitation severity classification. The keyframe-guided dual-space attention network includes a feature initialization module, a spatiotemporal encoder, and a classification fusion module connected sequentially. The spatiotemporal encoder includes multiple consecutive processing layers, each including a keyframe-guided dual-space attention module and a Video Swin Transformer module, with spatial downsampling between layers achieved through a Patch Merging module. The keyframe-guided dual-space attention module includes a keyframe attention module, a color-guided attention module, a large selectivity kernel spatial attention module, and an adaptive attention fusion module. The feature initialization module decomposes multiple video sequences into spatiotemporally continuous image blocks and maps them into high-dimensional feature vectors. The keyframe attention module enhances the high-dimensional feature vectors with keyframe features during the contraction phase. The color-guided attention module divides the blood flow region into color candidate regions based on the binary mask obtained from the preprocessing of the video sequence data, determines the color prior mask, and generates an attention map focused on the blood flow region by spatial attention weighting based on the color prior mask. The large selective kernel spatial attention module extracts the large receptive field branch features and local detail branch features of the enhanced features respectively, and weights and fuses them to obtain a spatial attention map. The adaptive attention fusion module weights and fuses the enhanced features, the blood flow region attention map, and the spatial attention map to obtain a spatial attention map. The Video Swin Transformer module further spatiotemporally encodes the feature map fused from the spatial attention map and the feature map spatially downsampled by the upper-layer Patch Merging module to obtain spatiotemporal features. The classification fusion module weights and fuses the spatiotemporal features from multiple aspects and maps them to a classification result at the mitral regurgitation patient level.
[0006] Preferably, the feature initialization module includes a three-dimensional image block partitioning module and a linear embedding layer; the three-dimensional image block partitioning module decomposes multiple video sequences into spatiotemporally continuous image blocks; the linear embedding layer maps each image block into a high-dimensional feature vector.
[0007] Preferably, the keyframe attention module performs keyframe feature enhancement on the high-dimensional feature vector during the shrinkage period, including the following steps: For high-dimensional feature vectors , B For batch size, C For the number of channels, T For sequence length, H For height, W Width; The global spatial information of each frame is aggregated by three-dimensional global average pooling to generate frame-level feature vectors. : ; Establish local inter-frame dependencies using a lightweight convolutional layer. : ; The attention weights, ranging from 0 to 1, are output using the Sigmoid function. : ; In the formula, Use the Sigmoid activation function; Based on the weights learned during training, the original input features are weighted and fused through a broadcast mechanism to obtain the enhanced features of the keyframes during the contraction period.
[0008] Preferably, the color-guided attention module generates an attention map focused on the blood flow region, including the following steps: For each frame of the echocardiogram video sequence, the Euclidean distance between the RGB vector of each pixel and its corresponding gray value is calculated to quantify the color saturation, and initial segmentation is performed by thresholding to obtain color candidate regions; The color candidate regions are sequentially expanded and morphologically closed to connect neighboring regions, fill internal holes and smooth boundaries, and output a discriminative color prior mask. The color prior mask is multiplied element-wise with the input feature map to initially concentrate the feature response in the colored region. A lightweight sub-network is used to learn and generate spatial attention weights in this region. After activation by the Sigmoid function, the weights are multiplied again with the color prior mask to generate an attention map focused on the blood flow region, ensuring that the attention weights are strictly distributed within the prior colored region.
[0009] Preferably, the large selective kernel spatial attention module generates a spatial attention map by including the following steps: The large selective kernel spatial attention module includes a shared depthwise separable convolutional layer, two cascaded parallel processing branches, a feature concatenation module, parallel global average pooling layers, global max pooling layers, convolutional layers, feature fusion layers, and convolutional layers. The parallel processing branches include a large receptive field branch and a local detail branch. The large receptive field branch is connected to a depthwise separable convolutional layer and a pointwise convolutional layer in sequence. The local detail branch is connected to a pointwise convolutional layer. The basic spatial representation of the enhanced features is extracted through depthwise separable convolutional layers; The basic spatial representation of this feature is obtained by capturing contextual information at different scales through two parallel branches: a large receptive field branch and a local detail branch, to obtain the large receptive field branch feature and the local detail branch feature. The large receptive field branch features and local detail branch features are concatenated along the channel dimension and then passed through a global average pooling layer and a global max pooling layer respectively to aggregate the channel information of features at different scales and obtain the pooled features. The pooled features are processed through a convolutional layer and a sigmoid activation function to generate a set of adaptive selection weights for spatial dimensions. By adaptively selecting weights, spatially adaptive weighted fusion of large receptive field branch features and local detail branch features is performed to obtain fused features. The fused features are passed through a lightweight convolutional layer with a sigmoid activation function to generate the final spatial attention map.
[0010] Preferably, the adaptive attention fusion module weightedly fuses the blood flow region attention map and the spatial attention map to obtain a spatial attention map, including the following steps: Blood flow area attention map Spatial attention map Stitching along the channel dimension: ; Where [,] indicates channel splicing; Two independent sets of fusion weights are generated through an adaptive attention fusion module. and The adaptive attention fusion module is a lightweight 3D convolutional network, consisting of a 3D convolutional layer, a ReLU activation function, and a pointwise 3D convolutional layer. Attention map of blood flow region using fused weights Spatial attention map Perform weighted summation to generate a spatial attention map. : ; Through learnable scaling parameters Spatial attention is achieved through residual connections. Temporal features enhanced by keyframe attention module Perform fusion and output a spatial attention map: .
[0011] Preferably, the classification fusion module weighted and fused the spatiotemporal features from multiple perspectives, mapping them to a classification result at the mitral regurgitation patient level, including the following steps: For a video containing N slices, each video undergoes spatiotemporal feature extraction via a spatiotemporal encoder. After global average pooling and a fully connected layer, probability distributions for four severity categories are generated: ; in, Indicates the first i The predicted probability vectors of each slice video correspond to the categories of Normal, Mild, Moderate, and Severe mitral regurgitation, respectively; the fusion process is achieved by averaging the category probabilities of all slice videos. ; in, This represents the integrated probability vector; the classification result is determined by the maximum average probability.
[0012] This invention also proposes an echocardiogram Figure 2 A mitral regurgitation grading system, the system comprising: The dataset construction module is used to acquire multi-sectional echocardiographic video sequence datasets of two-chamber, three-chamber, and four-chamber systems. A network construction module is used to construct a keyframe-guided dual-space attention network. The keyframe-guided dual-space attention network includes a feature initialization module, a spatiotemporal encoder, and a classification fusion module connected sequentially. The spatiotemporal encoder includes multiple consecutive processing layers, each layer including a keyframe-guided dual-space attention module and a Video Swin Transformer module, with spatial downsampling between layers achieved through a Patch Merging module. The keyframe-guided dual-space attention module includes a keyframe attention module, a color-guided attention module, a large selective kernel spatial attention module, and an adaptive attention fusion module. The network training module is used to train a keyframe-guided dual spatial attention network using a dataset. Specifically, the feature initialization module decomposes multiple video sequences into spatiotemporally continuous image blocks and maps them to high-dimensional feature vectors; the keyframe attention module enhances the high-dimensional feature vectors with keyframe features during the contraction phase; the color-guided attention module divides the blood flow region into color candidate regions based on a binary mask obtained from preprocessing the video sequence data, determines a color prior mask, and generates an attention map focused on the blood flow region by spatial attention weighting of the enhanced features based on the color prior mask; the large selective kernel spatial attention module extracts the large receptive field branch features and local detail branch features of the enhanced features, and weights and fuses them to obtain the spatial attention map; the adaptive attention fusion module weights and fuses the blood flow region attention map and the spatial attention map to obtain the spatial attention map; the Video Swin Transformer module performs spatiotemporal encoding on the feature map fused from the spatial attention map and the feature map spatially downsampled by the upper-layer Patch Merging module to obtain spatiotemporal features; and the classification fusion module weights and fuses the multi-faceted spatiotemporal features, mapping them to a classification result at the mitral regurgitation patient level.
[0013] The present invention also proposes a computer-readable storage medium storing a data processing program, which, when executed by a processor, implements the aforementioned echocardiography. Figure 2 The steps of the grading method for cusp regurgitation.
[0014] The beneficial effects of this invention are: This invention proposes an echocardiogram Figure 2 This invention introduces a keyframe-guided dual spatial attention module for mitral regurgitation grading. During training, an adaptive attention weight is generated for each frame in the input video segment. This weight reflects the importance of the frame to the final MR grading task, with high attention weights always associated with keyframes in the video sequence. The method proposes a collaborative dual-branch spatial attention mechanism. The large selective kernel spatial attention module dynamically selects the receptive field and adaptively fuses multi-scale spatial context to simultaneously model the overall cardiac motion and local regurgitation details. The color-guided spatial attention module innovatively utilizes prior knowledge from color Doppler blood flow imaging to ensure the network focuses on the blood flow regions most relevant to diagnosis. Through adaptive weighted fusion, the two modules achieve complementary enhancement of anatomical structure and blood flow information in the spatial dimension. The method visualizes the keyframes and key regions selected during model decision-making, demonstrating a high degree of consistency between the model's spatiotemporal attention and clinical diagnostic features, enhancing clinical confidence and providing data-driven support for healthcare professionals in diagnosing the severity of mitral regurgitation. Attached Figure Description
[0015] Figure 1 This is a flowchart of the mitral regurgitation grading method according to an embodiment of the present invention; Figure 2 This is a keyframe-guided dual-space attention network structure diagram according to an embodiment of the present invention; Figure 3 This is a structural diagram of the keyframe-guided dual-space attention module according to an embodiment of the present invention; Figure 4 This is a structural diagram of the color-guided attention module according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the color prior mask comparison in an embodiment of the present invention; Figure 6 This is a structural diagram of the large selective kernel spatial attention module according to an embodiment of the present invention; Figure 7 This is a visualization of the confusion matrix corresponding to the two test sets in this embodiment of the invention. Detailed Implementation
[0016] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.
[0017] Example 1 This invention proposes an echocardiogram Figure 2 The flowchart of the grading method for cusp regurgitation is as follows: Figure 1 As shown, the specific steps are as follows: S1: Obtain multi-sectional echocardiographic video sequence datasets of two-chamber, three-chamber, and four-chamber echocardiography.
[0018] S2: Construct a keyframe-guided dual-space attention network, which includes a feature initialization module, a spatiotemporal encoder, and a classification fusion module connected in sequence. The spatiotemporal encoder includes multiple consecutive processing layers, each of which includes a keyframe-guided dual-space attention module and a Video Swin Transformer module, and spatial downsampling between layers is performed through a Patch Merging module. The keyframe-guided dual-space attention module includes a keyframe attention module, a color-guided attention module, a large selective kernel spatial attention module, and an adaptive attention fusion module.
[0019] S3: A model for grading the severity of mitral regurgitation is obtained through training on a dataset. Specifically, the feature initialization module decomposes multiple video sequences into spatiotemporally continuous image blocks and maps them to high-dimensional feature vectors; the keyframe attention module enhances the systolic keyframe features of the high-dimensional feature vectors; the color-guided attention module divides the blood flow region into color candidate regions based on the binary mask obtained from preprocessing the video sequence data, determines the color prior mask, and generates an attention map focused on the blood flow region by spatial attention weighting based on the enhanced features according to the color prior mask; the large selective kernel spatial attention module extracts the large receptive field branch features and local detail branch features of the enhanced features respectively, and weights and fuses them to obtain the spatial attention map; the adaptive attention fusion module weights and fuses the blood flow region attention map and the spatial attention map to obtain the spatial attention map; the Video Swin Transformer module performs spatiotemporal encoding on the feature map fused from the spatial attention map and the feature map spatially downsampled by the upper-layer Patch Merging module to obtain spatiotemporal features; and the classification fusion module weights and fuses the spatiotemporal features from multiple facets, mapping them to the patient classification results for mitral regurgitation levels.
[0020] The overall architecture of the Key-frame Guided Dual Spatial Attention Network (KFGDSAN) of this invention is as follows: Figure 2 As shown in the figure. (A) is the feature initialization module, which includes a 3D patch partitioning module and a linear embedding layer. (B) is the spatio-temporal encoder, which includes a key-frame guided dual spatial attention module (KGDSAM). (C) is the classification fusion module, namely the multi-view classification fusion module (MCFM).
[0021] The spatiotemporal encoder is the core module for deep feature extraction in this network framework. Its structure is as follows: Figure 2As shown in (B), this module adopts a hierarchical design, comprising four consecutive processing stages. Each stage is processed collaboratively by three core sub-modules to progressively extract spatiotemporal features with rich discriminative information. To simulate the clinical judgment process of ultrasound physicians who "first locate keyframes and then assess reflux regions," this invention introduces a keyframe-guided dual-spatial attention module in each layer of the spatiotemporal encoder. KGDSAM can adaptively learn the temporal attention weights of the video sequence, guiding the model to focus on keyframes containing reflux during contraction, and further strengthening the focus on key regions through its dual-branch spatial attention mechanism. The features enhanced by this attention are fed into the Video Swin Transformer block. This module uses a multi-head self-attention mechanism with a moving window to achieve deep spatiotemporal feature extraction of the focused region. Spatial downsampling and channel dimension are performed between layers through PatchMerging operations, thereby reducing subsequent computational complexity while progressively expanding the receptive field and constructing a multi-scale feature pyramid to capture multi-level information from local details to global structure.
[0022] Specifically, the structure diagram of the keyframe-guided dual-space attention module is as follows: Figure 3 As shown, the module includes a Key-frame Attention Module (KFAM), a Color-guided Attention Module (CGAM), and a Large Selective Kernel Spatial Attention Module (LSKSA). This module employs a hierarchical design, sequentially using the Key-frame Attention Module to filter keyframes from the input features, and then using a dual-branch spatial attention mechanism with adaptive fusion to enhance features in key regions.
[0023] Given input feature tensor First, a keyframe attention module is used to enhance the representation of keyframes during the contraction period: ; in This is the feature recalibrated by the keyframe attention module. Subsequently, this feature is fed in parallel into two spatial attention sub-branches. The color-guided attention module utilizes the blood flow color prior from color Doppler to generate an attention map focused on the blood flow region. The large selective kernel spatial attention module generates an attention map that simultaneously models the overall motion of the heart and local details by dynamically selecting receptive fields and fusing multi-scale context. .
[0024] To adaptively fuse these two complementary spatial attention patterns, this invention designs a learnable Adaptive Attention Fusion Module (AAFM). This module first concatenates the two attention maps along the channel dimension:
[0025] ; Two independent sets of fusion weights are generated through an adaptive attention fusion module. and The adaptive attention fusion module is a lightweight 3D convolutional network, consisting of a 3D convolutional layer, a ReLU activation function, and a pointwise 3D convolutional layer. Attention map of blood flow region using fused weights Spatial attention map Perform weighted summation to generate a spatial attention map. : ; Through learnable scaling parameters Spatial attention is achieved through residual connections. Temporal features enhanced by keyframe attention module Perform fusion and output a spatial attention map: .
[0026] in, This is the final output of the KGDSAM module. This design achieves synergistic enhancement of diagnostic-related spatiotemporal regions through adaptive fusion of temporal keyframe localization and dual-branch spatial attention, providing strong guidance for subsequent deep feature extraction.
[0027] Furthermore, the keyframe attention module can adaptively learn an importance weight for each frame in the video sequence, thereby guiding the model to focus on diagnostic frames that contain condensation reflux.
[0028] For high-dimensional feature vectors , B For batch size, C For the number of channels, T For sequence length, H For height, W Width; The global spatial information of each frame is aggregated by three-dimensional global average pooling to generate frame-level feature vectors. : ; Establish local inter-frame dependencies using a lightweight convolutional layer. : ; The attention weights, ranging from 0 to 1, are output using the Sigmoid function. : ; In the formula, The learned weights are then weighted and fused with the original input features via a broadcast mechanism, thereby enhancing the feature response of keyframes while suppressing the contribution of non-keyframes.
[0029] Inspired by the significant prior knowledge that there is a strong correlation between blood flow signals and specific color codes in color Doppler ultrasound, this invention designs a color-guided attention module to guide the network to focus on key areas related to blood flow. For example... Figure 4 As shown, CGAM adopts a two-stage design: (1) Color-Guided Prior Mask Generation and (2) Attention Learning.
[0030] The color prior mask generation stage generates a high-quality binary mask for each input video preprocessing, serving as a spatial prior for subsequent attention. Specifically, for each frame of the video, this invention calculates the Euclidean distance between the RGB vector of each pixel and its corresponding grayscale value to quantify its "color saturation." Initial segmentation is performed using a threshold to obtain color candidate regions. To ensure the continuity and robustness of the mask, a dilation operation with a kernel size of 25×25 and a morphological closing operation with a kernel size of 15×15 are applied sequentially to connect neighboring regions, fill internal holes, and smooth boundaries, ultimately outputting a discriminative color prior mask. Figure 5 (b) the green area, Figure 5 (a) is the original image.
[0031] During forward propagation, the aforementioned mask is used to constrain the learning range of spatial attention. First, the color prior mask is multiplied element-wise with the input feature map, initially concentrating the feature responses within the colored region. Then, a lightweight subnetwork learns and generates spatial attention weights within this region. Finally, after activation by the sigmoid function, the weights are multiplied again with the color prior mask to ensure that the attention weights are strictly distributed within the prior colored region, thereby achieving enhanced focusing on blood flow-related features.
[0032] The structure of the large selective kernel spatial attention module is as follows: Figure 6As shown, this module aims to adaptively fuse multi-scale spatial context to simultaneously capture the overall motion of the heart (such as left ventricular systole and diastole) and local lesion details (such as regurgitation jet morphology). At its core is a dynamic spatial selection mechanism that adaptively assigns appropriate receptive field sizes to different spatial regions based on the content of the input features.
[0033] This module employs a cascaded dual-path architecture based on shared fundamental features. First, the input features are processed through a shared 5×5 depthwise separable convolutional layer to extract a basic spatial representation. This feature is then fed into two parallel processing branches to capture contextual information at different scales. The large receptive field branch cascades a 7×7 depthwise separable convolution (dilation rate of 3) on top of the fundamental features, followed by a pointwise convolution (1×1 Conv). This path, through the concatenation of two levels of convolutions, is equivalent to a standard 23×23 convolutional kernel on the receptive field, effectively modeling the overall cardiac motion patterns (such as the overall contraction and relaxation of the left ventricle), while the pointwise convolution promotes information exchange between channels. The local detail branch directly feeds the output of the shared 5×5 convolution into a pointwise convolution (1×1 Conv), focusing on preserving and enhancing local fine structural features, such as the morphology of the mitral regurgitation jet. This design employs a cascaded decomposition strategy, which significantly expands the effective receptive field while maintaining low computational complexity, thereby efficiently achieving collaborative modeling of global dynamics and local lesions in cardiac ultrasound images.
[0034] To achieve content-aware spatial adaptive selection, the output features of the two paths are concatenated along the channel dimension, and then subjected to global average pooling and global max pooling respectively to aggregate channel information of features at different scales. The pooled features are processed by a convolutional layer and a sigmoid activation function to generate a set of spatially adaptive selection weights. These weights are used to perform spatially adaptive weighted fusion of the features from the two paths, thereby highlighting the most relevant scale information in different regions of the image.
[0035] Finally, the fused features are passed through a lightweight convolutional layer and a sigmoid function to generate the final spatial attention weight map. This weight map dynamically enhances diagnostically relevant regions in the image, such as cardiac chamber structures, mitral valve anatomy, and regurgitation tract morphology. Through this mechanism, the model can accurately focus on local pathological features while maintaining perception of the overall cardiac structure, thereby significantly improving its ability to discriminate multi-scale visual patterns.
[0036] Building upon the temporal keyframe localization and spatial region focusing achieved by the KGDSAM module, this invention introduces the Video Swin Transformer Block as a spatiotemporal coding unit to construct a hierarchical deep spatiotemporal representation. This module employs a 3D Shifted Window Multi-Head Self-Attention mechanism. First, the input features are divided into multiple non-overlapping local 3D windows along the time, height, and width dimensions. Then, the self-attention computation is strictly confined to within each window, thereby reducing the computational complexity from a global quadratic to near linear.
[0037] To facilitate cross-window information interaction, the modules alternately use two window partitioning strategies when stacked consecutively: regular window partitioning and shifted window partitioning that translates in the three-dimensional spatiotemporal direction. Through cyclic shifting and masking techniques, shifted window partitioning achieves feature interaction and global context fusion between different local windows while maintaining computational efficiency.
[0038] This design enables the module to effectively model short-range spatiotemporal dependencies within local regions and gradually integrate them into a global spatiotemporal correlation through progressive cross-window information transmission. The resulting multi-level, highly discriminative spatiotemporal representation provides a key feature foundation for accurately distinguishing mitral regurgitation of different severity levels.
[0039] Furthermore, this invention proposes a Multi-view Classification Fusion Module (MCFM). This module can simulate the characteristics of ultrasound physicians integrating the features of the apical two-chamber, three-chamber, and four-chamber views, making full use of the different observation angles of different views on mitral regurgitation, and ultimately forming a patient-level classification result.
[0040] For a patient with N cross-sectional videos, each video is first feature-extracted based on a spatiotemporal encoder, and then processed through Global Average Pooling (GAP) and a Fully Connected Layer (FC) to generate probability distributions for four severity categories: ; in, Indicates the first i The predicted probability vectors of each slice video correspond to the categories of Normal, Mild, Moderate, and Severe mitral regurgitation, respectively; the fusion process is achieved by averaging the category probabilities of all slice videos. ; in, This represents the integrated probability vector; the classification result is determined by the maximum average probability.
[0041] This embodiment uses color Doppler echocardiography video data from two independent medical centers. The data includes 3188 videos from 767 patients. Of these, 727 videos are from 128 patients at Hospital A, and 2461 videos are from 639 patients at Hospital B. All data are authorized by the hospitals and are not used for any other purpose. All cases were categorized into four severity levels according to clinical guidelines: normal (349 cases), mild regurgitation (261 cases), moderate regurgitation (81 cases), and severe regurgitation (76 cases). Standard apical two-chamber, three-chamber, and four-chamber echocardiography videos were acquired for each patient, with each video containing at least three complete cardiac cycles.
[0042] To ensure rigorous model development and evaluation, this invention randomly divided the 639 cases from Hospital B into a training set (383 cases), a validation set (128 cases), and an internal test set (128 cases) in a 6:2:2 ratio. The 128 cases from Hospital A were used as an independent external test set to evaluate the model's generalization ability across different devices and patient populations.
[0043] All echocardiogram videos were uniformly adjusted to a spatial resolution of 224×224 pixels. A 16-frame segment was randomly sampled from each original long video as input, with a 2-frame time interval between adjacent frames. The model was trained end-to-end using the AdamW optimizer with an initial learning rate of 4e-4, a weight decay of 0.02, and a linear warm-up and cosine annealing strategy for learning rate scheduling.
[0044] Evaluation metrics are standards for measuring the generalization ability of a model. This embodiment uses four metrics to evaluate the model's performance: accuracy, precision, recall, and F1 score. Tables 1 and 2 show in detail the performance comparison of the model on the external test set of Hospital A and the internal test set of Hospital B, respectively. Figure 7 (a) and (b) further provide visualizations of the confusion matrices for the two test sets.
[0045] Table 1. Performance comparison of the external test set of Hospital A; Table 2. Performance comparison on the internal test set of Hospital B; On the internal test set, the proposed KGDSAM achieved state-of-the-art performance across all evaluation metrics, with an accuracy of 0.8682 and an F1 score of 0.8196, significantly outperforming the listed baseline models. Compared to the second-best performing TSM model (accuracy 0.8372, F1 score 0.7723), KGDSAM achieved relative improvements of 3.7% and 6.1% in accuracy and F1 score, respectively. This advantage validates the effectiveness of the keyframe-guided dual-space attention mechanism, where KFAM adaptively focuses on keyframes during the contraction phase, and the collaborative spatial dual-branch attention enhances the feature representation of discriminative regions, thus achieving superior classification performance with similar model parameter count and computational cost to the Video Swin Transformer. To evaluate the model's generalization performance, this invention was validated on an independent external test set. As shown in Table 2, due to differences in imaging equipment and patient population distribution across different medical centers, the performance of all models on this external test set showed a reasonable decline. In this cross-center evaluation, KGDSAM still maintained the best overall performance, with both its accuracy (0.8281) and F1 score (0.8106) being the best. This indicates that the features learned by the model are highly robust and can adapt to differences in data distribution among different centers.
[0046] comprehensive Figure 7 Confusion matrix analysis revealed that, on both test sets, the model achieved relatively high accuracy in identifying non-reflux and severe reflux categories, while exhibiting some confusion in classifying mild and moderate reflux. Specifically, a certain proportion of mild reflux samples were misclassified as normal, primarily because mild reflux appears subtle on ultrasound images, making it visually indistinguishable from normal samples. Moderate reflux samples were more easily misclassified as either mild or severe, as the characteristics of moderate samples often fall between mild and severe, making it difficult for both the AI model and specialist physicians to make a definitive distinction. This finding is consistent with existing research, indirectly confirming the alignment of its decision-making logic with real clinical challenges and enhancing the clinical rationality of its decision-making process.
[0047] In summary, both internal and external test results consistently demonstrate that the KGDSAN model achieves a significant and robust performance improvement in the mitral regurgitation video grading task. Furthermore, the error patterns revealed by the confusion matrix align with the actual challenges of clinical diagnosis, further enhancing the model's interpretability. Future work could further optimize the model's discriminative ability at inter-class boundaries and its clinical applicability by incorporating multi-center prospective data and frame-level weakly supervised labels.
[0048] The above is an example of echocardiography provided in this embodiment. Figure 2 Based on the same approach, this embodiment also provides a corresponding echocardiographic method for grading mitral regurgitation. Figure 2Triage system for mitral regurgitation, and information about echocardiography Figure 2 For specific limitations of the mitral regurgitation grading system, please refer to the section on echocardiography above. Figure 2 The limitations of the grading method for mitral regurgitation will not be elaborated upon here. The above echocardiography... Figure 2 The modules in a cusp regurgitation grading system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or they can be stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.
[0049] This embodiment also provides a computer-readable storage medium storing a computer program that can be used to execute the above-described... Figure 1 Echocardiography provided Figure 2 Terminological regurgitation grading method.
[0050] Those skilled in the art will understand that implementing all or part of the processes in the methods of the above embodiments can be accomplished by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.
[0051] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A method for grading mitral regurgitation by echocardiography, characterized in that, Includes the following steps: Acquire datasets of multi-sectional echocardiographic video sequences from two-chamber, three-chamber, and four-chamber chambers. A keyframe-guided dual-space attention network is constructed and trained on a dataset to obtain a keyframe-guided dual-space attention network for mitral regurgitation severity classification. The keyframe-guided dual-space attention network includes a feature initialization module, a spatiotemporal encoder, and a classification fusion module connected sequentially. The spatiotemporal encoder includes multiple consecutive processing layers, each including a keyframe-guided dual-space attention module and a Video Swin Transformer module, with spatial downsampling between layers achieved through a Patch Merging module. The keyframe-guided dual-space attention module includes a keyframe attention module, a color-guided attention module, a large selectivity kernel spatial attention module, and an adaptive attention fusion module. The feature initialization module decomposes multiple video sequences into spatiotemporally continuous image blocks and maps them into high-dimensional feature vectors. The keyframe attention module enhances the high-dimensional feature vectors with keyframe features during the contraction phase. The color-guided attention module divides the blood flow region into color candidate regions based on the binary mask obtained from the preprocessing of the video sequence data, determines the color prior mask, and generates an attention map focused on the blood flow region by spatial attention weighting of the enhanced features based on the color prior mask. The large selective kernel spatial attention module extracts the large receptive field branch features and local detail branch features of the enhanced features respectively, and weights and fuses them to obtain a spatial attention map. The adaptive attention fusion module weights and fuses the enhanced features, the blood flow region attention map, and the spatial attention map to obtain a spatial attention map. The VideoSwin Transformer module further spatiotemporally encodes the feature map fused from the spatial attention map and the feature map spatially downsampled by the upper-layer Patch Merging module to obtain spatiotemporal features. The classification fusion module weights and fuses the spatiotemporal features from multiple aspects and maps them to a classification result at the mitral regurgitation patient level.
2. The echocardiographic mitral regurgitation grading method according to claim 1, characterized in that, The feature initialization module includes a three-dimensional image block partitioning module and a linear embedding layer; the three-dimensional image block partitioning module decomposes multiple video sequences into spatiotemporally continuous image blocks; the linear embedding layer maps each image block into a high-dimensional feature vector.
3. The echocardiographic mitral regurgitation grading method according to claim 1, characterized in that, The keyframe attention module performs keyframe feature enhancement on the high-dimensional feature vector during the shrinkage period, including the following steps: For high-dimensional feature vectors , B For batch size, C For the number of channels, T For sequence length, H For height, W Width; The global spatial information of each frame is aggregated by three-dimensional global average pooling to generate frame-level feature vectors. : ； Establish local inter-frame dependencies using a lightweight convolutional layer. : ； The attention weights, ranging from 0 to 1, are output using the Sigmoid function. : ； In the formula, Use the Sigmoid activation function; Based on the weights learned during training, the original input features are weighted and fused through a broadcast mechanism to obtain the enhanced features of the keyframes during the contraction period.
4. The echocardiographic mitral regurgitation grading method according to claim 1, characterized in that, The color-guided attention module generates an attention map focused on the blood flow region, including the following steps: For each frame of the echocardiogram video sequence, the Euclidean distance between the RGB vector of each pixel and its corresponding gray value is calculated to quantify the color saturation, and initial segmentation is performed by thresholding to obtain color candidate regions; The color candidate regions are sequentially expanded and morphologically closed to connect neighboring regions, fill internal holes and smooth boundaries, and output a discriminative color prior mask. The color prior mask is multiplied element-wise with the input feature map to initially concentrate the feature response in the colored region. A lightweight sub-network is used to learn and generate spatial attention weights in this region. After activation by the Sigmoid function, the weights are multiplied again with the color prior mask to generate an attention map focused on the blood flow region, ensuring that the attention weights are strictly distributed within the prior colored region.
5. The echocardiographic mitral regurgitation grading method according to claim 1, characterized in that, The large selective kernel spatial attention module generates a spatial attention map, including the following steps: The large selective kernel spatial attention module includes a shared depthwise separable convolutional layer, two cascaded parallel processing branches, a feature concatenation module, parallel global average pooling layers, global max pooling layers, convolutional layers, feature fusion layers, and convolutional layers. The parallel processing branches include a large receptive field branch and a local detail branch. The large receptive field branch is connected to a depthwise separable convolutional layer and a pointwise convolutional layer in sequence. The local detail branch is connected to a pointwise convolutional layer. The basic spatial representation of the enhanced features is extracted through depthwise separable convolutional layers; The basic spatial representation of this feature is obtained by capturing contextual information at different scales through two parallel branches: a large receptive field branch and a local detail branch, to obtain the large receptive field branch feature and the local detail branch feature. The large receptive field branch features and local detail branch features are concatenated along the channel dimension and then passed through a global average pooling layer and a global max pooling layer respectively to aggregate the channel information of features at different scales and obtain the pooled features. The pooled features are processed through a convolutional layer and a sigmoid activation function to generate a set of adaptive selection weights for spatial dimensions. By adaptively selecting weights, spatially adaptive weighted fusion of large receptive field branch features and local detail branch features is performed to obtain fused features. The fused features are passed through a lightweight convolutional layer with a sigmoid activation function to generate the final spatial attention map.
6. The echocardiographic mitral regurgitation grading method according to claim 1, characterized in that, The adaptive attention fusion module weightedly fuses the blood flow region attention map and the spatial attention map to obtain a spatial attention map, including the following steps: Blood flow area attention map Spatial attention map Stitching along the channel dimension: ； Where [,] indicates channel splicing; Two independent sets of fusion weights are generated through an adaptive attention fusion module. and The adaptive attention fusion module is a lightweight 3D convolutional network, consisting of a 3D convolutional layer, a ReLU activation function, and a pointwise 3D convolutional layer. Attention map of blood flow region using fused weights Spatial attention map Perform weighted summation to generate a spatial attention map. : ； Through learnable scaling parameters Spatial attention is achieved through residual connections. Temporal features enhanced by keyframe attention module Perform fusion and output a spatial attention map: 。 7. The echocardiographic mitral regurgitation grading method according to claim 1, characterized in that, The classification fusion module weighted and fused the spatiotemporal features from multiple perspectives, mapping them to a classification result at the mitral regurgitation patient level, including the following steps: For a video containing N slices, each video undergoes spatiotemporal feature extraction via a spatiotemporal encoder. After global average pooling and a fully connected layer, probability distributions for four severity categories are generated: ； in, Indicates the first i The predicted probability vectors of each slice video correspond to the categories of Normal, Mild, Moderate, and Severe mitral regurgitation, respectively; the fusion process is achieved by averaging the category probabilities of all slice videos. ； in, This represents the integrated probability vector; the classification result is determined by the maximum average probability.
8. An echocardiographic mitral regurgitation grading system, characterized in that, The system includes: The dataset construction module is used to acquire multi-sectional echocardiographic video sequence datasets of two-chamber, three-chamber, and four-chamber systems. A network construction module is used to construct a keyframe-guided dual-space attention network. The keyframe-guided dual-space attention network includes a feature initialization module, a spatiotemporal encoder, and a classification fusion module connected sequentially. The spatiotemporal encoder includes multiple consecutive processing layers, each layer including a keyframe-guided dual-space attention module and a VideoSwin Transformer module, with spatial downsampling between layers achieved through a Patch Merging module. The keyframe-guided dual-space attention module includes a keyframe attention module, a color-guided attention module, a large selective kernel spatial attention module, and an adaptive attention fusion module. The network training module is used to train a keyframe-guided dual spatial attention network using a dataset. Specifically, the feature initialization module decomposes multiple video sequences into spatiotemporally continuous image blocks and maps them to high-dimensional feature vectors; the keyframe attention module enhances the high-dimensional feature vectors with keyframe features during the contraction phase; the color-guided attention module divides the blood flow region into color candidate regions based on a binary mask obtained from preprocessing the video sequence data, determines a color prior mask, and generates an attention map focused on the blood flow region by spatial attention weighting of the enhanced features based on the color prior mask; the large selective kernel spatial attention module extracts the large receptive field branch features and local detail branch features of the enhanced features, and weights and fuses them to obtain the spatial attention map; the adaptive attention fusion module weights and fuses the blood flow region attention map and the spatial attention map to obtain the spatial attention map; the Video Swin Transformer module performs spatiotemporal encoding on the feature map fused from the spatial attention map and the feature map spatially downsampled by the upper-layer Patch Merging module to obtain spatiotemporal features; and the classification fusion module weights and fuses the spatiotemporal features from multiple facets, mapping them to the patient classification results for mitral regurgitation levels.
9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a data processing program, which, when executed by a processor, implements the steps of the echocardiographic mitral regurgitation grading method as described in any one of claims 1 to 7.