A method and system for detecting cardiac rhythm abnormalities from echocardiographic videos

Abstract

The application provides a method and system for detecting cardiac rhythm abnormalities in echocardiographic videos, relating to the technical field of medical image processing. An echocardiographic video of a heart section is collected and preprocessed to obtain a continuous frame video segment. The video segment is input into a trained cardiac rhythm analysis model. Based on parallel convolution branches arranged in a pyramid, the spatiotemporal characteristics of local myocardial motion and deformation are dynamically captured, and fused spatiotemporal characteristics are obtained. Then, through a self-attention mechanism and a phase-guided attention mask, cross-frame motion correlations are established based on the fused spatiotemporal characteristics, and myocardial motion characteristics across cardiac cycles are obtained. Based on the myocardial motion characteristics, a cardiac rhythm abnormality detection result is obtained. The cardiac rhythm analysis model adopts a scalable architecture, which can configure different processing modes during the inference stage according to requirements. The application realizes efficient and accurate detection of cardiac rhythm abnormalities in echocardiographic videos.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and in particular to a method and system for detecting cardiac rhythm abnormalities in echocardiographic video. Background Technology

[0002] Echocardiography is a core clinical tool that uses ultrasound technology to non-invasively assess cardiac structure, motion, and hemodynamics. Its forms, including two-dimensional ultrasound video, can capture micro-spatial changes in ventricular wall motion in real time, providing a unique data foundation for the detection of rhythm abnormalities and has become a key support for the diagnosis of cardiovascular diseases.

[0003] Accurate detection of cardiac arrhythmias is crucial for early disease intervention, but current clinical and technological applications face significant bottlenecks, directly hindering efficient and accurate detection of cardiac arrhythmias in echocardiography. Existing solutions suffer from a number of prominent shortcomings: the mainstream-reliant electrocardiogram (ECG) is difficult to obtain in real-time during emergency transport, primary care, and home monitoring, and cannot simultaneously provide visual information about cardiac structure; various detection methods relying on electrical signals (including simplified ECG and photoplethysmography) are limited by their application scenarios; while some solutions are based on echocardiography, they only generate segmental strain curves through speckle tracking and compare them with templates, failing to fully utilize the original high-dimensional spatiotemporal information, thus limiting accuracy due to technological limitations; traditional ultrasound diagnosis relies heavily on physician experience, resulting in significant subjective differences, low diagnostic consistency, and a high rate of missed diagnoses; and existing equipment lacks intelligent recognition capabilities.

[0004] The aforementioned technical limitations prevent echocardiography from fully realizing its core advantages, and existing solutions cannot simultaneously address the issues of convenience, accuracy, and scenario adaptability in detection. To address the core problems of low efficiency and insufficient accuracy in detecting cardiac arrhythmias in echocardiographic video, there is an urgent need for an intelligent solution that can achieve accurate detection of arrhythmias without ECG assistance, relying solely on dynamic ultrasound images. Summary of the Invention

[0005] To address the aforementioned issues, this invention proposes a method and system for detecting cardiac rhythm abnormalities in echocardiogram videos. Through the collaborative design of a pyramid parallel convolutional branch and a self-attention mechanism, the spatiotemporal features and cross-frame correlations of myocardial motion are accurately captured. Combined with a scalable model architecture, this achieves efficient and accurate detection of cardiac rhythm abnormalities in echocardiogram videos.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: In a first aspect, the present invention provides a method for detecting cardiac rhythm abnormalities using echocardiography video, comprising: Echocardiographic videos of the heart section were acquired and preprocessed to obtain continuous frame video segments; The video clips are input into a trained cardiac rhythm analysis model. Based on parallel convolutional branches arranged in a pyramid, the model dynamically captures the spatiotemporal features of local myocardial motion and deformation, and fuses them to obtain fused spatiotemporal features. Then, through a self-attention mechanism and a phase-guided attention mask, cross-frame motion correlations are established based on the fused spatiotemporal features to obtain myocardial motion features across cardiac cycles. Based on the myocardial motion features, the results of cardiac rhythm abnormality detection are obtained. The cardiac rhythm analysis model adopts a scalable architecture, which can be configured with different processing modes according to needs during the inference phase.

[0007] In a second aspect, the present invention provides a cardiac rhythm abnormality detection system based on echocardiography video, comprising: The data processing module is configured to acquire echocardiographic videos of cardiac sections and perform preprocessing to obtain continuous frame video segments; An anomaly detection module is configured to input the video segment into a trained cardiac rhythm analysis model, dynamically capture the spatiotemporal features of local myocardial motion and deformation based on a pyramid-shaped arrangement of parallel convolutional branches, and fuse them to obtain fused spatiotemporal features; then, through a self-attention mechanism and a phase-guided attention mask, establish cross-frame motion correlations based on the fused spatiotemporal features to obtain myocardial motion features across cardiac cycles; and obtain cardiac rhythm anomaly detection results based on the myocardial motion features. The cardiac rhythm analysis model adopts a scalable architecture, which can be configured with different processing modes according to needs during the inference phase.

[0008] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method for detecting cardiac rhythm abnormalities in echocardiographic video as described in the first aspect.

[0009] Fourthly, the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method for detecting cardiac rhythm abnormalities in echocardiographic video as described in the first aspect.

[0010] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention inputs preprocessed consecutive video frames into a deep learning model. Pyramid-arranged parallel convolutional branches dynamically capture the spatiotemporal features of local myocardial motion and deformation in a hierarchical manner. Combined with a self-attention mechanism and phase-guided attention masking, it efficiently establishes cross-frame motion correlations, deeply mines myocardial motion features across cardiac cycles, significantly improves the accuracy of rhythm abnormality identification, and effectively avoids missed and false detections due to incomplete feature extraction. Furthermore, the model employs a scalable architecture, allowing for flexible configuration of processing modes during the inference phase based on actual detection scenarios and computing power, balancing detection efficiency with scenario adaptability, and providing an efficient and reliable auxiliary solution for clinical cardiac rhythm abnormality screening.

[0011] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0012] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute a limitation thereof.

[0013] Figure 1 This is a flowchart illustrating the main steps of a method for detecting cardiac rhythm abnormalities using echocardiography video, as provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of the unidirectional and bidirectional mode structural design provided in the embodiments of the present invention; Figure 3 This is a schematic diagram of the multi-faceted structural design provided in an embodiment of the present invention. Detailed Implementation

[0014] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0015] Example 1 like Figure 1 As shown in the figure, this embodiment discloses a method for detecting cardiac rhythm abnormalities in echocardiography video, including the following steps: S1: Acquire echocardiographic videos of the heart section and preprocess them to obtain continuous frame video segments; S2: The video segment is input into the trained cardiac rhythm analysis model. Based on the parallel convolution branches arranged in a pyramid, the spatiotemporal features of local myocardial motion and deformation are dynamically captured and fused to obtain fused spatiotemporal features. Then, through a self-attention mechanism and a phase-guided attention mask, cross-frame motion correlations are established based on the fused spatiotemporal features to obtain myocardial motion features across cardiac cycles. Based on the myocardial motion features, the cardiac rhythm abnormality detection results are obtained. The cardiac rhythm analysis model adopts a scalable architecture, which can be configured with different processing modes according to needs during the inference phase.

[0016] Next, combined Figure 1 This embodiment provides a detailed description of a method for detecting cardiac rhythm abnormalities using echocardiography video.

[0017] S1, Acquisition and preliminary processing of echocardiogram video.

[0018] First, dynamic video of relevant cardiac sections (such as apical four-chamber view, apical two-chamber view, apical three-chamber view, and parasternal short-axis view) is acquired using echocardiography equipment. The video stream output from the echocardiography equipment is then acquired by a video capture card, and continuous image frame sequences are extracted for subsequent processing.

[0019] As one implementation method, the hardware devices involved in this embodiment include an echocardiogram acquisition device (ultrasound host), a video capture card, an artificial intelligence computer, and a monitor. During the detection of cardiac rhythm abnormalities, an ultrasound physician uses the echocardiogram device to acquire relevant cardiac sections, and the video capture card acquires the video stream data displayed on the echocardiogram device screen.

[0020] The ultrasound host supports imaging of standard cardiac sections (apical four-chamber, two-chamber, three-chamber, and parasternal short axis, etc.) at ≥30fps, outputting uncompressed raw DICOM or HDMI video streams. The video capture card captures a 60Hz lossless stream with zero copy and a latency of <1ms, ensuring timing integrity.

[0021] Next, the acquired image frames undergo preprocessing. The core step is to locate and crop out a fan-shaped region containing only the cardiac structure. Since the original images contain non-cardiac interference content such as probe identification, device parameters, and patient information, cropping aims to eliminate background noise, focus on the analysis target, and simultaneously desensitize the data to protect patient privacy. Specifically, cardiac sector images are cropped based on the fan-shaped region.

[0022] For example, for multiple consecutive frames (e.g., the first 10 frames) of an ultrasound video stream, an object detection model is used to locate the bounding rectangle of the ultrasound sector. A mean calculation is performed based on the vertex coordinates of the sector rectangle to anchor the final clipping range. Clipping is performed along the sector boundary, removing non-cardiac areas such as probe text and rulers, allowing subsequent networks to focus only on the myocardial structure. The object detection model encompasses, but is not limited to, lightweight object detection architectures such as the YOLO series and NanoDet, enabling rapid inference on AI edge computers.

[0023] After cropping, bilinear interpolation is used to scale the cropped image proportionally to achieve the predetermined size, such as 224×224 or 448×448 pixels. To ensure that the image maintains its original aspect ratio and avoids the loss of key information during scaling, an edge-adaptive padding method is used, employing a black background (i.e., pixel value (0,0,0)) to fill the narrow borders of the image.

[0024] Divide the image pixel value by the maximum pixel value of 255 to normalize it to a 32-bit floating-point number in the range of 0 to 1.

[0025] The normalized video clips are dimensionally adjusted by changing their dimensional order to [number of channels, temporal length, spatial height, spatial width], so that they meet the input requirements for training deep learning models.

[0026] Furthermore, video segments are extracted from the preprocessed consecutive frames. This operation integrates multiple frame extraction strategies, including random extraction of consecutive frames and extraction of consecutive frames at intervals (the interval can be flexibly set to every 1, 2, or 3 frames). The final temporal length of the resulting video segments includes, but is not limited to, 8, 16, 24, or 32 frames.

[0027] When extracting video segments, the determination of the frame interval and segment length is primarily based on the frame rate acquired by the ultrasound equipment or acquisition card. The core principle is to ensure that the extracted video segment completely covers at least one cardiac cycle. Taking an echocardiogram video with a frame rate of 24 frames per second as an example, the following strategy can be adopted: extract 16 frames at one-frame intervals to form a video segment. The segment extracted in this way originates from 32 consecutive frames of images in the original echocardiogram video, with a duration of 1.33 seconds. Since the duration of a typical cardiac cycle is calculated as 1 second, this segment can completely cover at least one cardiac cycle, providing a reliable data foundation for subsequent analysis.

[0028] S2, input the preprocessed continuous frame echocardiogram image sequence into the trained deep learning model to extract temporal features that can characterize the rhythmicity of myocardial motion.

[0029] Cardiac arrhythmias can lead to disordered myocardial contraction rhythms and asynchronous movement. These abnormalities manifest as discontinuities, repetitions, or temporal misalignments in motion trajectories across consecutive frames. To directly and efficiently capture these abstract pathological patterns from raw pixels, this embodiment uses a deep learning model to extract high-dimensional temporal features of myocardial motion trajectory and structural changes. Although these features do not possess actual anatomical and physical significance, they may include spatial deformations of the heart structure (changes in ventricular wall thickness, valve opening and closing morphology, etc.) and temporal motion trajectories (displacement rates of myocardial segments, consistency of motion direction, etc.), which can assist in the accurate identification of bradycardia, tachycardia, and various arrhythmias.

[0030] Traditional feature engineering-based methods, such as optical flow, speckle tracking, or myocardial segmentation, rely on manually designed feature extraction rules, have high computational complexity, are sensitive to image quality (such as noise and artifacts), and are prone to losing subtle motion information.

[0031] To capture the aforementioned features directly and end-to-end, the deep learning model proposed in this embodiment analyzes the echocardiogram video clips. (Where C, T, H, and W represent the number of channels, temporal length, spatial height, and width, respectively) Temporal features are extracted, and dynamic deformation features of myocardial tissue and cardiac anatomy are automatically learned through three-dimensional convolution operations, thereby capturing spatiotemporal patterns that characterize cardiac rhythm abnormalities.

[0032] The deep learning model employs a multi-layered 3D convolutional neural network stacked structure. The convolutional kernel configuration maintains the principle of "temporal-spatial decoupling," i.e., the temporal dimension {1, 3, 5} and the spatial dimension {3, 5, 7}. After combination, various pyramid kernel groups are formed, which take into account both high temporal resolution and high spatial resolution to extract 3D feature maps. Among them, the three dimensions of the 3D convolutional kernel correspond to the [temporal length, spatial height, and spatial width] of the input image sequence, respectively.

[0033] For example, the pyramid kernel group can be {1×3×3,3×3×3,3×5×5,5×5×5,3×7×7}, where (1×3×3), (3×3×3), and (5×5×5) are classic pyramid combinations that can ensure a smooth growth of the spatiotemporal receptive field from local to global; (3×5×5) and (3×7×7) are combinations with a fixed temporal receptive field and an increased spatial receptive field. Both focus on motion patterns of moderate duration (temporal dimension=3), but the latter observes a larger spatial range within a frame and may be used to capture the overall deformation of the ventricular wall, while the former may focus on local valve motion.

[0034] By employing convolutional kernels that are decoupled in terms of time and space, a multi-scale pyramid combination is constructed to simultaneously capture fast local motion and slow global rhythmic patterns, achieving a balance between feature requirements and computational cost.

[0035] After extracting the 3D feature map, it is input into a lightweight Temporal Transformer, which uses a self-attention mechanism to capture myocardial motion features across the cardiac cycle.

[0036] (a) Temporally-spatial decoupled 3D convolutional kernel groups The first stage of a deep learning model aims to extract diverse local motion and deformation primitives. The fixed spatiotemporal receptive field of traditional 3D convolutions makes it difficult to simultaneously capture rapid tremors (such as high-frequency valve jitter caused by atrial fibrillation) and slow rhythm changes (such as bradycardia).

[0037] Therefore, this embodiment employs a set of parallel convolutional branches that are decoupled in terms of time and space and arranged in a pyramidal scale. Each branch uses a 3D convolutional kernel of a different size; for example, the kernel size of branch i is ( ).in, Control the temporal receptive field; the temporal kernel is used to capture motion patterns at different temporal resolutions. The spatial receptive field is controlled, while the spatial kernel focuses on structural deformation. The output feature maps of these branches... Attention gating mechanism through a learnable channel Adaptive fusion is performed to obtain fused spatiotemporal features. : ; in, The function is Sigmoid, and GAP is global average pooling. and For learnable parameters, This indicates channel-by-channel multiplication. This design allows the network to dynamically weight features at different spatiotemporal scales based on the input content. For example, it strengthens the contribution of short-time kernels for high-frequency tremors (t=1) and strengthens the contribution of long-time kernels for overall rhythm disorders (t=5).

[0038] (II) Introduction of Lightweight Temporal Transformer Building upon the initial features extracted by 3D convolution, a self-attention mechanism is used to establish cross-frame motion associations and capture the temporal dependencies of myocardial motion. This module can learn the long-range dependencies in the trajectory of myocardial motion, compensating for the limited receptive field of convolution operations.

[0039] Specifically, while the aforementioned convolutional operations can capture local motion patterns, their receptive field is limited, making it difficult to establish long-range dependencies across the entire cardiac cycle or even multiple cycles. The essence of rhythm abnormalities lies precisely in the disruption of this long-range temporal order. Therefore, this embodiment integrates spatiotemporal features... Based on this, a lightweight temporal self-attention module is introduced.

[0040] First, the spatial dimension is flattened to obtain the temporal feature sequence. Where D is the feature dimension. Then, the association weights between frames are calculated using a multi-head self-attention (MHSA) mechanism: ; Here, the query Q, key K, and value V all originate from the linear projection of Z.

[0041] Furthermore, to force the module to focus on diagnostically significant key motion phases (such as the onset of ventricular systole and the mitral valve opening apex), a phase-guided attention mask was designed. This mask is not fixed, but rather a probability vector predicted by a light quantum network based on input features, representing the probability that each of the T time points belongs to a key phase. generate: ; in, Indicates the first The probability that a given time point is predicted to be a critical motion phase (such as the onset of contraction). Indicates the first The probability that a given time point is predicted to be a critical motion phase. , This represents the index of a time point in the input sequence. It is a bias added to the attention weights to strengthen or weaken the first... Frame and the Inter-frame association; This is the attenuation coefficient.

[0042] The mask is used to teach the model to focus on truly important moments in clinical diagnosis. It automatically identifies which frames in a video sequence correspond to key cardiac motion events (such as the apex of valve opening and closing) through a sub-network and generates a probability distribution. Mask according to Construction: When two frames and When the probability of being identified as a critical phase is high, the attentional bias between them... The larger the time distance between the two frames, the more encouraged the model will be to strengthen the correlation between them. Simultaneously, the bias will decrease as the temporal distance between the two frames increases, which aligns with the characteristic that key events often exhibit localized clustering in time. This simulates the diagnostic logic of a doctor searching for and comparing key frames when reviewing images, allowing the network to focus on more discriminative temporal segments and improving its sensitivity to rhythm abnormalities.

[0043] This mechanism enables the model to prioritize time points identified as key phases when establishing inter-frame correlations, and encourages stronger connections between adjacent key phases. This simulates the diagnostic logic of clinicians searching for key frames for comparison, effectively improving the model's sensitivity to rhythm disturbances (such as premature beats and pauses).

[0044] After processing by the lightweight Temporal Transformer module described above, a feature representation rich in long-range temporal dependencies is obtained, namely, myocardial motion features across cardiac cycles. This feature integrates local motion primitives and global rhythm patterns, encoding the coordination and temporal regularity of cardiac motion within one or more complete cycles.

[0045] Based on myocardial motion characteristics across cardiac cycles, anomaly identification is performed using a classification head. This classification head maps the myocardial motion characteristics across cardiac cycles into a scalar probability value or a multi-dimensional vector, corresponding to the probability of different types of rhythm abnormalities. Finally, based on a preset threshold or argmax operation, the system outputs the detection result indicating whether the heart rhythm is abnormal, such as normal, bradycardia, tachycardia, atrial fibrillation, etc.

[0046] As one implementation method, the probability value of cardiac rhythm abnormalities is obtained by deep learning model inference performed by an artificial intelligence computer. The AI ​​edge computer is configured with a CPU of ≥8 cores at 3.2GHz; a GPU of NVIDIA RTX-A4000 or equivalent, 16GB GDDR6, and supports TensorRT INT8 quantization; and an inference time of <40ms.

[0047] Furthermore, to ensure that the features learned by the model can generalize to frame rate differences between different devices and heart rate variations among different patients, and remain stable in the presence of interference such as breathing and probe jitter, this embodiment employs a dual adversarial strategy during the training phase to improve model sensitivity and enhance the sensitivity, specificity, and robustness of the method.

[0048] 1) Enhanced Video Consistency Countermeasures To enrich the training data, this embodiment designs a data augmentation method based on video segments. Frame-synchronized transformation operations are performed on the input video segments to ensure that all frames within a segment receive the exact same spatial / photometric transformation parameters. Data transformations include: flipping, rotating, translating, scaling, cropping, affine transformations, as well as adjustments to brightness, contrast, and saturation, and the addition of noise and blur.

[0049] Overcoming the shortcomings of traditional single-frame enhancement that disrupts temporal continuity, this method strictly preserves the continuity of myocardial motion while expanding data diversity and suppressing motion artifact interference.

[0050] Specifically, an auxiliary temporal discriminator D is introduced, whose goal is to distinguish between real video segments and segments generated after the aforementioned data augmentation (especially temporal distortion). The main feature extraction network G is trained to "deceive" the discriminator, extracting features that make it as indistinguishable as possible from whether the input has been augmented. This forms a minimax game:

[0051] in, The enhanced video. Through this adversarial training, By being forced to learn those essential motion characteristics that remain unchanged in response to temporal perturbations, the robustness of the model in complex real-world environments is greatly improved.

[0052] 2) Dynamic timing sampling and perturbation A training sample set with different temporal resolutions is constructed by extracting consecutive frame sequences at random starting points and variable intervals (0, 1, 2, 3 frames).

[0053] Simulate the frame rate differences of clinical ultrasound equipment (30-120fps) to force the model to learn the invariance of the representation of heart rate abnormalities across variable time scales, thus solving the problem of misjudgment caused by fluctuations in equipment sampling rate.

[0054] Specifically, for the training videos, not only are the starting points randomly selected, but also at random intervals... Downsampling is performed to simulate frame rate changes. Furthermore, local temporal distortions are applied to the sampled sequence with a certain probability. A subsequence is randomly selected and slightly speed-changed (e.g., slowed down by 0.8x or sped up by 1.2x) or a few frames are inserted / deleted to synthesize richer abnormal rhythmic expressions. This forces the model to learn the relative order and form of motion patterns instead of relying on absolute inter-frame time differences.

[0055] Based on the core deep learning model mentioned above, this embodiment further designs a network architecture extension method that can be flexibly configured in the inference and application stages. By combining different time-series processing paths and aspect analysis paths, a variety of operation modes that balance performance and efficiency are formed.

[0056] This embodiment constructs a dual scalable architecture in terms of both temporal direction and aspect dimension. By combining the following two types of superimposed operation modes, the accuracy and robustness of rhythm anomaly detection are improved: 1) Unidirectional / Bidirectional Timing Processing Mode Basic configuration (unidirectional mode): The video segments generated by segment extraction maintain a forward temporal sequence (i.e., the frame sequence is arranged in forward order of time occurrence), and the sequence is processed by a single deep learning model to capture the unidirectional spatiotemporal evolution characteristics of myocardial motion.

[0057] Enhanced configuration (bidirectional mode): The original video frame sequence is reversed to generate a temporal mirror sequence, and the forward and reverse sequences are processed by two independent networks respectively; the bidirectional temporal analysis results are integrated through feature layer fusion mechanisms (such as weighted splicing or attention fusion); Technical advantages: The bidirectional mode can collaboratively capture the positive dynamics and negative pathological features of the myocardial systolic-diastolic cycle (such as the playback trajectory of valvular thrill).

[0058] Resource cost: The number of network parameters increases to twice that of the one-way mode, which may increase inference latency.

[0059] Structural diagram as follows Figure 2 As shown.

[0060] 2) Single / Multi-faceted Analysis Mode Basic configuration (single-section mode): Independently analyzes a single section of ultrasound image (such as the apical four-chamber view or the parasternal short-axis view) and outputs the probability value of rhythm abnormality for that section.

[0061] Enhanced configuration (multi-section mode): Simultaneously input multiple anatomical section images (e.g., apical four-chamber view + apical two-chamber view + apical three-chamber view parasternal short-axis view), with joint feature extraction by each individual neural network; cardiac rhythm abnormalities may be more pronounced on a particular echocardiogram section, so the abnormality probability value obtained from a single section may lead to misjudgment. Therefore, integrated analysis of multiple sections is expected to improve the accuracy and stability of judgment; the cost is an increase in the number of model parameters, which may increase inference time.

[0062] Technical advantages: Eliminates the limitations of a single-section view (such as the sensitivity of the apical three-chamber view to left ventricular outflow tract abnormalities); increases accuracy and robustness.

[0063] Structural diagram as follows Figure 3 As shown.

[0064] Furthermore, the settings mode is combined with an adaptive strategy.

[0065] In this embodiment, the unidirectional / bidirectional timing processing mode and the single / multi-faceted analysis mode are combined to form four basic operation modes, which are as follows: 1) Mode A: Unidirectional timing processing + single-face analysis; 2) Mode B: Unidirectional timing processing + multi-faceted analysis; 3) Mode C: Bidirectional timing processing + single-faceted analysis; 4) Mode D: Bidirectional timing processing + multi-faceted analysis; Among them, mode D (bidirectional + multi-faceted) showed the best detection accuracy and robustness in the early verification, and each mode has applicable scenarios.

[0066] 1. Specific implementation of pattern combination (1) Mode A: Unidirectional timing processing + single-face analysis This method employs a single deep learning model to process forward temporal video segments of a single cardiac profile (such as the apical four-chamber view) and outputs the probability of rhythm abnormalities in that profile. This approach has the lowest computational resource requirements and the fastest inference speed, making it suitable for deployment on edge devices. It is applicable to scenarios with high real-time requirements and limited computational resources, such as emergency screening, primary healthcare institutions, and home monitoring.

[0067] (2) Mode B: Unidirectional timing processing + multi-faceted analysis This approach simultaneously inputs multiple cardiac profiles (such as apical four-chamber, two-chamber, three-chamber, and parasternal short axis), each processed by an independent unidirectional temporal network. The outputs from each profile are then integrated through a feature fusion layer to generate a comprehensive anomaly probability. This method maintains high inference efficiency while enhancing detection stability through complementary perspectives from multiple profiles. It is suitable for scenarios requiring a high balance between accuracy and efficiency, such as routine outpatient screenings and health checkups.

[0068] (3) Mode C: Bidirectional timing processing + single-face analysis This model processes the same video segment using both forward and backward temporal networks for a single cardiac section. It integrates bidirectional temporal features through feature layer fusion (e.g., splicing followed by a fully connected layer) to output the probability of anomalies in that section. Inverse temporal analysis enhances the ability to capture subtle rhythmic anomalies. It is suitable for in-depth analysis of specific sections (e.g., the parasternal short axis) or when temporal compensation is needed for motion artifacts in a single viewpoint. It is applicable to routine outpatient screenings, health checkups, and other scenarios.

[0069] (4) Mode D: Bidirectional timing processing + multi-faceted analysis Each cardiac slice is processed using a bidirectional temporal network. The bidirectional features of each slice are fused through a cross-slice attention module, ultimately outputting a comprehensive abnormality probability. Specifically, let the number of slices be... The system contains Each network has one forward and one backward branch (one for each slice), and the contributions of each branch are dynamically aggregated through learnable weights. This model maximizes feature representation capabilities and significantly improves the identification accuracy of complex arrhythmias (such as atrial fibrillation with premature ventricular contractions). It is suitable for scenarios with extremely high requirements for detection sensitivity and specificity, such as high-precision diagnosis, hospitalized patients, and consultations for difficult cases.

[0070] 2. Adaptive Mode Selection Strategy To optimize resource utilization and adapt to diverse clinical needs, this embodiment introduces an adaptive mode selection mechanism based on multi-factor decision-making. The specific strategy is as follows: 1) Mode selection based on equipment resources When resources are sufficient (GPU memory ≥ 5GB), mode D is automatically enabled; when resources are limited (e.g., GPU memory ≤ 5GB), it degrades to mode B or mode A in sequence. In scenarios such as emergency rooms, where device resources fluctuate greatly, the system prioritizes real-time performance and uses mode A by default, switching to mode B only when resources allow.

[0071] 2) Mode switching based on anomaly type prediction The system uses a lightweight pre-analysis network to quickly evaluate video clips and initially identify the type of abnormality; when high-frequency tremor (suspected atrial fibrillation) is detected, it automatically switches to mode D.

[0072] 3) Model selection based on clinical scenarios Operating room real-time monitoring: Low latency is required; Mode A or B is preferred, with Mode D activated only for verification upon alarm triggering. Remote consultation and secondary diagnosis: Higher latency is permissible; Mode D is used by default, and visualized reports of bidirectional analysis across various aspects are provided. Epidemiological screening: Requires processing large amounts of data; Mode B is used for batch processing, with Mode D activated only for high-risk cases.

[0073] Each of the above operating modes (A, B, C, D) requires its corresponding architecture and independent end-to-end training on the training dataset to obtain the optimal model parameters for that mode. During training, the aforementioned dynamic temporal sampling and adversarial enhancement strategies can be combined to improve the model's robustness. In actual deployment, the system can select a strategy based on the adaptive mode, load and execute the trained model for the corresponding mode for inference.

[0074] In this embodiment, four configurable operating modes are formed by modularly combining unidirectional / bidirectional temporal processing with single / multi-faceted analysis. This architecture achieves a flexible trade-off between model performance and computational overhead: when resources are limited, the efficient "unidirectional + single-faceted" mode can be used to ensure real-time initial screening; when resources are sufficient, the high-precision "bidirectional + multi-faceted" mode can be enabled, deeply analyzing the spatiotemporal patterns of complex arrhythmias through inverse temporal compensation and multi-perspective feature fusion. This dual scalable architecture enables the same core model to adaptively cover diverse clinical scenarios, from basic screening to complex case consultations, significantly improving the method's practicality and clinical deployment potential.

[0075] This specific embodiment achieves automatic detection of cardiac arrhythmias solely based on echocardiography video without the need for simultaneous electrocardiogram (ECG), effectively overcoming the limitation of insufficient ECG device accessibility. The model employs temporal-spatial decoupled convolution and phase-aware attention mechanisms, directly capturing subtle spatiotemporal disturbances in myocardial motion from the video, significantly improving the accuracy and robustness in identifying bradycardia, tachycardia, and complex arrhythmias. Furthermore, the designed dual scalable architecture supports flexible deployment from low-resource edge devices to high-performance diagnostic terminals, adaptively selecting processing modes according to scenario requirements. While ensuring real-time performance, it meets the sensitivity and specificity requirements of different clinical auxiliary scenarios, providing an efficient and scalable intelligent auxiliary solution for early screening, real-time monitoring, and accurate diagnosis of cardiac arrhythmias.

[0076] Example 2 This embodiment provides a cardiac rhythm abnormality detection system based on echocardiography video, including: The data processing module is configured to acquire echocardiographic videos of cardiac sections and perform preprocessing to obtain continuous frame video segments; An anomaly detection module is configured to input the video segment into a trained cardiac rhythm analysis model, dynamically capture the spatiotemporal features of local myocardial motion and deformation based on a pyramid-shaped arrangement of parallel convolutional branches, and fuse them to obtain fused spatiotemporal features; then, through a self-attention mechanism and a phase-guided attention mask, establish cross-frame motion correlations based on the fused spatiotemporal features to obtain myocardial motion features across cardiac cycles; and obtain cardiac rhythm anomaly detection results based on the myocardial motion features. The cardiac rhythm analysis model adopts a scalable architecture, which can be configured with different processing modes according to needs during the inference phase.

[0077] Example 3 This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in the method for detecting cardiac rhythm abnormalities using echocardiography video as described in Embodiment 1 above.

[0078] Example 4 This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in the method for detecting cardiac rhythm abnormalities using echocardiography video as described in Embodiment 1 above.

[0079] The steps or modules involved in Embodiments 2 to 4 above correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0080] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for detecting cardiac rhythm abnormalities using echocardiography video, characterized in that, include: Echocardiographic videos of the heart section were acquired and preprocessed to obtain continuous frame video segments; The video clips are input into a trained cardiac rhythm analysis model. Based on parallel convolutional branches arranged in a pyramid, the model dynamically captures the spatiotemporal features of local myocardial motion and deformation, and fuses them to obtain fused spatiotemporal features. Then, through a self-attention mechanism and a phase-guided attention mask, cross-frame motion correlations are established based on the fused spatiotemporal features to obtain myocardial motion features across cardiac cycles. Based on the myocardial motion features, the results of cardiac rhythm abnormality detection are obtained. The cardiac rhythm analysis model adopts a scalable architecture, which can be configured with different processing modes according to needs during the inference phase.

2. The method for detecting cardiac rhythm abnormalities using echocardiography video as described in claim 1, characterized in that, The acquisition of echocardiographic videos of the heart section and the preprocessing to obtain continuous frame video segments specifically include: performing fan-shaped region detection and cropping on the acquired ultrasound videos, removing non-cardiac regions, and obtaining a video sequence containing only the heart structure. The sequence is normalized in size and pixel value, and continuous frame segments covering at least one complete cardiac cycle are extracted.

3. The method for detecting cardiac rhythm abnormalities using echocardiography video as described in claim 1, characterized in that, The method of dynamically capturing the spatiotemporal features of local myocardial motion and deformation based on parallel convolutional branches arranged in a pyramidal pattern includes: employing multiple parallel three-dimensional convolutional branches, each branch using convolutional kernels with different temporal and spatial dimensions, and adjusting the temporal and spatial kernel sizes to capture motion patterns at different time scales and spatial ranges.

4. The method for detecting cardiac rhythm abnormalities using echocardiography video as described in claim 1, characterized in that, The phase-guided attention mask is specifically described as follows: a dynamic mask is introduced into the self-attention mechanism. The mask is used to enhance attention to key phase frames of cardiac motion. A light quantum network is used to predict the probability that each time point belongs to the key phase, and the inter-frame attention weights are adjusted accordingly to make the model focus on motion events with diagnostic significance.

5. The method for detecting cardiac rhythm abnormalities using echocardiography video as described in claim 1, characterized in that, The cardiac rhythm analysis model adopts a scalable architecture, which can be configured with different processing modes according to needs during the inference phase, specifically including: In terms of timing processing, one can choose to perform only forward timing analysis or perform both forward and reverse timing analysis simultaneously; in terms of utilizing anatomical information, one can choose to analyze a single cross section or perform fusion analysis on multiple complementary cross sections. It offers four configurable operation modes: Mode A consists of forward time series analysis and single-section analysis; Mode B is a combination of forward time series analysis and multi-faceted fusion analysis; Mode C is a combination of bidirectional time series analysis and single-section analysis; Mode D is a combination of bidirectional time series analysis and multi-faceted fusion analysis; Based on computing resources, anomaly type prediction, or clinical scenario, the system adaptively selects and loads the corresponding trained model for inference.

6. The method for detecting cardiac rhythm abnormalities using echocardiography video as described in claim 1, characterized in that, The training phase of the cardiac rhythm analysis model adopts a dual adversarial strategy, including a video consistency adversarial enhancement strategy and a dynamic temporal sampling and perturbation strategy. The video consistency adversarial enhancement strategy specifically involves: during the training process, implementing frame-synchronized spatial and photometric transformations on the input video segments and introducing a temporal discriminator. Through adversarial training, the feature extraction network learns the essential motion features that remain unchanged in the face of temporal perturbations.

7. The method for detecting cardiac rhythm abnormalities using echocardiography video as described in claim 6, characterized in that, The dynamic temporal sampling and perturbation strategy is as follows: when constructing training samples, video frames are extracted at variable intervals to simulate different device frame rates, and local temporal distortions are applied to the segments with a certain probability, forcing the model to learn rhythmic representations based on motion patterns and improving the generalization ability to heart rate variability and abnormal rhythms.

8. A cardiac rhythm abnormality detection system based on echocardiography video, characterized in that, include: The data processing module is configured to acquire echocardiographic videos of cardiac sections and perform preprocessing to obtain continuous frame video segments; An anomaly detection module is configured to input the video segment into a trained cardiac rhythm analysis model, dynamically capture the spatiotemporal features of local myocardial motion and deformation based on a pyramid-shaped arrangement of parallel convolutional branches, and fuse them to obtain fused spatiotemporal features; then, through a self-attention mechanism and a phase-guided attention mask, establish cross-frame motion correlations based on the fused spatiotemporal features to obtain myocardial motion features across cardiac cycles; and obtain cardiac rhythm anomaly detection results based on the myocardial motion features. The cardiac rhythm analysis model adopts a scalable architecture, which can be configured with different processing modes according to needs during the inference phase.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in the method for detecting cardiac rhythm abnormalities in echocardiographic video as described in any one of claims 1-7.

10. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the method for detecting cardiac rhythm abnormalities in echocardiographic video as described in any one of claims 1-7.

Images