A deep learning-based echocardiogram biplane left ventricular ejection fraction calculation method

Abstract

This invention discloses a deep learning-based method for calculating left ventricular ejection fraction (LVEF) in biplane echocardiography, belonging to the field of medical image processing and computer-aided diagnosis technology. The method includes: simultaneously acquiring ultrasound video streams from the apical four-chamber (A4C) and apical two-chamber (A2C) positions; geometrizing these streams into paired binary mask sequences using a deep segmentation network; determining and extracting a unified complete cardiac cycle based on biplane area curves, and performing synchronous temporal resampling standardization; decoupling the biplane geometric feature sequence using a temporal distribution spatial encoder; fusing the biplane features at each time step to construct a spatiotemporal feature matrix; and finally modeling the dynamic process using a temporal network and regressing to predict the ejection fraction. This invention, through decoupling, alignment, and deep fusion, fully utilizes biplane dynamic information to achieve highly accurate, robust, and interpretable fully automated ejection fraction calculation.

Description

Technical Field

[0001] This invention relates to the field of medical image processing and computer-aided diagnosis technology, specifically a method for calculating left ventricular ejection fraction in biplane echocardiography based on deep learning. Background Technology

[0002] Left ventricular ejection fraction (LVEF) is a core quantitative indicator for assessing cardiac pumping function, diagnosing heart failure, and guiding treatment. Currently, the most common clinical practice method involves an ultrasound physician manually selecting images from two orthogonal planes (A4C and A2C) at end-diastole and end-systole, tracing the endocardial contour, and then calculating the LVEF using the biplane Simpson method. This method has significant limitations: 1) It is highly subjective and inefficient, heavily reliant on operator experience, and has poor repeatability; 2) It does not fully utilize information, using only static information from two discrete time points, completely ignoring the dynamic temporal information of the entire cardiac contraction and relaxation process; 3) The model assumptions deviate from clinical reality. For patients with complex changes in ventricular morphology, such as myocardial infarction or dilated cardiomyopathy, the Simpson method calculation formula based on regular geometry will produce significant errors.

[0003] In recent years, automated methods based on deep learning have become a research hotspot. The mainstream approaches can be divided into two categories: The first is the "two-frame segmentation + formula calculation" approach, which uses a neural network to automatically segment the contours of end-diastole and end-systole and then inputs them into the Simpson formula. While this method achieves partial automation, it essentially does not transcend the framework of traditional methods, failing to utilize full-cycle information and being limited by geometric assumptions. The second is the "end-to-end" approach, which directly inputs the raw ultrasound video into a deep learning model to regress and calculate the LVEF value. Although this method can utilize temporal information, the model becomes an uninterpretable "black box" with unclear decision-making logic. Furthermore, due to directly processing high-dimensional video data, the computational complexity is high, and the model is prone to overfitting to image textures or device-specific noise in the training data that are unrelated to pathology, resulting in insufficient generalization ability and low clinical reliability.

[0004] More importantly, existing methods have significant shortcomings in how to deeply and collaboratively utilize dynamic information from two planes. They either only use a single cross section, losing the complementary and verification information from orthogonal perspectives, or the processing of the two planes is merely a formality, simply averaging the results at the end, failing to achieve deep alignment and collaborative fusion throughout the entire process from data preprocessing and feature extraction to dynamic modeling. Summary of the Invention

[0005] This invention aims to overcome the shortcomings of existing technologies and provide a fully automated method and system for calculating left ventricular ejection fraction (LVEF) based on dual-plane ultrasound video, through feature decoupling and adaptive temporal alignment. The core objective of this invention is to achieve more accurate, stable, and transparent automated calculation of LVEF by decoupling the original image data into purely spatial geometric and temporal dynamic information and standardizing the input using adaptive periodic alignment technology.

[0006] To achieve the above objectives, this invention proposes a deep learning-based method for calculating left ventricular ejection fraction in a two-plane echocardiogram, comprising the following steps:

[0007] S1: Acquire strictly synchronized A4C and A2C standard section ultrasound video streams within the same cardiac cycle of the subject.

[0008] S2: Utilize a pre-trained deep segmentation network to process the video streams of two slices in parallel, generating paired, detextured binary mask sequences.

[0009] S3: Calculate the left ventricular area-time curve based on the dual-plane mask sequence; automatically identify and extract a unified and complete cardiac cycle through joint analysis; and synchronously resample the dual-path data within this cycle to the standard length.

[0010] S4: By constructing a geometric feature encoding, the standardized A2C and A4C mask sequences are encoded frame by frame to obtain two low-dimensional geometric feature vector sequences.

[0011] S5: At each time step, the feature vectors from the two planes are concatenated to construct a fused two-plane spatiotemporal feature matrix.

[0012] S6: Input the spatiotemporal feature matrix into the time decoder for sequence modeling, and then output the final LVEF calculation value via regression.

[0013] The duration of the A2C and A4C ultrasound video streams obtained in step S1 must include at least one complete cardiac cycle to ensure that step S3 can identify and extract the complete segment from the end of systole to the next end of systole.

[0014] The image segmentation network in step S2 adopts the UNet3+ network architecture, which consists of three core parts: an encoding path, full-scale skip connections, and a decoding path. The encoding path extracts feature information from ultrasound images at different scales. A multi-scale fusion module achieves full complementarity between global semantic features and local detail features. Combined with a deep supervision mechanism, the training process is optimized. Finally, the decoding path accurately maps the target region, training a model suitable for ultrasound image segmentation tasks. The trained UNet3+ network is then used independently to perform semantic segmentation frame-by-frame on ultrasound video, accurately separating the cardiac anatomical structure from background noise and texture artifacts, obtaining a clean binary mask of the cardiac region.

[0015] In step S2, the UNet3+-based deep learning model employs an end-to-end convolutional neural network architecture of "encoder-fusion-decoder". This model consists of three key parts: the first part is the encoding module, composed of stacked convolutional blocks and downsampling operations, responsible for progressively reducing the dimensionality of the input ultrasound image while continuously extracting feature information from different levels, from shallow edge details to deep global semantics, to obtain more comprehensive ultrasound image features; the middle part is the multi-scale deep supervised fusion module, responsible for upsampling / downsampling all scale feature maps output by the encoding module, aligning their sizes, concatenating channels, integrating features through fusion convolutions, and reducing dimensionality, while setting multiple sets of auxiliary supervised branches to optimize the training gradient; the second part is the decoding module, composed of upsampling operations and decoding convolutional blocks, responsible for progressively increasing the dimensionality of the fused feature map, performing accurate spatial mapping based on the full-scale features extracted by the encoding module, and finally obtaining the target segmentation mask. The encoder, fusion module, and decoder are all composed of convolutional neural networks, where all parameters are continuously trained and iteratively optimized using the ultrasound image dataset until the expected segmentation accuracy is achieved, eliminating the need for manually designing feature extraction rules or segmentation thresholds.

[0016] The steps for automatic periodic identification and alignment of the ultrasound image in step S3 are as follows:

[0017] S31. Sum the pixels of the left ventricular region in each frame of the mask sequence and generate an area-time curve.

[0018] S32. Use the peak-finding algorithm to find the local maxima of the area-time curve. Theoretically, the maxima correspond to the end of diastole, and the minima correspond to the end of systole.

[0019] S33. Select the segment between two adjacent troughs as a complete cardiac cycle.

[0020] S34. If the number of frames in the extracted segment is not equal to the preset standard number of frames T, then normalization is performed using trilinear interpolation.

[0021] The geometric feature encoding in step S4 adopts a Siamese network architecture, that is, the encoder used to process the A2C mask sequence and the A4C mask sequence shares the same set of convolution kernel parameters to ensure that the geometric features under different cross-sections are measured in the same manifold space.

[0022] The full-cycle dynamic regression process in step S6 does not rely on the traditional volume integral formula, but instead establishes a direct mapping relationship between the rate of morphological change and cardiac pumping function by learning the trajectory characteristics of the geometric mask sequence over time.

[0023] Compared with the prior art, the present invention has the following significant advantages:

[0024] This invention effectively removes inherent noise and artifacts from ultrasound images through geometric feature encoding, allowing the model to focus on structural changes essential to cardiac function. A collaborative adaptive period alignment mechanism eliminates the influence of heart rate variability and ensures strict temporal synchronization of biplane data, providing clean and standardized input for subsequent modeling. Orthogonal deep fusion of biplane features enables the model to comprehensively utilize complementary information from two orthogonal perspectives, mutually verifying and supplementing each other, significantly improving the robustness of assessment for cases with poor single-section image quality or abnormal ventricular motion.

[0025] The masked sequences generated in the middle can be visually reviewed and corrected by clinicians. The relative separation of spatial features and temporal dynamics modeling allows us to interpret the decision-making basis of the model through visualization technology, thereby breaking down the barriers of end-to-end black box models and enhancing doctors' trust in automated results.

[0026] This invention overcomes the limitations of traditional methods that utilize four static frames and surpasses the shallow strategy of simply fusing dual-plane results. It achieves in-depth mining and synergistic utilization of dual-plane, full-cycle, and continuous spatiotemporal information, enabling cardiac function assessment to be based on a more comprehensive and dynamic physiological information foundation. Attached Figure Description

[0027] Figure 1 This is a flowchart illustrating the overall process architecture of the method of this invention.

[0028] Figure 2 This is a schematic diagram of ultrasound image segmentation.

[0029] Figure 3 This is a schematic diagram of automatic cardiac cycle recognition.

[0030] Figure 4 This is a schematic diagram of segmented mask sequence resampling.

[0031] Figure 5 This is a schematic diagram of a dual-plane geometric feature coding network structure.

[0032] Figure 6 This is a schematic diagram of the network structure for calculating ejection fraction using time-series regression. Detailed Implementation

[0033] This embodiment is a possible implementation based on Python. The invention will be described and explained in more detail below with reference to the accompanying drawings and embodiments.

[0034] 1. Figure 1 This is a flowchart illustrating an embodiment of a deep learning-based method for calculating left ventricular ejection fraction in echocardiography using a biplane approach. It includes modules for data acquisition, image segmentation, cardiac cycle recognition and alignment, geometric feature encoding, biplane feature fusion, and temporal dynamic regression.

[0035] 2. Figure 2 This is a schematic diagram of image segmentation, illustrating a UNet3+ network architecture. The encoder part includes 4 downsampling operations, and the decoder part includes 4 upsampling operations and skip connections. The input is a frame-by-frame raw video sequence with dimensions (3, 112, 112), and the output is a binary mask sequence, where a pixel value of 1 represents a left ventricular section, and 0 represents a background section other than the left ventricular section. The A4C and A2C video streams share the same pre-trained UNet3+ model weights, and this process transforms the original video stream into two binary mask sequences.

[0036] 3. Figure 3 This diagram illustrates the automatic cardiac cycle identification. Since raw ultrasound videos typically contain continuous heartbeats, this embodiment first performs automatic cardiac cycle identification to extract data with complete physiological significance. To prevent interference from ultrasound image noise or minor fluctuations, a minimum peak interval of 15 frames is set, and a peak-finding algorithm is used to detect local minima in the area-time series curve. Because the heart has its smallest volume at the end of systole, each local minima corresponds to a moment at the end of systole. The algorithm automatically selects two adjacent troughs and extracts the data from the mask sequence between these two troughs, thus forming a complete cardiac cycle segment.

[0037] 4. Figure 4 This is a schematic diagram of resampling the segmented mask sequence. Because different patients have different heart rates, the number of frames L contained in a single extracted cardiac cycle segment varies, making it unsuitable as direct batch input for a deep learning model. Therefore, temporal alignment and resampling are necessary. In this embodiment, the standard input frame number T=32. For the extracted cardiac cycle segment: if its frame length L>32, it indicates a slower heart rate, and downsampling using interval decimation or linear interpolation is performed to compress the sequence to 32 frames; if its frame length L<32, it indicates a faster heart rate, and upsampling using trilinear interpolation is performed to smoothly generate missing frames. Finally, this step aligns the variable-length mask data, and the dimensions are strictly normalized to (32, 112, 112).

[0038] 5. Figure 5 This is a schematic diagram of a dual-plane geometric feature encoding network structure. This embodiment uses a lightweight 4-layer convolutional neural network as a spatial encoder to compress high-dimensional mask images into low-dimensional geometric feature vectors. The network is specifically configured as follows: the first convolutional layer is followed by batch normalization and ReLU activation; and so on, with the second to fourth layers having 32, 64, and 64 channels respectively, and downsampling performed in each layer. After the last convolutional layer, a global average pooling layer and a flattening operation are connected, ultimately outputting a feature vector of dimension 64. When processing the aligned mask sequence, a temporal distribution strategy is adopted, that is, the tensor (32, 1, 112, 112) of temporal length 32 is unpacked along the time axis, allowing the CNN to process each mask independently, and finally merged back into a temporal feature matrix. The normalized mask sequences of A4C and A2C share the weights of this CNN network, ultimately outputting geometric feature vector sequences of size (32, 64) respectively.

[0039] 6. Figure 6 This diagram illustrates the network structure for calculating ejection fraction in time-series regression. In the fusion phase, the A4C and A2C geometric feature vectors at the same time step are directly concatenated along the feature channel dimension to construct a spatiotemporal feature matrix with dimensions (32, 128) that integrates biplane orthogonal information. Subsequently, in the time-series regression phase, this embodiment uses a stacked long short-term memory (LSTM) network to dynamically model the feature matrix. The LSTM network's hidden layer size is set to 128, and a dropout rate is set to prevent overfitting. The network progressively reads the fused feature sequence across 32 frames, memorizing the trajectory information of ventricular contraction and relaxation. Finally, the hidden state output of the last time step of the LSTM is extracted and sequentially passed through a fully connected layer with 32 neurons, an activation function, and a fully connected layer with only one output node. It is then activated by the Sigmoid function, outputting a value between (0, 1). Multiplying this value by 100 yields the final predicted left ventricular ejection fraction percentage.

[0040] Based on the deep learning method for calculating ejection fraction proposed in this invention, the implementation process of calculating ejection fraction in this invention is as follows:

[0041] S1. Use an echocardiogram device to acquire cardiac images. The patient is required to lie on their left side. Simultaneously acquire video streams of the apical four-chamber view and the apical two-chamber view. The frame rate is set to 60fps and the image resolution is 800×600 pixels.

[0042] S2. Adjust the resolution of the acquired apical four-chamber and apical two-chamber sections to 112×112 pixels, use UNet3+ with shared weights to process the two sections respectively, and output a binary mask.

[0043] S3. Calculate the area curve of the two planes, determine the starting point of the period by its average peak position, and resample synchronously to 32 frames.

[0044] S4. Construct a lightweight CNN containing four convolutional layers, each followed by batch normalization and activation and a global average pooling layer, ultimately outputting a 64-dimensional feature vector.

[0045] S5. Concatenate the feature channels to obtain a dual-plane spatiotemporal feature matrix with dimensions (32, 128), where the first 64 dimensions represent the geometric features of the apical two-chamber left ventricle and the last 64 dimensions represent the geometric features of the apical four-chamber left ventricle.

[0046] S6. Construct a two-layer LSTM network with 128 hidden units. Input a 32×128 fusion feature matrix. The LSTM reads the sequence step by step, memorizes the dynamic process of ventricular contraction, takes the hidden state at the last moment, passes through a fully connected layer and a Sigmoid activation function, and outputs the calculated LVEF value.

Claims

1. A deep learning-based method for calculating left ventricular ejection fraction in biplane echocardiography, characterized in that, Includes the following steps: S1. Acquire raw ultrasound video streams of the apical two-chamber (A2C) and apical four-chamber (A4C) sections of the subject within the same cardiac cycle; S2. Utilize a pre-trained image segmentation network to process the video streams of the two slices in parallel, generating paired, detextured binary mask sequences; S3. Calculate the waveform curve of the change of the left ventricular cross-sectional area over time based on the binary mask sequence, automatically extract the complete cardiac cycle segment by identifying the peak feature points of the curve, and use a time-series resampling algorithm to unify the extracted mask sequence to the standard frame length. S4. Construct a convolutional neural network as a geometric encoder, use a temporal distribution mode to process the standardized mask sequence, and map each frame of two-dimensional mask into a low-dimensional geometric feature vector, thereby converting the video data into a geometric feature sequence. S5. Concatenate the A2C geometric feature vector and the A4C geometric feature vector at the same time along the feature channel dimension to construct a biplane spatiotemporal feature matrix containing information about the orthogonal sectional plane of the heart; S6. Use a time-series dynamics model to perform dynamic modeling on the dual-plane spatiotemporal feature matrix, and calculate the ejection fraction by regression.

2. The method for calculating left ventricular ejection fraction in biplane echocardiography based on deep learning according to claim 1, characterized in that, The lengths of the A2C and A4C ultrasound video streams obtained in step S1 contain at least one complete cardiac cycle.

3. The method for calculating left ventricular ejection fraction in biplane echocardiography based on deep learning according to claim 1, characterized in that... The image segmentation network in step S2 is used to identify the contour of the left ventricle in the ultrasound image, remove image texture noise, gain differences and rib artifacts, and generate a mask sequence.

4. The method for calculating left ventricular ejection fraction based on deep learning in biplane echocardiography according to claim 1, characterized in that, The specific implementation of step S3 includes: S31. Calculate the total number of pixels in the left ventricular region of each frame in the mask sequence and generate an area-time curve; S32. Detect the peak (corresponding to end-diastole) and trough (corresponding to end-systole) of the curve; S33. Select the segment between two adjacent troughs as the effective cardiac cycle; S34. If the number of frames of the extracted segment is not equal to the preset standard number of frames N, then the trilinear interpolation method is used for normalization.

5. The method for calculating left ventricular ejection fraction in biplane echocardiography based on deep learning according to claim 1, characterized in that, The geometric encoder in step S4 adopts a Siamese network architecture, which shares the same set of convolutional kernel weights when processing the resampled A2C and A4C mask sequences, so as to ensure that geometric features are extracted in a unified metric space and reduce the number of model parameters.

6. The method for calculating left ventricular ejection fraction based on deep learning in biplane echocardiography according to claim 1, characterized in that, The dual-plane feature orthogonal fusion module achieves feature fusion by concatenating feature vectors from A2C and A4C along the feature channel dimension.

7. The method for calculating left ventricular ejection fraction in biplane echocardiography based on deep learning according to claim 1, characterized in that, The temporal regression network in step S6 employs one of the following: Long Short-Term Memory (LSTM) network, Gated Cyclic Unit (GRU) or Transformer encoder, to capture the rate of change of velocity and the synchronicity characteristics of ventricular wall motion during ventricular contraction.

Images