Three-dimensional convolutional neural network-based cardiac health assessment method and system

Abstract

Disclosed in the present invention are a three-dimensional convolutional neural network-based cardiac health assessment method and system. The method comprises: processing electrocardiogram signals to obtain key information of cardiac electrophysiological activity; identifying abnormal features of cardiac structure and function in a cardiac ultrasound image by means of a 2D convolutional neural network; acquiring spatial features and time-series data of each frame of image in a cardiac ultrasound video by means of a 3D convolutional neural network; and performing comprehensive cardiac health assessment according to the key information of cardiac electrophysiological activity, the abnormal features of cardiac structure and function, and the spatial features and time-series data of each frame of image. By employing the 3D CNN to capture temporal relationships between cardiac ultrasound video frames and taking into account the 2D CNN-extracted two-dimensional features of cardiac ultrasound images and the key information of electrocardiogram data, the present invention performs comprehensive cardiac health status assessment, which can significantly improve the accuracy and efficiency of cardiovascular disease diagnosis.

Description

A heart health assessment method and system based on three-dimensional convolutional neural network Technical Field

[0001] The embodiments of the present invention relate to the technical field of cardiac health assessment, and in particular to a cardiac health assessment method and system based on a three-dimensional convolutional neural network. Background Art

[0002] Heart disease is one of the major health problems worldwide, and its diagnosis and monitoring rely on accurate and effective medical imaging technology. Echocardiography and electrocardiography (ECG) are two key technologies for assessing heart health. Echocardiography provides a non-invasive way to visualize heart structure and function, including ventricular size, heart valve movement, and myocardial activity. On the other hand, ECG provides an important means of assessing cardiac electrical activity, which can be used to detect arrhythmias, myocardial infarction, and other signs of heart disease.

[0003] With the advancement of machine learning, deep learning networks are also being used to process cardiac ultrasound data. Existing technologies typically use two-dimensional convolutional neural networks for cardiac ultrasound image recognition. However, traditional 2D CNNs can only process single image frames and cannot effectively capture the dynamic characteristics of the heartbeat and other movements, making it difficult to accurately identify cardiac health conditions.

[0004] Summary of the Invention

[0005] The present invention provides a heart health assessment method and system based on a three-dimensional convolutional neural network, which accurately identifies heart health by combining a 3D convolutional neural network with a 2D convolutional neural network to analyze cardiac ultrasound images and electrocardiogram data.

[0006] In a first aspect, an embodiment of the present invention provides a heart health assessment method based on a three-dimensional convolutional neural network, comprising:

[0007] S1. Process the electrocardiogram signal to obtain key information about the cardiac electrophysiological activity;

[0008] S2. Identify abnormal features of cardiac structure and function in cardiac ultrasound images using a 2D convolutional neural network;

[0009] S3, obtain the spatial features and time series data of each frame image in the cardiac ultrasound video through 3D convolutional neural network;

[0010] S4. Perform a comprehensive assessment of cardiac health based on the key information of cardiac electrophysiological activity, abnormal characteristics of cardiac structure and function, and spatial characteristics and time series data of each frame of image.

[0011] In a second aspect, an embodiment of the present invention further provides a heart health assessment system based on a three-dimensional convolutional neural network, comprising:

[0012] An electrocardiogram signal processing module is used to process the electrocardiogram signal to obtain key information about the cardiac electrophysiological activity;

[0013] A cardiac ultrasound image processing module, used to identify abnormal features of cardiac structure and function in cardiac ultrasound images through a 2D convolutional neural network;

[0014] A cardiac ultrasound video processing module is used to obtain spatial features and time series data of each frame of cardiac ultrasound video through a 3D convolutional neural network;

[0015] The heart health assessment module is used to perform a comprehensive assessment of heart health based on key information of the cardiac electrophysiological activity, abnormal characteristics of cardiac structure and function, and spatial characteristics and time series data of each frame of image.

[0016] The present invention uses 3D CNN to capture the temporal relationship between cardiac ultrasound video frames, and combines the two-dimensional features of cardiac ultrasound images extracted by 2D CNN with key information from electrocardiogram data to assess cardiac health status. This can improve the accuracy and efficiency of heart disease diagnosis, reduce dependence on operator skills, and improve the objectivity and consistency of diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] FIG1 is a flowchart of a heart health assessment based on a three-dimensional convolutional neural network provided by an embodiment of the present invention;

[0018] FIG2 is a structural diagram of a 2D convolutional neural network provided by an embodiment of the present invention;

[0019] FIG3 is a structural diagram of a 3D convolutional neural network provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0020] The present invention will be further described in detail below with reference to the accompanying drawings and examples. It will be understood that the specific embodiments described herein are intended only to illustrate the present invention and are not intended to limit the present invention. It should also be noted that, for ease of description, the accompanying drawings only illustrate portions relevant to the present invention, not all structures.

[0021] Example

[0022] FIG1 is a flowchart of a heart health assessment based on a three-dimensional convolutional neural network according to an embodiment of the present invention, which specifically includes the following steps:

[0023] S1. Process the electrocardiogram signal to obtain key information about the cardiac electrophysiological activity.

[0024] Since ECG signals may be subject to various interferences, such as power supply noise, myoelectric interference, and respiratory movement, which may affect accurate signal interpretation, in this embodiment, to improve the accuracy of the ECG signal, the ECG signal is first denoised, then the R-wave peak is identified from the denoised ECG signal, and finally the heart rate is calculated by measuring the interval between R waves.

[0025] Specifically, when processing an ECG, electrocardiogram (ECG) data is first read from a file. This data is typically a digital recording of the heart's electrical activity. Baseline drift is then corrected by subtracting the signal's mean. This helps remove long-term, low-frequency fluctuations, making the ECG waveform clearer. Next, a bandpass filter is applied to remove high-frequency noise and low-frequency interference, preserving the main components of the ECG signal. This allows for clearer observation of ECG waveforms, such as the QRS complex. The ECG signal is then de-noised using the wavelet transform. The wavelet transform is an effective time-frequency analysis tool that can remove noise while preserving signal detail, which is crucial for improving the accuracy of ECG analysis. The final step in data processing is data normalization, which normalizes the signal to have zero mean and unit variance. This ensures consistency and accuracy in subsequent analysis. The interquartile range is also used to identify and address outliers, minimizing the impact of extreme fluctuations on the analysis.

[0026] S2. Identify abnormal features of cardiac structure and function in cardiac ultrasound images using 2D convolutional neural networks.

[0027] Cardiac ultrasound images are often interfered with by noise and artifacts, resulting in poor quality of the original images, which in turn affects the clarity and interpretability of the images. Therefore, in this embodiment, geometric transformation and histogram equalization are first performed on the cardiac ultrasound images.

[0028] In cardiac ultrasound image analysis, geometric transformations are used to adjust the orientation and position of images. For example, by slightly translating an image, different parts of the heart can be better aligned, ensuring that the model can effectively identify cardiac structures during training and testing.

[0029] Histogram equalization is a method for enhancing image contrast, particularly in cardiac ultrasound images, where it improves the visibility of cardiac structures. By converting the image to grayscale and then applying histogram equalization, the contrast between the cardiac region and surrounding tissue in the image is enhanced.

[0030] Geometric transformation ensures the consistency of input data in spatial direction, which helps the model learn more general features rather than being limited to specific image directions or positions, thereby improving data consistency. In addition, histogram equalization enhances image contrast, making the boundaries and internal structures of the heart clearer and enhancing image features, which is especially important for the model to identify subtle pathological changes. Through these preprocessing steps, the model is able to learn more representative features under a variety of different conditions and changes, improving the model's generalization ability on unseen data. Since ultrasound image quality is affected by the equipment and operator's technical level, the preprocessing steps help reduce the potential negative impact of these factors on model performance and also reduce the bias of the model. As a data augmentation technique, it increases the diversity of training data by creating slightly changed versions of the image, thereby preventing the model from overfitting.

[0031] Furthermore, the method of this embodiment specifically targets key areas of the heart: the four-chamber heart and the left ventricle long-axis view at the left bony border. These two views are crucial for diagnosing heart disease, particularly heart failure. By focusing on these specific areas, the model can more accurately identify changes associated with heart failure, such as abnormal wall motion, cardiomegaly, or decreased pumping function.

[0032] The 2D convolutional neural network model in this embodiment is shown in FIG2 , and specifically includes the following structure:

[0033] 1. First layer: convolutional layer ('Conv2D(32,(3,3),activation='relu',input_shape=(150,150,3))')

[0034] Input dimensions: '(150,150,3)', which is a color image of 150x150 pixels (3 color channels).

[0035] Output dimension: '(148,148,32)', due to the use of a 3x3 convolution kernel, the size is reduced by 2 in each direction.

[0036] Function: This layer uses 32 convolution kernels to extract image features. The ReLU activation function is used to increase nonlinearity.

[0037] 2. Second layer: Maximum pooling layer ('MaxPooling2D(2,2)')

[0038] Input dimensions: '(148,148,32)'.

[0039] Output dimension: '(74,74,32)', 2x2 pooling reduces the size by half.

[0040] Function: Reduce the spatial dimension of the feature map, reduce the amount of calculation, while retaining important features.

[0041] 3. The third layer: convolutional layer ('Conv2D(32,(3,3),activation='relu')')

[0042] Input dimensions: '(74,74,32)'.

[0043] Output dimensions: '(72,72,32)'.

[0044] Function: Further extract features using the same number of convolution kernels.

[0045] 4. Fourth layer: Maximum pooling layer ('MaxPooling2D(2,2)')

[0046] Input dimensions: '(72,72,32)'.

[0047] Output dimensions: '(36,36,32)'.

[0048] Function: Continue to reduce the spatial dimension of the feature map.

[0049] 5. Fifth layer: convolutional layer ('Conv2D(64,(3,3),activation='relu')')

[0050] Input dimensions: '(36,36,32)'.

[0051] Output dimensions: '(34,34,64)'.

[0052] Function: Use more convolution kernels (64) to extract more complex features.

[0053] 6. Sixth layer: Maximum pooling layer ('MaxPooling2D(2,2)')

[0054] Input dimensions: '(34,34,64)'.

[0055] Output dimensions: '(17,17,64)'.

[0056] Function: Further reduce the size of the feature map.

[0057] 7. Seventh layer: Flatten layer ('Flatten()')

[0058] Input dimensions: '(17,17,64)'.

[0059] Output dimension: '(18496)', which is the 3D feature map flattened into a 1D array.

[0060] Function: Prepare data for the fully connected layer.

[0061] 8. Eighth layer: fully connected layer ('Dense(1024, activation='relu')')

[0062] Input dimensions: '(18496)'.

[0063] Output dimension: '(1024)'.

[0064] Function: Learn high-level relationships between features.

[0065] 9. Ninth layer: output layer ('Dense(3, activation='softmax')')

[0066] Input dimension: '(1024)'.

[0067] Output dimension: '(3)', corresponding to 3 classification labels.

[0068] Function: Use the softmax activation function for multi-classification. Compared with the traditional convolutional neural network of 2DCNN, the softmax result is more optimized.

[0069] The model was compiled to use the Adam optimizer and the categorical_crossentropy loss function, primarily for multi-classification tasks. The entire model is designed to extract features from the input color image, progressively reduce the spatial dimensionality of the feature map through a series of convolutional and pooling layers, and finally perform classification through a fully connected layer. The resulting post-training accuracy and loss are comparable to those achieved by 3DCNN. It can be assumed that the convolutional neural network in this mode learns specific features to classify patients with left ventricular heart failure from those without obvious abnormalities.

[0070] S3. Obtain the spatial features and time series data of each frame image in the cardiac ultrasound video through a 3D convolutional neural network.

[0071] In this embodiment, the cardiac ultrasound video is provided in the DICOM medical-specific format. First, the cardiac ultrasound data is preprocessed:

[0072] 1. Use professional medical software to decode it into the MP4 standard format. 2. Then use tools such as imageJ to cut each frame of the image to remove noise, and take the first 40 frames of data to unify the video length. 3. After removing the noise, use python to adjust the scale to ensure the aspect ratio of each frame of the image. The initial value is 520*435, which can improve the accuracy of model training. 4. Then divide the obtained .MP4 video data into two categories. The first category is the four-chamber heart section data, and the second category is the left ventricular long axis section position data of the left bony edge area, and perform subsequent network training separately. At the same time, use the corresponding patient's Doppler tissue ultrasound and blood flow ultrasound as well as some information such as ejection fraction and shortening fraction to determine whether there are symptoms of heart failure. Each section data class is divided into whether there are symptoms of heart failure.

[0073] After preprocessing, geometric transformation and histogram equalization are performed on each frame of the image to improve the generalization ability of the model and improve the contrast of the image.

[0074] The 3D convolutional neural network in this embodiment includes the following components arranged in sequence:

[0075] Input layer, 3D convolution layer, 3D maximum pooling layer, Dropout layer, second 3D convolution layer, second 3D maximum pooling layer, second Dropout layer, flat layer, fully connected layer, third Dropout layer, output layer.

[0076] See Figure 3 for details. The 3D convolutional neural network specifically includes:

[0077] 1. Input layer ('input_shape=(40,174,208,1)'):

[0078] The input layer defines the shape of the data received by the network. Here it is a 40-frame video, each frame is 174x208 in size and is a single channel (grayscale image).

[0079] 2. First layer: 3D convolution layer ('Conv3D(32,kernel_size=(3,3,3),activation='relu')'):

[0080] This layer uses 32 convolution kernels of size 3x3x3 to slide over the input data to extract features.

[0081] Each convolution kernel applies the following operation: Y = WX + b, where W is the kernel weight, X is a fraction of the input data, and b is the bias term. Y represents the output. The ReLU activation function is applied to the convolution result to add nonlinearity.

[0082] The output dimension becomes (None, 38, 172, 206, 32), where None represents the batch size (BATCH SIZE).

[0083] 3. Second layer: 3D maximum pooling layer ('MaxPooling3D(pool_size=(2,2,2))'):

[0084] This layer downsamples the output of the previous layer to reduce the number of parameters and computation while retaining important features. The pooling window size is 2x2x2.

[0085] The output dimension becomes '(None,19,86,103,32)'.

[0086] 4. Third layer: Dropout layer ('Dropout(0.25)'): The Dropout layer randomly drops 25% of the activation units to prevent overfitting.

[0087] 5. Fourth layer: Second 3D convolutional layer ('Conv3D(64,kernel_size=(3,3,3),activation='relu')'): Similar to the first convolutional layer, but uses 64 convolution kernels to further extract features. The output dimensions become '(None,17,84,101,64)'.

[0088] 6. Fifth layer: Second 3D max pooling layer ('MaxPooling3D(pool_size=(2,2,2))'): Downsampling is performed again, and the output dimension becomes '(None,8,42,50,64)'.

[0089] 7. Sixth layer: Second Dropout layer ('Dropout(0.25)'): Dropout is applied again to prevent overfitting.

[0090] 8. Layer 7: Flatten ('Flatten()'): Flattens the 3D data to 1D for use in the fully connected layer. The output dimension becomes '(None, 1075200)'.

[0091] 9. Layer 8: Fully connected layer ('Dense(256, activation = 'relu')'): The fully connected layer uses 256 neurons to further process features and uses the ReLU activation function.

[0092] 10. Ninth layer: The third Dropout layer ('Dropout(0.5)'): Increase the Dropout ratio to 50% to further prevent overfitting.

[0093] 11. Layer 10: Output layer ('Dense(1, activation='sigmoid')'): The output layer uses the sigmoid activation function for binary classification. Compared with other classification activation functions, such as elu and softmax, it is found that sigmoid can better explain the results.

[0094] The 3D convolutional neural network in this example uses binary cross-entropy as the loss function, which is suitable for binary classification problems. The Adam optimizer is used, which is suitable for fast and efficient deep learning training. The Adam (Adaptive Moment Estimation) optimizer is a widely used deep learning optimization algorithm that combines the advantages of momentum and root mean square propagation (RMSprop). Compared with other optimizers such as adagrad and sgd, Adam's optimization training results are the best.

[0095] In addition, the Adam optimizer also includes the following advantages:

[0096] 1. Adaptive learning rate adapts to complex data characteristics: When processing cardiac ultrasound image data, due to the complexity and variability of images (for example, cardiac images from different patients can differ significantly), the Adam optimizer can more effectively handle this diversity by adjusting the learning rate for each parameter. 2. Excellent sparse gradient handling: Cardiac ultrasound data may contain regions with sparse gradients, which is particularly important when distinguishing important features from background noise. The Adam optimizer excels at handling these sparse signals, enabling more effective learning of important features. 3. Efficient large dataset processing: Given that cardiac ultrasound data can be very large, especially when including multiple video sequences, Adam's high computational efficiency and low memory requirements make it possible to handle large datasets. 4. Adaptability to non-stationary targets: Cardiac ultrasound images can exhibit some non-stationarity, such as variations between cardiac cycles. The Adam optimizer's self-adjusting capabilities help adapt to these variations, improving model accuracy. 5. Smooth gradient adjustment improves model stability: Model stability is particularly important in cardiac ultrasound image analysis. Adam's smooth parameter updates improve model stability during learning and reduce fluctuations during training. 6. Minor hyperparameter adjustments reduce the difficulty of tuning: In cardiac ultrasound image analysis projects, using an optimizer that doesn't require extensive hyperparameter tuning greatly simplifies model training. The default parameters of the Adam optimizer provide good performance in most cases, reducing the burden of tuning.

[0097] Building on the previous example, the 3D convolutional neural network in this example uses a custom test callback to monitor model performance in real time. Adding a test callback allows for real-time monitoring and recording of performance by evaluating the test set at the end of each training cycle. This callback allows real-time viewing of the model's performance on the test set, which is very useful for monitoring whether the model is overfitting or underfitting. This approach facilitates timely adjustments to training strategies, such as fine-tuning hyperparameters, to optimize model performance. The callback also records the performance of each epoch to a log file, facilitating subsequent analysis and visualization.

[0098] In summary, the 3D CNN in the embodiments of the present invention has the following advantages:

[0099] 1. 3D CNN uses multiple 3D convolutional layers (Conv3D), which can process data in both spatial and temporal dimensions. Each 3D convolution kernel moves not only in space (height and width) but also along the time axis, extracting features that change over time.

[0100] 2. Use 3D max pooling layer ('MaxPooling3D') to reduce the amount of computation and the number of parameters while retaining important temporal and spatial features, which helps reduce overfitting and improve the generalization ability of the model.

[0101] 3. By analyzing video frame sequences, 3D CNNs are far superior to traditional 2D methods in capturing temporal correlations. This enables the model to understand both continuity and variability in cardiac activity, such as changes in heart rate.

[0102] 4. Deeply mine complex patterns in cardiac ultrasound video data through a multi-layer 3D convolutional network. These deep features are crucial for understanding complex cardiac conditions.

[0103] 5. At the end of the network, the features extracted by the 3D convolutional layer are converted into the final prediction results through the flattening layer ('Flatten') and the dense layer ('Dense').

[0104] The 3D CNN in this embodiment is able to capture the temporal relationship between video frames and understand the continuity and variability in cardiac activity, such as changes in heart rate, making it particularly suitable for analyzing cardiac ultrasound videos. This temporal dimension analysis provides additional information for identifying cardiac dysfunction, especially in the case of heart failure, where dynamic changes and attenuation of the heart's pumping ability may be more obvious in consecutive frames. By combining time series analysis with attention to specific cardiac regions, this technology has the potential to significantly improve the diagnostic accuracy of heart failure. Especially in the early stages of heart failure, these subtle temporal and structural changes may be key indicators. Therefore, this technology may facilitate early detection and intervention, thereby improving patient treatment outcomes.

[0105] Furthermore, when processing high-dimensional video data, 3D CNN is able to effectively manage and extract key features, which is crucial for cardiac ultrasound video analysis. By optimizing the network architecture to handle large amounts of video data, not only does it improve the model's performance, but it also increases its usability in real-world clinical settings.

[0106] S4. Perform a comprehensive assessment of cardiac health based on the key information of cardiac electrophysiological activity, abnormal characteristics of cardiac structure and function, and spatial characteristics and time series data of each frame of image.

[0107] In this embodiment, each data source is independently used to generate a diagnostic decision or prediction result, and then these results are combined to obtain a more comprehensive diagnosis.

[0108] First, two-dimensional and three-dimensional imaging provides important visual evidence of cardiac structure and function, helping to detect structural problems such as ventricular hypertrophy, heart valve disease, and cardiomyopathy.

[0109] The electrocardiogram (ECG) provides key information about the electrical activity of the heart and is used to assess the risk of arrhythmia, myocardial ischemia, or infarction.

[0110] By combining these different types of data, a more comprehensive and accurate assessment of heart health can be provided. For example, the co-occurrence of structural abnormalities (found from imaging data) and electrophysiological abnormalities (found from ECG) may indicate a higher cardiac risk.

[0111] This embodiment combines 2D CNN, 3D CNN, and electrocardiogram signal processing technology to form a comprehensive and multi-dimensional heart health assessment system, which can achieve the following beneficial effects:

[0112] 1. Multi-dimensional Data Analysis: By combining ECG (time series data), 2D CNN (for processing static images), and 3D CNN (for processing dynamic videos), the system is able to analyze cardiac health from multiple perspectives. This multimodal analysis provides more comprehensive diagnostic information. 2. Enhanced Diagnostic Capabilities: ECG analysis provides deep insights into cardiac electrophysiological activity, while 2D CNN and 3D CNN processing ultrasound images provide structural and functional information. Combining these technologies enables more effective identification and assessment of cardiac diseases, particularly heart failure. 3. Data Complementarity and Fusion: Different types of data (ECG signals, 2D and 3D ultrasound images) complement each other. For example, certain cardiac abnormalities may be more apparent in an ECG, while structural changes may be more easily observed in ultrasound images. This data fusion enables more accurate and comprehensive disease identification. 4. Improved Model Generalization: Combining different types of data and analytical methods can improve the model's generalization across diverse clinical scenarios. This is particularly important for handling complex clinical cases. 5. Complete Heart Health Assessment: This comprehensive approach goes beyond identifying specific heart diseases to assess the overall health of the heart, including its structural integrity, functional status, and electrophysiological characteristics. 6. Advanced Diagnostic and Treatment Support Tool: By combining these three technologies, physicians gain a powerful diagnostic and treatment support tool. I have integrated these three functions into a single software, which not only improves diagnostic accuracy but also helps physicians make more comprehensive treatment decisions by providing more relevant information.

[0113] An embodiment of the present invention further provides a heart health assessment system based on a three-dimensional convolutional neural network, comprising:

[0114] An electrocardiogram signal processing module is used to process the electrocardiogram signal to obtain key information about the cardiac electrophysiological activity;

[0115] A cardiac ultrasound image processing module, used to identify abnormal features of cardiac structure and function in cardiac ultrasound images through a 2D convolutional neural network;

[0116] A cardiac ultrasound video processing module is used to obtain spatial features and time series data of each frame of cardiac ultrasound video through a 3D convolutional neural network;

[0117] The heart health assessment module is used to perform a comprehensive assessment of heart health based on key information of the cardiac electrophysiological activity, abnormal characteristics of cardiac structure and function, and spatial characteristics and time series data of each frame of image.

[0118] Among them, the electrocardiogram signal processing module is specifically used to: perform denoising on the electrocardiogram signal;

[0119] Identify the R wave peak from the denoised ECG signal;

[0120] Heart rate is calculated by measuring the intervals between R waves.

[0121] Optionally, the system further includes a preprocessing module for performing geometric transformation and histogram equalization processing on the cardiac ultrasound image.

[0122] The 3D convolutional neural network includes the following components:

[0123] Input layer, 3D convolution layer, 3D maximum pooling layer, Dropout layer, second 3D convolution layer, second 3D maximum pooling layer, second Dropout layer, flat layer, fully connected layer, third Dropout layer, output layer.

[0124] Use custom test callbacks to monitor test set performance during 3D convolutional neural network training.

[0125] Optionally, the 3D convolutional neural network uses a ReLU activation function.

[0126] The 3D convolutional neural network is trained using the Adam optimizer.

[0127] The output layer of the 3D convolutional neural network uses a sigmoid activation function for binary classification.

[0128] A heart health assessment system based on a three-dimensional convolutional neural network provided in an embodiment of the present invention can execute a heart health assessment method based on a three-dimensional convolutional neural network provided in any embodiment of the present invention, and has functional modules and beneficial effects corresponding to the execution method.

[0129] Note that the above are only preferred embodiments of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments, and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments and may include many other equivalent embodiments without departing from the concept of the present invention. The scope of the present invention is determined by the scope of the appended claims.

Claims

1. A heart health assessment method based on a three-dimensional convolutional neural network, characterized in that: include: S1. Process the electrocardiogram signal to obtain key information about the cardiac electrophysiological activity; S2. Identify abnormal features of cardiac structure and function in cardiac ultrasound images using a 2D convolutional neural network; S3, obtain the spatial features and time series data of each frame image in the cardiac ultrasound video through 3D convolutional neural network; S4. Perform a comprehensive assessment of cardiac health based on the key information of cardiac electrophysiological activity, abnormal characteristics of cardiac structure and function, and spatial characteristics and time series data of each frame of image.

2. The method according to claim 1, characterized in that Said S1 specifically includes: De-noising of ECG signals; Identify the R wave peak from the denoised ECG signal; Heart rate is calculated by measuring the intervals between R waves.

3. The method according to claim 1, characterized in that Before the S2, it also includes: Perform geometric transformation and histogram equalization on cardiac ultrasound images.

4. The method according to claim 1, wherein The 3D convolutional neural network includes the following steps: Input layer, 3D convolution layer, 3D maximum pooling layer, Dropout layer, second 3D convolution layer, second 3D maximum pooling layer, second Dropout layer, flat layer, fully connected layer, third Dropout layer, output layer.

5. The method according to claim 1, wherein Use custom test callbacks to monitor test set performance during 3D convolutional neural network training.

6. The method according to claim 1, characterized in that The 3D convolutional neural network uses the ReLU activation function.

7. The method according to claim 1, characterized in that The 3D convolutional neural network is trained using the Adam optimizer.

8. The method according to claim 1, characterized in that The output layer of the 3D convolutional neural network uses a sigmoid activation function for binary classification.

9. A heart health assessment system based on a three-dimensional convolutional neural network, characterized in that: include: An electrocardiogram signal processing module is used to process the electrocardiogram signal to obtain key information about the cardiac electrophysiological activity; A cardiac ultrasound image processing module, used to identify abnormal features of cardiac structure and function in cardiac ultrasound images through a 2D convolutional neural network; A cardiac ultrasound video processing module is used to obtain spatial features and time series data of each frame of cardiac ultrasound video through a 3D convolutional neural network; The cardiac health assessment module is used to assess the key information of cardiac electrophysiological activity, abnormalities of cardiac structure and function. Features, as well as spatial features and time series data of each frame of image are used to conduct a comprehensive assessment of heart health.

Images