Classification method of multimodal echocardiogram multi cross section based on deep learning algorithm

Abstract

To provide a multimodal multi cross section classification method of an echocardiogram based on a deep learning algorithm.SOLUTION: A multimodal multi cross section classification method of an echocardiogram based on a deep learning algorithm includes: collecting a moving image and an image for adult echocardiogram and preprocessing them; labeling a moving image and an image of the adult echocardiogram preprocessed to generate a data set of the adult echocardiogram; and classifying the data set of the adult echocardiogram into a training set, a verification set and a test set. On the basis of a ResNet network, an adult echocardiogram cross section classification model is established and trained according to the training set, an optimal classification model is screened by a verification set, performance evaluation is performed with respect to an optimal adult echocardiogram cross section classification model on the basis of the test set, and an image or moving image to measure is inputted to the adult echocardiogram cross section to obtain a classification result.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to the field of echocardiogram classification, in particular based on deep learning algorithms A method for classifying multiple planes in multimodal echocardiograms Regarding. [Background technology]

[0002] Echocardiography has the advantages of non-invasiveness, non-radiation, high time resolution, bedside operation, etc., and exerts important clinical value in the diagnosis and treatment of cardiovascular diseases. Current echocardiography includes multiple modalities such as M-shape, two-dimensional grayscale, color Doppler, and acoustic contrast. Multimodal ultrasound images provide abundant cardiac anatomical structure information, such as atrial and ventricular positions, sizes, and hemodynamic information. Accurate recognition of echocardiographic reference planes is important to analyze cardiac function for quantification and post-processing, but post-processing analysis software, such as QLab and EchoPAC, rely on standard apical approach plane images in artificial screening to perform secondary analysis of cardiac structure and function, and preliminary classification of planes is time-consuming, especially for big sample data. Therefore, the need for rapid and accurate automatic identification of echocardiographic reference planes is gradually increasing.

[0003] In the prior art, the methods for recognizing echocardiographic reference slices are mainly for 2D grayscale images, and there are few studies on 3D contrast and color Doppler ultrasound images. Therefore, a method based on deep learning algorithms A method for classifying multiple planes in multimodal echocardiograms It is necessary to design. Summary of the Invention

[0004] The object of the present invention is to develop a deep learning algorithm based A method for classifying multiple planes in multimodal echocardiograms Provide Multimodal echocardiography It is possible to realize automatic classification of multiple cross sections, which has high classification accuracy and saves time and effort.

[0005] In order to achieve the above object, the present invention provides the following means.

[0006] Based on deep learning algorithms A method for classifying multiple planes in multimodal echocardiograms comprising the steps of:

[0007] Step 1: Multi-section We collected videos and images of adult echocardiograms.

[0008] Step 2: Preprocess the collected adult echocardiogram videos and images.

[0009] Step 3: Label the preprocessed adult echocardiogram videos and images to generate an adult echocardiogram dataset;

[0010] Step 4: Divide the adult echocardiogram dataset into a training set, a validation set, and a test set.

[0011] Step 5: Based on the ResNet network, build an adult echocardiogram cross-sectional classification model, train it by the training set, and screen the optimal classification model by the validation set;

[0012] Step 6: Based on the test set, perform performance evaluation against the optimal adult echocardiogram cross-sectional classification model.

[0013] Step 7: The measurement target image or video is input into the adult echocardiogram cross-sectional classification model to obtain the classification result.

[0014] Optionally, in step 1, Multi-section We collected videos and images of adult echocardiograms, specifically:

[0015] Multi-sectionand collects adult echocardiogram videos and images, including 3D grayscale echocardiogram, color Doppler echocardiogram, 2D grayscale parasternal left ventricular long axis view, 2D grayscale parasternal left ventricular short axis view, 2D grayscale parasternal aortic short axis view, 2D grayscale subxiphoid view, 2D grayscale apical two chamber view, 2D grayscale apical three chamber view, 2D grayscale apical four chamber view, 2D left echocardiogram apical two chamber view, 2D left echocardiogram apical three chamber view, 2D left echocardiogram apical four chamber view, and 2D left echocardiogram parasternal left ventricular short axis view.

[0016] Optionally, in step 2, preprocessing is performed on the collected adult echocardiogram videos and images, specifically,

[0017] The third-party Python library OpenCV is used to batch process the adult echocardiogram videos and images, and a sector-shaped region of interest is extracted based on the pixel changes of consecutive frame images in conjunction with image preprocessing operations, and the video is saved in PNG format for each frame.

[0018] Optionally, in step 4, the adult echocardiogram dataset is divided into a training set, a validation set, and a test set, specifically:

[0019] The adult echocardiogram dataset is divided into a training set, a validation set, and a test set in a ratio of 8:1:1, where the cross-sectional data amount with the smallest sample amount is used as the basis to sample the cross-sectional datasets of other categories in equal proportions to construct a dataset with equal sample amounts, so that the cross-sectional sample amounts of each category are equal, where the training set is used to train the adult echocardiogram cross-sectional classification model, the validation set is used to adjust the model hyperparameters and screen the optimal classification model, and the test set is used to evaluate the classification performance of the model.

[0020] Optionally, in step 5, build an adult echocardiogram cross-sectional classification model based on the ResNet network, train it by the training set, and screen the optimal classification model by the validation set, specifically,

[0021] Based on a 101-layer ResNet network, an adult echocardiogram cross-section classification model is constructed and trained by a training set. A transfer learning strategy is adopted in the training phase, the pre-training weights adopt the results trained on the ImageNet dataset, the output layer classifier is a Softmax function, the number of classification categories is set to 13 categories, an Adam optimizer with an initial learning rate of 0.0001 is used, and the batch processing size is 128. The model is fine-tuned for 100 iterations, and the ultrasound image features are extracted by a convolutional neural network with residual structure. After training is completed, the model performance is evaluated on the validation set by minimizing the cross-entropy loss value between the true label and the predicted result, and the model weights with the highest classification accuracy rate are selected as the optimal adult echocardiogram cross-section classification model.

[0022] Optionally, in step 6, a performance evaluation is performed on the optimal adult echocardiogram cross-sectional classification model based on the test set, specifically,

[0023] The performance of the optimal adult echocardiogram cross-sectional classification model is evaluated on the validation and test sets using confusion matrix, accuracy, precision, recall, specificity and F1 score.

[0024] Optionally, in step 7, the measurement target image is input into an adult echocardiogram slice classification model to obtain a classification result, specifically,

[0025] The measurement target image is input into the adult echocardiogram cross-sectional classification model, where for the adult echocardiogram video, the classification result of each image is predicted by the adult echocardiogram cross-sectional classification model, for the adult echocardiogram video, 10 frame images are sampled at equal intervals in each video to make a prediction, the average value is taken for the predicted results, and the cross-sectional category corresponding to the maximum predicted probability is used as the classification result of the video, and a gradient-weighted activation graph visualization analysis method is used to generate a heat map using the feature weights of the last layer of the adult echocardiogram cross-sectional classification model, and the visualized cross-sectional classification model focuses on the key attention areas, and performs an interpretable analysis on the classification results.

[0026] According to the specific embodiments provided by the present invention, the present invention provides the following technical effects: Based on the deep learning algorithm provided in the present invention A method for classifying multiple planes in multimodal echocardiograms And, Multi-sectionThe method includes the steps of collecting adult echocardiogram videos and images, performing pre-processing on the collected adult echocardiogram videos and images, labeling the pre-processed adult echocardiogram videos and images, generating an adult echocardiogram dataset, dividing the adult echocardiogram dataset into a training set, a validation set and a test set, constructing an adult echocardiogram cross-section classification model based on a ResNet network, training it with the training set, screening an optimal classification model with the validation set, performing performance evaluation on the optimal adult echocardiogram cross-section classification model based on the test set, inputting the measurement target image or video into the adult echocardiogram cross-section classification model to obtain a classification result, wherein the method performs pre-processing on the adult echocardiogram videos and images. The method performs labeling on the data of echocardiograms, extracts regions of interest, and hides the patient's examination and personal information; the method can be applied to multi-modal (2D grayscale, Doppler, 3D, and acoustic contrast), multi-section (mainly for the sections commonly used in 2D grayscale clinical examinations) echocardiogram data; the method is labeled by ultrasound doctors with extensive clinical experience, and can ensure the accuracy of the labels; the method randomly divides the datasets proportionally, and can ensure that the data diversity and sample amount are equal; the method evaluates the model prediction performance on independent datasets using multiple classification indicators, and can ensure the robustness of the model prediction results; the method employs anatomical regions of focus in the class activation heatmap visualization model classification process to improve the interpretability of the model. [Brief description of the drawings]

[0027] In order to more clearly describe the embodiments of the present invention or the technical solutions of the prior art, the drawings that need to be used in the description of the embodiments will be briefly described below. Obviously, the drawings in the following description are only some embodiments of the present invention, and those skilled in the art can also obtain other drawings based on these drawings without paying creative labor. [Figure 1]1 is a flowchart of a multi-plane classification method for multi-modal echocardiograms based on a deep learning algorithm according to an embodiment of the present invention. [Figure 2a] This is a visualization analysis result diagram for cross-sectional categories 2D_2C, 2D_3C, 2D_4C, and 2D_BX. [Figure 2b] This is a visualization analysis result diagram for cross-sectional categories 2D_P, 2D_SAX, 2D_SAoA, 3D, C2D_2C, and C2D_3C. [Figure 2c] This is a visualization analysis result diagram for cross-sectional categories C2D_4C, C2D_SAX, and CDFI. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0028] The object of the present invention is to develop a deep learning algorithm based A method for classifying multiple planes in multimodal echocardiograms Provide Multimodal echocardiography It is possible to realize automatic classification of multiple cross sections, which has high classification accuracy and saves time and effort.

[0029] In order to make the above objects, features and advantages of the present invention more apparent, the present invention will be described in more detail below with reference to the drawings and specific embodiments.

[0030] As shown in FIG. 1, based on the deep learning algorithm provided in the embodiment of the present invention, A method for classifying multiple planes in multimodal echocardiograms comprising the steps of:

[0031] Step 1: Multi-section Step 2: Collect adult echocardiogram videos and images, and perform preprocessing on the collected adult echocardiogram videos and images.

[0032] Step 3: Label the preprocessed adult echocardiogram videos and images to generate an adult echocardiogram dataset;

[0033] Step 4: Divide the adult echocardiogram dataset into a training set, a validation set, and a test set.

[0034] Step 5: Based on the ResNet network, build an adult echocardiogram cross-sectional classification model, train it by the training set, and screen the optimal classification model by the validation set;

[0035] Step 6: Based on the test set, perform performance evaluation against the optimal adult echocardiogram cross-sectional classification model.

[0036] Step 7: The measurement target image or video is input into the adult echocardiogram cross-sectional classification model to obtain the classification result.

[0037] In selection step 1, Multi-section We collected videos and images of adult echocardiograms, specifically:

[0038] Multi-section and collects adult echocardiogram videos and images, including 3D grayscale echocardiogram, color Doppler echocardiogram, 2D grayscale parasternal left ventricular long axis view, 2D grayscale parasternal left ventricular short axis view, 2D grayscale parasternal aortic short axis view, 2D grayscale subxiphoid view, 2D grayscale apical two chamber view, 2D grayscale apical three chamber view, 2D grayscale apical four chamber view, 2D left echocardiogram apical two chamber view, 2D left echocardiogram apical three chamber view, 2D left echocardiogram apical four chamber view, and 2D left echocardiogram parasternal left ventricular short axis view.

[0039] In selection step 2, preprocessing is performed on the collected adult echocardiogram videos and images, specifically,

[0040] Ultrasound video data is stored in DICOM format and usually contains redundant information such as examination information (examination date and time, electrocardiogram, heart rate, frame frequency) and ultrasound device parameters, which may reduce the image signal-to-noise ratio and affect the prediction performance of the AI ​​model. Third-party Python libraries such as Pydicom and OpenCV are used to batch process adult echocardiogram videos and images, and based on the pixel changes in consecutive frame images, combine image preprocessing operations such as expansion and erosion to extract fan-shaped regions of interest, and save the videos in PNG format for each frame. For subsequent training cross-sectional classification models, the Pydicom library is used to process the original DICOM files and remove sensitive information such as the patient's name, and the OpenCV library is used to extract the region of interest (fan-shaped region) in the echocardiogram video and remove surrounding useless information (e.g., the patient's age, name, machine model number, etc.).

[0041] In step 3, the preprocessed adult echocardiogram videos and images are labeled to generate an adult echocardiogram dataset, specifically:

[0042] The labeling work of the cross-sectional data sets is completed by ultrasound physicians with rich clinical experience to ensure the authenticity and reliability of the data labels.

[0043] In step 4, the adult echocardiogram dataset is divided into a training set, a validation set, and a test set. Specifically,

[0044] The adult echocardiogram dataset is divided into a training set, a validation set, and a test set in a ratio of 8:1:1, where the cross-sectional data amount with the smallest sample amount is used as a reference, and the cross-sectional datasets of other categories are sampled in equal proportions to construct a dataset with equal sample amounts, whereby a dataset with equal cross-sectional sample amounts of each category is constructed. Here, the training set is used to train an adult echocardiogram cross-sectional classification model, the validation set is used to adjust the model hyperparameters and screen the optimal classification model, and the test set is used to evaluate the classification performance of the model.

[0045] In step 5, an adult echocardiogram cross-sectional classification model is constructed based on the ResNet network, trained by the training set, and screened for the optimal classification model by the validation set. Specifically,

[0046] Convolutional neural networks have strong feature representation capabilities, and are usually composed of a combination of convolution kernels, activation functions, pooling layers and fully connected layers. The convolution operation can extract features of different hierarchical levels of the input image. The activation function improves the feature identification ability by increasing the nonlinear relationship between the network layers. The pooling layer is space and scale invariant, which reduces the feature dimension and retains the main features of the image. The fully connected layer maps the extracted features to the classification sample space. However, as the number of network layers becomes deeper, it is easy to stack convolutional layers, which will cause the gradient vanishing or explosion problem, and reduce the model performance. The deep residual network ResNet is one of the most innovative convolutional neural networks in the field of computer vision. It uses a residual learning reconstruction model to perform fitting on the residual map, solving the problem of "degeneration" that exists in the backpropagation process of the deep network, greatly increasing the network depth and effectively improving the network performance.

[0047] The basic idea of ​​ResNet is that the input is x, and H(x) represents the learned features, i.e., the features extracted by a nonlinear layer stacked with several convolutional layers. The other branch is a linear layer, which shows the identity mapping or projection, and learns that the residual features are F(x)=H(x)-x. Therefore, the original mapping becomes F(x)+x, and the convolutional layer learns new features based on the input features, increasing the network depth and improving the model performance.

[0048] The present invention builds an adult echocardiogram cross-section classification model based on a 101-layer ResNet network, trains it by a training set, and adopts a transfer learning strategy in the training stage, the pre-training weight adopts the result trained on the ImageNet dataset, the output layer classifier is a Softmax function, the number of classification categories is set to 13 categories, the Adam optimizer with the initial learning rate of 0.0001 is used, the batch processing size is 128, the model is fine-tuned, and iterated 100 times, and the features of the ultrasound image are extracted by a convolutional neural network with residual structure, where the specific steps are: first, input one ultrasound image with the original size, and call the transforms.Resize() method in Pytorch to adjust the image size to 256*256*3. In the training set, call the transforms.RandomRotation() method to randomly rotate the image horizontally at any angle within the range of 0-15°, increasing the diversity and robustness of the data and improving the generalization ability of the model. In order to improve the convergence speed, stability and generalization ability of the model, image standardization processing is performed using the transforms.Normalize() method in the training set and validation set, and the average value of the three channels of the image is 0.07, and the standard deviation is 0.15, 0.15 and 0.14, respectively. A 7x7 convolution kernel is input to the preprocessed image to perform a convolution operation, the step size is 2, the number of output channels is 64, and the boundary is filled to 3. Then, batch normalization and ReLU activation function are applied, and a max pooling operation is performed using a 3x3 pooling kernel, the step size is 2, and the size of the feature map is reduced. Next, the basic block group 1 contains three basic block configurations. Each basic block contains two 1x1 convolution kernels (step size is 1, the number of output channels is 64, 256) and one 3x3 convolution kernel (step size is 1, the number of output channels is 64, the boundary is filled to 1). We apply batch normalization after each convolution kernel and employ a ReLU activation function after the second 1x1 convolution kernel.The inputs of these three basic blocks bypass the convolutional layer by skip connections and directly add the inputs to the output of the convolutional layer to avoid the vanishing gradient problem. Next, the output channels are increased from 256 to 512, 1024, and 2048 using the same configuration as basic block group 1. The feature map is converted to a vector 2048*1*1 using a global average pooling operation. Finally, the output layer uses a linear full-connection operation to map the feature vector to a 13-category probability distribution by adopting a Softmax function to predict the category of the cross section of the ultrasound image.

[0049] After training is completed, the model performance is evaluated on the validation set by minimizing the cross-entropy loss value between the true label and the predicted result, and the model weight with the smallest loss value is selected as the optimal adult echocardiogram cross-section classification model. The model development and validation are realized using Python (version 3.7.10) and PyTorch (version 1.7.1) software on a server equipped with an NVIDIA GeForce RTX3090 GPU (24GB memory).

[0050] In step 6, the performance of the optimal adult echocardiogram cross-sectional classification model is evaluated based on the test set, specifically,

[0051] The performance of the optimal adult echocardiogram cross-sectional classification model is evaluated in the validation and test sets using polynomial metrics such as confusion matrix, accuracy, precision, recall (also known as "positive predictive value"), specificity (also known as "sensitivity") and F1 score.

[0052] In step 7, the measurement target image or video is input into the adult echocardiogram slice classification model to obtain a classification result, specifically,

[0053] The measurement target image is input into the adult echocardiogram cross-section classification model, where for the adult echocardiogram video, the adult echocardiogram cross-section classification model predicts the classification result of each image. For the adult echocardiogram video, 10 frames of images are sampled from each video and other intervals to perform prediction, and the average value is taken for the prediction result, and the cross-section category corresponding to the maximum prediction probability is the classification result of the video. A gradient weighting type activation graph visualization analysis method is used to generate a heat map using the feature weight of the last layer of the adult echocardiogram cross-section classification model, and the visualized cross-section classification model is focused on classifying the area, and an interpretable analysis is performed on the classification result, which is helpful to understand the deep logic of the cross-section classification model and improve the interpretability of the model.

[0054] In one embodiment of the present invention, we train different classification models on a multi-modal, multi-cross-section dataset, and evaluate the performance of the classification models at the image level of the validation set, as well as at the image and video levels of the test set. The results are shown in Table 1. JPEG2025001618000008.jpg96117

[0055] At the image level of the validation set, we evaluate the performance of the classification model using the accuracy rate and F1 score, and show that ResNet101 has the highest F1 score and its classification performance is optimal (see Table 1). Therefore, we select ResNet101 as the final cross-sectional classification model and evaluate the classification performance of the model at the image level and video level of the test set, respectively. JPEG2025001618000009.jpg74116

[0056] Note: 2D_2C indicates a two-dimensional grayscale apical two-chamber view, 2D_3C indicates a two-dimensional grayscale apical three-chamber view, 2D_4C indicates a two-dimensional grayscale apical four-chamber view, 2D_BX indicates a two-dimensional grayscale subxiphoid view, 2D_P indicates a two-dimensional grayscale parasternal left ventricular long-axis view, 2D_SAX indicates a two-dimensional grayscale parasternal left ventricular short-axis view, 2D_SAoA indicates a two-dimensional grayscale parasternal aortic short-axis view, 3D indicates a three-dimensional grayscale echocardiogram, C2D_2C indicates a two-dimensional left heart echocardiogram apical two-chamber view, C2D_3C indicates a two-dimensional left heart echocardiogram apical three-chamber view, C2D_4C indicates a two-dimensional left heart echocardiogram apical four-chamber view, C2D_SAX indicates a two-dimensional left heart echocardiogram parasternal left ventricular short-axis view, and CDFI indicates a color Doppler echocardiogram.

[0057] As can be seen from the results, at the image level, the average accuracy of ResNet101 classification of 13 categories cross-section is 97.13, and the average F1 score is 96.86 (see Table 2), while at the video level, the average F1 score of ResNet101 classification of 13 categories cross-section is 97.07.

[0058] According to the Grad-CAM visualization method, an interpretability analysis of the results is performed on the test set, and it is shown that the cross-section classification model determines that the key attention areas of the image category and the ultrasound physician are consistent, for example, in the two-dimensional grayscale apical four-chamber cross-section as shown in Figures 2a-2c, where both the model and the physician focus on the crisscross area of ​​the atrium and ventricle.

[0059] Based on the deep learning algorithm provided by the present invention A method for classifying multiple planes in multimodal echocardiograms The method comprises: Multi-sectionThe method includes collecting adult echocardiogram videos and images, preprocessing the collected adult echocardiogram videos and images, and labeling the preprocessed adult echocardiogram videos and images to generate an adult echocardiogram dataset. The adult echocardiogram dataset is divided into a training set, a validation set and a test set, and an adult echocardiogram slice classification model is constructed based on a ResNet network. Then, it is trained by the training set, the validation set is used to screen the optimal classification model, and the optimal adult echocardiogram slice classification model is evaluated for performance based on the test set. The measurement target image is input into the adult echocardiogram slice classification model to obtain the classification result, and the method preprocesses the adult echocardiogram videos and images, extracts the region of interest, and hides the patient's examination and personal information. The method can be applied to multi-modal (2D grayscale, Doppler, 3D and acoustic contrast), multi-section (mainly for slices commonly used in 2D grayscale clinical examination) echocardiogram data. The method is labeled by ultrasound doctors with rich clinical experience, and can guarantee the accuracy of the labels. The method can randomly divide the dataset proportionally to ensure equal data diversity and sample amount. The method can evaluate the model prediction performance in independent datasets by multiple classification indices to ensure the robustness of the model prediction results. The method employs anatomical regions of focus in the class activation heatmap visualization model classification process to improve the interpretability of the model.

[0060] The principle and embodiment of the present invention are described by applying specific examples in the present specification, but the above description of the examples is intended to help understand the method of the present invention and its central idea, and at the same time, those skilled in the art may find modifications in the specific embodiments and application scope based on the idea of ​​the present invention. In short, the description in this specification should not be understood as limiting the present invention.

Claims

1. A multi-modal multi-plane classification method for echocardiograms based on a deep learning algorithm, comprising: Step 1: Collect multi-module, multi-plane adult echocardiogram videos and images. Step 2: Preprocessing the collected adult echocardiogram videos and images is performed. Step 3: Label the preprocessed adult echocardiogram videos and images to generate an adult echocardiogram dataset; Step 4: Divide the adult echocardiogram dataset into a training set, a validation set, and a test set. Step 5: Based on the ResNet network, build an adult echocardiogram cross-section classification model, train it by the training set, and screen the optimal classification model by the validation set; Step 6: Perform performance evaluation against the optimal adult echocardiogram cross-sectional classification model based on the test set; Step 7: inputting the measurement target image or video into an adult echocardiogram slice classification model to obtain a classification result; A multi-modal multi-plane classification method for echocardiograms based on a deep learning algorithm, characterized by:

2. In step 1, multi-module, multi-plane adult echocardiogram videos and images are collected, specifically:

2. The method for multi-modal and multi-plane classification of echocardiograms based on deep learning algorithms according to claim 1, further comprising: collecting multi-module, multi-plane adult echocardiogram videos and images, including a three-dimensional grayscale echocardiogram, a color Doppler echocardiogram, a two-dimensional grayscale parasternal left ventricular long axis section, a two-dimensional grayscale parasternal left ventricular short axis section, a two-dimensional grayscale parasternal aortic short axis section, a two-dimensional grayscale subxiphoid section, a two-dimensional grayscale apical two-chamber section, a two-dimensional grayscale apical three-chamber section, a two-dimensional grayscale apical four-chamber section, a two-dimensional left echocardiogram apical two-chamber section, a two-dimensional left echocardiogram apical three-chamber section, a two-dimensional left echocardiogram apical four-chamber section, and a two-dimensional left echocardiogram parasternal left ventricular short axis section.

3. In step 2, preprocessing is performed on the collected adult echocardiogram videos and images, specifically, The method for multi-modal and multi-section classification of echocardiograms based on deep learning algorithms as claimed in claim 2, characterized in that batch processing is performed on adult echocardiogram videos and images using a third-party Python library OpenCV, and a sector-shaped region of interest is extracted according to pixel changes in successive frame images in accordance with image pre-processing operations, and the video is saved in PNG format for each frame.

4. In step 4, the adult echocardiogram dataset is divided into a training set, a validation set, and a test set, specifically: The method for multi-modal multi-section classification of echocardiograms based on a deep learning algorithm as claimed in claim 3, characterized in that the adult echocardiogram dataset is divided into a training set, a validation set and a test set in a ratio of 8:1:1, in which the amount of cross-sectional data with the smallest sample amount is taken as the basis, and the cross-sectional datasets of other categories are sampled in equal proportions to construct a dataset with equal sample amounts, so that the cross-sectional sample amounts of each category are equal, in which the training set is used to train an adult echocardiogram cross-section classification model, the validation set is used to adjust the model hyperparameters and screen the optimal classification model, and the test set is used to evaluate the classification performance of the model.

5. In step 5, an adult echocardiogram cross-sectional classification model is constructed based on the ResNet network, trained by the training set, and screened for the optimal classification model by the validation set, specifically, The method for multi-modal multi-plane classification of echocardiograms based on deep learning algorithms as claimed in claim 4, characterized in that: based on a 101-layer ResNet network, an adult echocardiogram slice classification model is constructed, and trained by a training set; in the training phase, a transfer learning strategy is adopted, and the pre-training weights adopt the results trained on the ImageNet dataset; the output layer classifier is a Softmax function, the number of classification categories is set to 13 categories, and an Adam optimizer with an initial learning rate of 0.0001 is used, and the batch processing size is 128; model fine-tuning is performed for 100 iterations; and the features of the ultrasound image are extracted by a convolutional neural network with residual structure; after training is completed, the model performance is evaluated in the validation set by minimizing the cross-entropy loss value between the true label and the predicted result; and the model weights with the highest classification accuracy rate are selected as the optimal adult echocardiogram slice classification model.

6. In step 6, a performance evaluation is performed on the optimal adult echocardiogram cross-sectional classification model based on the test set, specifically, The method for multi-modal multi-plane classification of echocardiograms based on a deep learning algorithm as claimed in claim 5, characterized in that the performance of the optimal adult echocardiogram plane classification model is evaluated in the validation set and the test set using a confusion matrix, accuracy rate, precision, recall rate, specificity and F1 score.

7. In step 7, the measurement target image is input into the adult echocardiogram slice classification model to obtain a classification result, specifically, The method for multi-modal multi-plane classification of echocardiograms based on deep learning algorithms as claimed in claim 6, characterized in that: the measurement target image is input into an adult echocardiogram cross-section classification model; for adult echocardiogram videos, the classification result of each image is predicted by the adult echocardiogram cross-section classification model; for adult echocardiogram videos, 10 frame images are sampled at equal intervals for each video to make a prediction; the average value is taken for the prediction results; and the cross-section type corresponding to the maximum prediction probability is used as the classification result of the video; and a gradient-weighted activation graph visualization analysis method is used to generate a heat map using the feature weights of the last layer of the adult echocardiogram cross-section classification model; and the visualized cross-section classification model focuses on key attention areas and performs an interpretable analysis of the classification results.

Images