Intelligent auxiliary evaluation system for systolic and diastolic functions of right ventricle in echocardiography

Abstract

The invention provides an intelligent auxiliary evaluation system for right ventricular systole and diastole functions of echocardiography, and aims to solve the problems of low accuracy and efficiency in right ventricular function evaluation in the prior art. The system comprises an image acquisition unit, an image processing unit, a feature extraction unit, an intelligent evaluation unit and a result output unit, and through an intelligent algorithm, especially a deep learning network DL and a convolutional neural network CNN, a feature extraction result is obtained. Key indexes such as a two-dimensional right ventricular area change fraction 2D-FAC, a right ventricular ejection fraction RVEF and tricuspid valve ring plane contraction period displacement TAPSE are automatically calculated. The system also realizes accurate evaluation of the right ventricular function through a specific CNN structure and loss function design. The system can provide rapid, accurate and non-invasive right ventricular function evaluation in clinical practice, and is helpful for improving the diagnosis and treatment effects of cardiovascular diseases.

Description

Technical Field

[0001] The present invention relates to the field of medical diagnosis, and particularly to an intelligent auxiliary evaluation system for the systolic and diastolic functions of the right ventricle in echocardiography. Background Art

[0002] In the process of diagnosing cardiovascular diseases, accurately evaluating the right heart function is of great significance for disease diagnosis, treatment, and prognosis. The existing evaluation of right ventricular function mainly relies on ultrasound examination, but there are many limitations: The right ventricle has a unique structure and shape, making traditional ultrasound measurements relatively complex. The evaluation indicators of right ventricular systolic function, the two-dimensional right ventricular area change fraction (2D-FAC) and the right ventricular ejection fraction (RVEF), have accuracy problems due to the irregular shape of the right ventricle and the limitations of measurement methods. Among them: The formula for 2D-FAC is: 2D-FAC = (right ventricular end-diastolic area - right ventricular end-systolic area) / right ventricular end-diastolic area × 100%, and the formula for RVEF is: RVEF = (right ventricular end-diastolic volume - right ventricular end-systolic volume) / right ventricular end-diastolic volume × 100%. At present, the evaluation of right ventricular function abnormalities mainly relies on manual visual interpretation, which is inefficient and requires high doctor experience and technical level. Summary of the Invention

[0003] The technical problems to be solved include but are not limited to providing a more accurate and efficient intelligent auxiliary evaluation system for the systolic and diastolic functions of the right ventricle, which can automatically evaluate the systolic and diastolic functions of the right ventricle based on two-dimensional echocardiography, and improve the accuracy and speed of evaluation through intelligent algorithms, including:

[0004] An image acquisition unit for real-time collecting two-dimensional echocardiogram data of a patient and retrieving existing ultrasound images from a database;

[0005] An image processing unit for preprocessing the collected images, including preprocessing work such as denoising, edge enhancement, and enhancing contrast;

[0006] A feature extraction unit for extracting the morphological features of the right ventricle by using the preprocessed data provided by the image processing unit;

[0007] An intelligent evaluation unit that uses machine learning algorithms to evaluate the systolic and diastolic functions of the right ventricle through the features extracted by the feature extraction unit, and the algorithms include a deep learning network (DL) and a convolutional neural network (CNN);

[0008] A result output unit for presenting the evaluation results to the doctor in a graphical and numerical manner.

[0009] Preferably, the intelligent evaluation unit is constructed by using the CNN algorithm in a CNN model specifically for two-dimensional echocardiograms. The model includes multiple convolutional layers, pooling layers, and fully connected layers; a specific loss function is used for training, the Dice loss function, to improve the accuracy of evaluation; the CNN model has a pixel-level image segmentation function to achieve precise recognition of the right ventricular region and its structure.

[0010] Preferably, when training the CNN network, the intelligent evaluation unit uses the Hausdorff distance loss function to reduce the segmentation error. During the feature extraction and annotation process of the network, an image dataset manually annotated is used to train the CNN model to identify the key anatomical landmarks of the right ventricle and the motion characteristics of the ventricular wall.

[0011] Preferably, the result output unit displays the image of the right ventricle on the graphical interface of the two-dimensional echocardiogram and marks the key structures of the right ventricle; at the same time, it outputs the numerical values of the right ventricular function evaluation indexes, including but not limited to the two-dimensional right ventricular area change fraction 2D-FAC and the right ventricular ejection fraction RVEF.

[0012] Preferably, the image acquisition unit is further designed to be able to collect two-dimensional echocardiogram data of the patient's cardiac activity state in real time and has the function of retrieving two-dimensional echocardiogram image data from an existing medical image database.

[0013] Preferably, the feature extraction unit includes an extraction device configured to identify and extract feature data such as the area change of the right ventricle, the motion of the ventricular wall, and the morphological change from the preprocessed image.

[0014] Preferably, the intelligent evaluation unit includes an evaluation device configured to evaluate the right ventricular function using a deep learning algorithm and capable of automatically calculating the key indexes of the two-dimensional right ventricular area change fraction 2D-FAC, the right ventricular ejection fraction RVEF, and the tricuspid annular plane systolic excursion TAPSE from the feature data received from the feature extraction unit.

[0015] Advantages of the present invention:

[0016] The present invention uses an image acquisition unit that real-time collects two-dimensional echocardiogram data of patients and retrieves existing ultrasound images from a database; an image processing unit for preprocessing the collected images, and the methods include but are not limited to denoising, edge enhancement, and enhancing contrast; a feature extraction unit that uses the data provided by the image processing unit to extract the morphological features of the right ventricle, such as area change, ventricular wall movement, and morphological change, etc.; and an intelligent evaluation unit that uses a machine learning algorithm to evaluate the systolic and diastolic functions of the right ventricle through the features extracted by the feature extraction unit. The algorithm includes a deep learning network, such as a convolutional neural network (CNN), and can automatically calculate key indicators such as 2D-FAC, RVEF, and TAPSE. It can provide a fast, accurate, and non-invasive evaluation of right ventricular function in clinical practice, which helps to improve the diagnosis and treatment effects of cardiovascular diseases. Description of the Drawings

[0017] Appendix Figure 1 : The workflow diagram of the intelligent auxiliary evaluation system of the present invention.

[0018] Appendix Figure 2 : The CNN network structure diagram of the present invention.

[0019] Appendix Figure 3 : The data processing flowchart of the present invention. Detailed Embodiments

[0020] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention; obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without making creative efforts shall fall within the protection scope of the present invention.

[0021] Next, the technical solutions of the present invention will be clearly and completely described in conjunction with the embodiments. Obviously, the described embodiments are a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without making creative efforts shall fall within the protection scope of the present invention.

[0022] Embodiment 1

[0023] Referring to the appendix of the specification Figures 1-3 , for the automatic evaluation method of right ventricular function based on a convolutional neural network, this embodiment includes the following steps:

[0024] S01. Integrate multi-source data acquisition, use a GE Vivid E95 ultrasound device to collect two-dimensional dynamic images of the four-chamber view of the patient's right ventricle and the right ventricular outflow tract view, and continuously record 3 cardiac cycles for each view. Among them, the image acquisition unit (such as the appendixFigure 1 Module A) obtains image data in real time through the DICOM protocol, and synchronously retrieves historical case data from the hospital PACS system to construct a training set containing 8,000 labeled data.

[0025] S02. Perform normalization processing on the original ultrasound image, refer to the appendix Figure 3 , the spatial resolution is uniformly 512×512 pixels. Use the bicubic interpolation algorithm to eliminate device differences. Apply limited contrast adaptive histogram equalization (parameters: CLAHE, grid size 8×8, contrast limit 2.0) to enhance tissue boundaries. At the same time, use anisotropic diffusion filtering to suppress speckle noise. After edge enhancement with the Sobel operator, perform morphological closing operation with a 3×3 circular structural element to calculate the connected broken contours.

[0026] S03. Perform multi-scale feature extraction through the dedicated CNN architecture shown in the appendix Figure 2 . Receive the preprocessed 512×512 single-channel image in the input layer. The first 12 layers of the CNN use a 5×5 convolutional kernel with a stride of 2 and padding of 1 to extract the macroscopic morphological features of the right ventricle. The last 4 layers switch to a 3×3 convolutional kernel with a stride and padding of 1 to capture the details of ventricular wall movement. After every two convolutional layers, a 2×2 max pooling layer is connected, with a total of 8 layers, gradually reducing the resolution of the feature map. Then, add cross-layer connections after the fourth and eighth pooling layers to fuse shallow texture and deep semantic information.

[0027] S04. Precise segmentation and index calculation. Output the segmentation mask of the right ventricle region through a fully connected layer (1024→512 neurons). Use Dice loss with a weight of 0.6 + cross-entropy loss with a weight of 0.3 + edge continuity loss with a weight of 0.1 to make the segmentation Dice coefficient reach 0.91±0.03. As shown in the last step of the appendix Figure 2 , apply an additional Hausdorff distance loss with a threshold of 3mm to 5% of the low-quality images, reducing the segmentation error from 2.8mm to 1.5mm. Based on the segmentation results, calculate 2D-FAC (formula: (end-diastolic area - end-systolic area) / end-diastolic area × 100%), RVEF (three-dimensional reconstruction method), and TAPSE (tricuspid annular displacement tracking) using the following formulas.

[0028] S05. As shown in the output module in the appendix Figure 3 , the system generates a dynamic overlay view, marks the boundary of the right ventricle with a red contour on the original ultrasound image, and the blue arrow indicates the TAPSE measurement path. The data report shows that 2D-FAC is 35.2±6.8%, RVEF is 52.5±8.4%, TAPSE is 18.5±3.2mm, and its consistency with manual measurement (correlation coefficient r = 0.92 - 0.94, Bland-Altman bias < 3%).

[0029] Among them, the details of the Convolutional Neural Network (CNN) algorithm are as follows:

[0030] Image data collection and preprocessing: First, collect 8000 two-dimensional echocardiogram image data from clinical practice and perform preprocessing, such as image normalization and contrast enhancement. CNN structure construction: Build a CNN model specifically for two-dimensional echocardiograms. The model includes multiple 16 convolutional layers, 8 pooling layers, and 2 fully connected layers. A pooling layer is connected after every two convolutional layers. The convolutional kernels of the first 12 convolutional layers are 5x5, and the convolutional kernels of the last 4 convolutional layers are 3x3, which are used to extract local features of the image. The pooling layer reduces the resolution, and the fully connected layer is used to synthesize features and output the final result. To address the problem of uneven segmentation of image data, the Dice loss function is used in combination with the softmax activation function to ensure more accurate results. In 5% of the data with unclear samples and blurred edges, that is, 400 images, the Hausdorff distance loss function is used to directly reduce the segmentation error to evaluate the accuracy of the segmentation result. Feature extraction and annotation: Use the labeled dataset to train the CNN model. Through repeated learning, the model optimizes the network weights to identify and locate relevant anatomical landmarks of the right ventricle, such as the ventricular wall, apex, and tricuspid valve. Evaluation of the trained CNN network: After training, use the network to evaluate unlabeled two-dimensional echocardiograms. The network can automatically extract the key regions of the right ventricle and calculate three-dimensional morphological parameters such as the area change fraction and ejection fraction. Performance verification and optimization: Finally, verify and optimize the model performance to ensure accurate and reliable evaluation in actual clinical applications. Result output unit: Present the evaluation results to the doctor in graphical and numerical forms, including but not limited to the graphical interface of the two-dimensional echocardiogram and the numerical values of the right ventricle function evaluation indicators.

[0031] Example 2:

[0032] Combined with Appendix Figure 2 and 3 , this example is a method for volumetric quantitative evaluation of patients with right ventricular hypertrophy, including the following steps:

[0033] S01. Use a Philips EPIQ7C ultrasound instrument to collect the standard four-chamber view of the right ventricle at 70 - 90 frames per second, continuously record 5 cardiac cycles, synchronously obtain cardiac MRI data as the gold standard, and construct a validation set of 150 patients with right ventricular hypertrophy.

[0034] S02. Suppress motion artifacts in the dataset obtained in step S01 through time-domain registration method and spatial-domain denoising method. Specifically, in the preprocessing stage (see Appendix Figure 3) Align consecutive frame images using the phase correlation algorithm to compensate for respiratory motion displacement, and improve anisotropic diffusion filtering. Adaptively adjust the gradient magnitude threshold in the range of 15 - 25, and combine morphological opening operation with a 5×5 cross-shaped structuring element to eliminate papillary muscle artifacts.

[0035] S03. Perform three-dimensional volume reconstruction using an improved U-Net model, as shown in the appendix. Figure 2 Specifically, after the dataset obtained by processing through S01 and S02, an encoder composed of 16 convolutional layers (the first 12 layers with 5×5 kernels and the last 4 layers with 3×3 kernels), 8 max-pooling layers, 8 transposed convolutional layers with a stride of 2, fuse low-level detailed features through skip connections, and obtain three-dimensional point cloud data of the right ventricular end-diastolic volume (RVEDV) and end-systolic volume (RVESV) at the output layer. Compared with MRI, the volume calculation error is <5%.

[0036] S04. Under the calculation of the convolutional application network in S03 above, obtain a multi-parameter joint index analysis, calculate the right ventricular mass index (RVMI) and free wall thickness. The RVMI formula is (right ventricular myocardial mass / body surface area), and it is measured as 32.5 ± 4.8 g / m 2 , the intra-class correlation coefficient ICC = 0.95. Wall thickness measurement selects 6 standard sites at the end-diastolic phase, with an average thickness of 6.8 ± 1.2 mm, and there is no significant difference compared with MRI (P > 0.05).

[0037] S05. According to the results obtained in S04 above, generate a comprehensive report containing the following elements, as shown in the appendix. Figure 1 Result output unit, where the color-coded map shows the right ventricular wall thickness distribution in the form of a heat map. The red area indicates hypertrophy (>7 mm), the dynamic volume curve shows the volume change during the cardiac cycle, and the peak corresponds to the end-diastolic phase. The automatic diagnosis suggests that when RVMI > 35 g / m 2 and the wall thickness > 7 mm, mark "high possibility of right ventricular hypertrophy", with an accuracy rate of 94.7%.

[0038] Example 3:

[0039] The multi-modal assessment method for right ventricular diastolic function refers to the appendix. Figures 1-3 For the multi-modal method of right ventricular diastolic function of Apple, this example includes the following steps:

[0040] S01. Use the Siemens Acuson SC2000 ultrasound system to collect multi-parameter data. Specifically, for the blood flow spectrum, place the sampling volume at the tip of the tricuspid valve to obtain the velocities of the early diastolic E wave (0.62±0.15 m / s) and the atrial systolic A wave (0.48±0.12 m / s). For tissue Doppler, measure the longitudinal strain rate (2.1±0.4 s-1) on the free wall side, and increase the frame rate to 80 - 100 frames per second to ensure time resolution.

[0041] S02. Perform joint time-frequency domain analysis on the data collected in S01, and add spectrum correction and motion compensation correction in the preprocessing stage (attached Figure 3 ). Specifically, for spectrum correction, apply Gaussian filtering (σ = 1.5) and smoothing spline interpolation to the blood flow signal to eliminate the influence of heart rate variability; for motion compensation correction, use the block matching algorithm with parameters of a search window of 16×16 and a step size of 4 pixels to correct the inter-frame displacement of the tissue Doppler image.

[0042] S03. Feature fusion and classification, construct a random forest model (n_estimators = 200) to obtain feature fusion and classification. The input feature parameters of the model are the E / A ratio (1.42±0.35), the E wave deceleration time (DT = 180±25 ms), and the peak strain rate time (110±15 ms). The grading criteria are normal (E / A>1.0, DT<200 ms), mild insufficiency (0.8<E / A≤1.0), and moderate to severe insufficiency (E / A≤0.8). Obtain the performance indicators, with an overall accuracy of 91.3%, and a sensitivity of 87.9% for moderate to severe cases (AUC = 0.93).

[0043] S04. According to the attached Figure 1 result output unit interface, generate an interactive report, which includes a dynamic curve overlay of a chart showing the blood flow spectrum and the strain rate time history curve in the same coordinate system, distinguishing the diastolic function grades with green / yellow / red borders for automatic aircraft identification. The evaluation time of the system is 2.5±0.5 minutes, which is 65% shorter than manual measurement.

[0044] Example 4:

[0045] Refer to the attached Figures 2-3 , a method for system robustness verification and real-time optimization, including the following steps:

[0046] S01. For 5% low-quality images (motion blur or limited acoustic window), data augmentation methods were adopted. ±15° rotation, Gaussian noise (σ = 0.01), and elastic deformation (α = 30, σ = 5) were added to the training set, and the loss function of the final output of the CNN for the images was optimized. The weighted Hausdorff distance loss (α = 0.7) was used to optimize the 95% consistency limit of the segmentation results from -3.2 mm to -1.5 mm.

[0047] S02. In the 2,356 cases of data collected by GE, Philips, and Siemens devices, cross-platform compatibility tests were carried out. Among the various devices, the inter-group difference in 2D-FAC measurement was <2.8% (F = 1.32, P = 0.26), indicating the consistency of the device parameters. On the NVIDIA Tesla V100 GPU, the single-frame processing time was ≤85 ms, <100 ms, which could meet the clinical real-time requirements.

[0048] S03. According to the Figure 1 In the result display interface shown, 20 physicians were asked to provide feedback to improve the output interface, and the human-computer interaction was optimized. The confidence indicator showed the result reliability with a 0-100% progress bar. When the confidence was <90%, a warning was triggered, and a one-key review function was added. Clicking on the suspicious area could retrieve the original image and the segmentation process animation to assist manual verification.

[0049] Example 5:

[0050] Referring to the Figure 2 , the deep learning model training and compression method adopted in this example includes the following steps:

[0051] S01. 8,000 cases of data were divided into a training set, a validation set, and a test set in a ratio of 7:2:1. Among them, 3 experts independently outlined the right ventricular end-diastolic / end-systolic contours to perform standard annotation of the image data. The Kappa consistency coefficient was >0.85, and elastic deformation (σ = 25, α = 1500) and local brightness perturbation (Δ = ±15%) were applied to 5% of the difficult samples for image enhancement.

[0052] S02. The method of knowledge distillation compression by migrating the teacher model (ResNet50) to the student model (MobileNetV3) was used. The L2 distance loss (weight 0.3) was added to the fourth and eighth convolutional layers, and the model volume was compressed from 98 MB to 16 MB. The inference speed was increased by 3.2 times, and the Dice coefficient only decreased by 2.3%.

[0053] S03. The dynamic learning rate adjustment adopts the cosine annealing strategy. The initial learning rate is 1e-4, the minimum adjusted learning rate is 1e-6, and the training cycle length is 50 epochs, so that the training loss steadily drops to 0.12 ± 0.02.

[0054] As described above, it is only a preferred specific implementation manner of the present invention; however, the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution of the present invention and its improved concept, makes equivalent substitutions and changes, and should be covered by the protection scope of the present invention.

Claims

1. An intelligent auxiliary evaluation system for right ventricular systolic and diastolic function by echocardiography, characterized in that: include: An image acquisition unit, used for acquiring two-dimensional echocardiographic data of the patient in real time and retrieving existing ultrasound images from a database; An image processing unit is used to pre-process the collected images, including denoising, edge enhancement, and contrast enhancement; A feature extraction unit, used for extracting the morphological features of the right ventricle using the pre-processed data provided by the image processing unit; An intelligent evaluation unit, which uses a machine learning algorithm to evaluate the right ventricular systolic and diastolic function through the features extracted by the feature extraction unit, wherein the algorithm includes a deep learning network DL and a convolutional neural network CNN; The result output unit is used to present the evaluation results to the doctor in a graphical and numerical manner.

2. The intelligent assisted evaluation system according to claim 1, characterized in that: The intelligent evaluation unit adopts the CNN algorithm to build a CNN model specifically for two-dimensional echocardiography, which includes multiple convolutional layers, pooling layers and fully connected layers; a specific loss function, the Dice loss function, is used for training to improve the accuracy of the evaluation; the CNN model has a pixel-level image segmentation function to achieve accurate identification of the right ventricular area and its structure.

3. The intelligent assisted evaluation system according to claim 1, characterized in that: The intelligent evaluation unit adopts the Hausdorff distance loss function to reduce the segmentation error when training the CNN network. During the feature extraction and annotation process of the network, the CNN model is trained using the manually annotated image data set to identify the key anatomical landmarks of the right ventricle and the motion characteristics of the ventricular wall.

4. The intelligent assisted evaluation system according to claim 1, characterized in that: The result output unit displays the image of the right ventricle in a two-dimensional echocardiogram graphical interface and marks the key structures of the right ventricle; at the same time, it outputs the values ​​of right ventricular function evaluation indicators, including but not limited to the two-dimensional right ventricular area change fraction 2D-FAC and the right ventricular ejection fraction RVEF.

5. The intelligent assisted evaluation system according to claim 1, characterized in that: The image acquisition unit is further designed to be able to acquire two-dimensional echocardiographic data of the patient's cardiac activity status in real time, and has the function of retrieving two-dimensional echocardiographic image data from an existing medical imaging database.

6. The intelligent assisted evaluation system according to claim 1, characterized in that: The feature extraction unit comprises an extraction device configured to identify and extract feature data such as area change, cardiac wall movement and morphological change of the right ventricle from the preprocessed image.

7. The intelligent assisted assessment system according to claim 1, characterized in that: The intelligent evaluation unit includes an evaluation device, which is configured to evaluate right ventricular function using a deep learning algorithm and can automatically calculate key indicators such as two-dimensional right ventricular area change fraction 2D-FAC, right ventricular ejection fraction RVEF and tricuspid valve annular plane systolic displacement TAPSE from the feature data received by the feature extraction unit.

Images