A method for cardiac function evaluation based on mask adaptive scale alignment

Abstract

The present application relates to a kind of cardiac function evaluation methods based on mask adaptive scale alignment, comprising the following steps: S1, left ventricle segmentation model is trained using original data set, left ventricular region is positioned and segmented, to generate alignment factor;S2, using the alignment factor, interpolation alignment data set and variant data set are constructed on the basis of original data set;S3, using the interpolation alignment data set, left ventricle segmentation model is optimized and the segmentation result is reconstructed;S4, the variant data set is expanded to obtain flip data set, and using the variant data set and the flip data set training ejection fraction prediction model;S5, experimental result analysis.The present application can reduce the segmentation error of different frames, provide better left ventricular real-time guidance and positioning information, also can improve the generalization ability of model when performing ejection fraction prediction task, realize more accurate prediction and significant performance improvement.

Description

Technical Field

[0001] This invention relates to the fields of medical artificial intelligence and deep learning technology, specifically to a method for assessing cardiac function based on mask adaptive scale alignment. Background Technology

[0002] Left ventricular (LV) segmentation and ejection fraction (EF) measurement are frequently used in clinical settings. LV segmentation provides real-time guidance and location information for detecting cardiac features such as ventricular dilation and abnormal ventricular wall motion; EF prediction is often used to monitor cardiovascular health and diagnose heart diseases such as myocardial infarction and heart failure. Therefore, researching accurate LV segmentation and EF prediction algorithms is highly valuable for cardiac assessment. In recent years, deep learning techniques have been used to identify, quantify, and interpret ultrasound images. These models reduce the time required to analyze images / videos, accelerate clinical decision-making, and provide interactive feedback to train less experienced physicians. However, current methods mainly focus on improving model structure, with less attention paid to the characteristics of the data itself. In the field of medical data, there are common challenges such as difficulty in data acquisition and limited data annotation, while deep learning typically requires a large number of samples for training to achieve good generalization ability. This has led to significant differences in segmentation accuracy across different frames for some current LV segmentation models, and varying degrees of overfitting in EF prediction models during training. Summary of the Invention

[0003] The purpose of this invention is to provide a cardiac function assessment method based on mask adaptive scale alignment, which can reduce segmentation errors between different frames, provide better real-time guidance and positioning information for the left ventricle, and improve the generalization ability of the model when performing ejection fraction prediction tasks, thereby achieving more accurate prediction and significant performance improvement.

[0004] To achieve the above objectives, this invention provides a cardiac function assessment method based on mask adaptive scale alignment, comprising the following steps: S1, training a left ventricular segmentation model using the original dataset to segment the left ventricular region to generate an alignment factor; S2, using the alignment factor to construct an interpolated aligned dataset and a variant dataset based on the original dataset; S3, using the interpolated aligned dataset to optimize the left ventricular segmentation model and reconstruct the segmentation results; S4, expanding the variant dataset to obtain a flipped dataset, and using the variant dataset and the flipped dataset to train an ejection fraction prediction model; S5, analyzing experimental results.

[0005] Preferably, in step S1, two labeled frames from each original dataset's echocardiogram video and their corresponding labeling information are selected for cross-entropy loss, and several rounds of training are performed to obtain an M. best-rough Model.

[0006] The segmentation mask output by the model generates adaptive detection boxes for each frame of image, which serve as alignment factors.

[0007] Preferably, in S2, for the left ventricle segmentation task, an adaptive scale alignment mechanism is used to construct an interpolated aligned dataset.

[0008] Specifically, by applying an adaptive scale alignment operation and interpolation to the original dataset, the different detection boxes of the left ventricular region in each frame are enlarged to a uniform scale to obtain an interpolated aligned dataset. In the interpolated aligned dataset, the left ventricular regions of different frames are aligned in terms of spatial distribution and size.

[0009] Preferably, in S2, a variant dataset is constructed using a variant of the adaptive scale alignment mechanism for the ejection fraction prediction task.

[0010] Specifically, by using a variant operation of adaptive scale alignment on the original dataset, the left ventricular regions of different videos are spatially aligned without changing the relative size of key regions in different frames of the same video, thus obtaining a variant dataset.

[0011] Preferably, in S3, the interpolated alignment dataset is used to re-align the M... best-rough The model undergoes several rounds of optimization training to obtain the optimized model M. best .

[0012] More preferably, the model M best The segmentation mask obtained from the prediction comes from the interpolation alignment dataset. Therefore, the final prediction result needs to be subjected to an inverse transformation of the adaptive scale alignment mechanism to obtain the accurate segmentation result before the adaptive scale alignment mechanism.

[0013] Preferably, in S4, the variant dataset and the flipped dataset are used as inputs to the model, the R(2+1)d model is used to learn features, and then the ejection fraction is predicted.

[0014] In summary, compared with the prior art, the cardiac function assessment method based on mask adaptive scale alignment of the present invention has the following beneficial effects:

[0015] (1) The present invention can reduce the segmentation error between different frames when there are few annotations in the dataset video, and can also reduce the segmentation difference between different frames when the data is limited, thereby improving the accuracy of cardiac function assessment;

[0016] (2) The present invention can improve the accuracy of left ventricular segmentation in each frame. When segmenting the left ventricular, the video is directly used as input without changing the model architecture, and it can provide better real-time guidance and positioning information for the left ventricular.

[0017] (3) When performing the ejection fraction prediction task, the present invention can improve the generalization ability of the model, achieve more accurate prediction and significant performance improvement. Attached Figure Description

[0018] Figure 1 This is a flowchart of the method of the present invention;

[0019] Figure 2 This is a diagram of the left ventricular segmentation framework based on mask adaptive scale alignment in this invention;

[0020] Figure 3 This is a schematic diagram illustrating the interpolation principle of the adaptive scale alignment mechanism of the present invention;

[0021] Figure 4 This is a variant framework diagram of the mask-based adaptive scale alignment in this invention;

[0022] Figure 5 A comparison of left ventricular segmentation results with and without the use of mask-based adaptive scale alignment in this invention;

[0023] Figure 6 (a) is a comparison of the training loss reduction curves of different models in the ejection fraction prediction of this invention;

[0024] Figure 6 (b) is a comparison of the validation loss reduction curves of different models in the ejection fraction prediction of this invention. Detailed Implementation

[0025] The present invention will be further described below with reference to the accompanying drawings and by providing a detailed description of a preferred embodiment.

[0026] First, the original dataset, experimental hardware and software environment, and experimental evaluation metrics used in this invention will be introduced.

[0027] In a preferred embodiment of the present invention, the original dataset used is the EchoNet-Dynamic public EF dataset, which contains apical four-chamber two-dimensional echocardiograms. It includes echocardiographic videos of 10,030 independent individuals obtained at Stanford University Hospital between 2016 and 2018, with each video having a frame size of 112*112. This dataset specifically describes the end-systolic volume (ESV), end-diastolic volume (EDV), left ventricular contour, and ejection fraction (EF) for each patient's corresponding video. The dataset is derived from a split of mutually exclusive patients, including 7,465 training samples, 1,288 validation samples, and 1,277 test samples. Furthermore, only 12.7% of the samples in the training set have an EF ratio below 40%, indicating a relatively uneven distribution.

[0028] In a preferred embodiment of the present invention, the experimental hardware environment is: A800; the software environment is: python3.7, pytorch1.12.1+cu113.

[0029] Preferably, in the left ventricular segmentation task of this invention, two labeled frames from each video are selected for training. The basic architecture of DeepLabv3 with a 50-layer residual network is primarily adopted, and pixel-level binary cross-entropy loss is minimized. In this model, random weights are used for initialization, and a stochastic gradient descent optimizer is used for training with a learning rate of 0.00001, a momentum of 0.9, and a batch size of 20.

[0030] More preferably, in the ejection fraction prediction task of this invention, a decomposed R²+1d spatiotemporal convolution is selected as the feature extractor. The model input is a video segment with 32 frames sampled with a stride of 2, which is trained to minimize the squared loss between predictions. In this model, a stochastic gradient descent optimizer is used with an initial learning rate of 0.0001, momentum of 0.9, batch size of 16, and a duration of 45 epochs. The learning rate decays by a factor of 0.1 every 15 epochs.

[0031] In a preferred embodiment of the present invention, the performance of the segmentation model is evaluated using the Dice similarity coefficient, and the results are measured using MSE, RMSE, and R. 2 To evaluate the performance of the EF prediction model. Here, Dice is the set similarity metric, MSE represents the mean squared error, RMSE represents the root mean squared error, and R... 2 This represents the coefficient of determination.

[0032] like Figure 1 As shown in the figure, this is a schematic diagram of a cardiac function assessment method based on mask adaptive scale alignment according to the present invention, which includes the following steps:

[0033] S1. Train a left ventricular segmentation model using the original dataset to locate and segment the left ventricular region to generate alignment factors;

[0034] S2. Using the alignment factor, construct an interpolated aligned dataset and a variant dataset based on the original dataset;

[0035] S3. Optimize the left ventricular segmentation model and reconstruct the segmentation results using the interpolated and aligned dataset;

[0036] S4. Expand the variant dataset to obtain the flipped dataset, and use the variant dataset and the flipped dataset to train the ejection fraction prediction model;

[0037] S5. Analysis of experimental results.

[0038] Furthermore, in S1, a segmentation model needs to be trained to perform pixel-level localization of the left ventricular region in the original dataset echocardiogram, and an alignment factor is generated for each frame of the image based on the mask output by the model.

[0039] Specifically, two labeled frames (ED and ES frames) from each echocardiogram video and their corresponding annotation information (GT) are selected for cross-entropy loss, and m rounds of training are performed to obtain an M-series dataset that performs well on the validation set. best-rough The model outputs a segmentation mask to generate adaptive detection boxes for each frame of image, which serve as alignment factors. These alignment factors can provide height, width, and other parameter information for subsequent adaptive scale alignment mechanisms.

[0040] Furthermore, such as Figure 2 and Figure 3 As shown, in S2, for the left ventricle segmentation task, an adaptive scale alignment (ASA) mechanism needs to be applied to construct an interpolated aligned dataset:

[0041] J1, take the sample data X from the original dataset. i ∈D is input to model M best-rough The segmentation mask corresponding to each frame can be obtained. Based on the model's output, for each frame Expand outward Each pixel can adaptively generate a corresponding detection box β for each frame. i ,β i It includes the initial point coordinates (x) of the left ventricular region in the corresponding frame. i ,y i and the corresponding width w i and height h i The specific formula is as follows:

[0042]

[0043] J2, Generate a corresponding detection box β for each frame in J1. i The specific formula for performing an ASA operation is as follows:

[0044] ASA(X i ,Y i ) = interp(cut((X i ,Y i ),β i ),δ),

[0045] Among them, X i Y represents the original sample data. i This represents the corresponding label information, where δ is based on β. iThe scale alignment factor is calculated from the original image size. If there is no annotation information, only the sample data X is operated on, and the data containing annotation information needs to be synchronized and aligned.

[0046] J3, based on the detection box β of each frame i The original data is cropped, and a bicubic interpolation is performed on the data in the original dataset D using a delta-scale alignment factor:

[0047]

[0048]

[0049] Wherein, for the pixel (x, y) to be interpolated, its nearest 4×4 neighborhood point (x, y) is taken. i y i The weighted average of ) is obtained; W(x) is the basis function of bicubic interpolation, used to find the parameter x, so as to obtain the weight W(x) corresponding to the 16 pixels.

[0050] Understandably, by interpolating the original dataset D, the different detection boxes β of the left ventricular region in each frame can be obtained. i Magnifying to a uniform scale, this yields sample data (X′) for the interpolated aligned dataset. i ,Y i ′)∈D′, and in the interpolated aligned dataset D′, the left ventricular regions of different frames are aligned in terms of spatial distribution and size.

[0051] Accordingly, such as Figure 4 As shown, in S2, for the ejection fraction prediction task, a variant dataset needs to be constructed using a variant of the adaptive scale alignment mechanism (ASA′):

[0052] K1, the sample data X of the original dataset. i ∈D is input to model M best-rough The segmentation mask corresponding to each frame can be obtained. Based on the model's output, for each frame Expand outward Each pixel can adaptively generate a corresponding detection box β for each frame. i ,β i It includes the initial point coordinates (x) of the left ventricular region in the corresponding frame. i ,y i and the corresponding width w i and height h i The specific formula is as follows:

[0053]

[0054] K2. Take the union of the detection box positions corresponding to all frames in any video, that is:

[0055] β=β1∪β2∪…∪β n n = length(clip)

[0056] Here, clip is the video clip selected by the model for each sample. By taking the union of the clips, a detection box β containing the left ventricular region of all frames in the video can be obtained.

[0057] K3. Align each frame of the video using ASA':

[0058] ASA′(X i ,Y i ) = interp(cut((X i ,Y i ),β),δ),

[0059] Among them, ASA′ and ASA are only different in that the detection box β is different, while all other operations are the same.

[0060] Understandably, by performing the ASA′ operation on the original dataset D, the left ventricular regions of different videos can be spatially aligned without changing the relative sizes of key regions in different frames of the same video, thus obtaining a variant dataset with irrelevant regions removed.

[0061] Furthermore, in S3, the dataset D′ is aligned using interpolation before model M is aligned. best-rough By performing nm rounds of optimization training, the optimized model M can be obtained. best The aligned dataset D′ reduces the differences between different frames. Therefore, by interpolating and aligning the dataset D′, the model can be trained to reduce the difference in the region size of the segmentation mask that the trained model needs to predict in each frame, thereby improving the model's segmentation ability in different frames.

[0062] Furthermore, since the predicted segmentation mask is applicable to a multi-frame aligned dataset, ASA needs to be performed on the final predicted segmentation mask. -1 operate:

[0063]

[0064] Among them, ASA -1It is the inverse operation of ASA. It applies the prediction results of the scale alignment factor δ to perform region interpolation, uses the pixel region relationship to resample, and adjusts it back to the size and position of the original data so that the segmentation result is aligned with the original data. This allows for accurate segmentation results on data that has not undergone multi-frame scale alignment.

[0065] Furthermore, in S4, due to the difficulty in acquiring medical data and the small amount of data collected, while in deep learning model training, more dataset samples can be used to train a more accurate and stable model, therefore, by using variant datasets... A horizontal flip yields a flipped dataset. This method expands the sample space of the sample dataset and enhances the diversity of the data.

[0066] Furthermore, the variant dataset D1 and the flipped dataset D2 are used as inputs to the model. The R(2+1)d model is used to learn video features and then the ejection fraction (EF) is predicted.

[0067] Furthermore, such as Figure 5 and Figure 6 As shown, the experimental results in S5 are analyzed as follows:

[0068] In a preferred embodiment of the invention, the entire training process for the left ventricular segmentation task lasts for 50 cycles. The first 30 cycles use the original data for training and validation. After completing the first 30 cycles, all data is reset, and the previously trained model is used as the pre-trained model for the remaining 20 cycles of training. In contrast, the ejection fraction prediction task involves 45 cycles of training. After one round of training using the entire training set, the trained model is validated on the validation set. If the validation set error is less than the current best model, the training result is considered the current best model. After all training rounds are completed, the best model is obtained. Finally, the best model is tested on the test set to obtain the final experimental results. The performance of the segmentation model is evaluated using the Dice similarity coefficient, and the results are measured using MSE, RMSE, and R-squared. 2 The performance of the ejection fraction prediction model is evaluated. A higher Dice value indicates a higher similarity between the segmentation result and the annotation; lower MSE and RMSE indicate smaller prediction errors; and R... 2 A larger value indicates a better goodness of fit in the prediction. Experimental results are as follows... Figure 5 and Figure 6 As shown.

[0069] Specifically, whether or not a model M undergoes mask-based adaptive scale alignment will be considered. best With M best-roughIn comparison, according to experimental results, the segmentation results of this invention achieved Dice of 91.51% and 93.55% on ED and ES frames, respectively, with a comprehensive average segmentation accuracy of 92.76%. Compared with the unoptimized training model, this invention improved segmentation accuracy by 1.12% and 0.59%, respectively, demonstrating that the multi-frame alignment-based optimization training strategy has a significant effect on improving model performance. Furthermore, this invention also shows significant improvements compared to other models. Figure 5 As shown, the first set of comparison images demonstrates that the segmentation results based on the ASA method have smaller errors and are more accurate; the second set of comparison images shows that even in blurred ultrasound images, M... best It can still accurately segment the left ventricular region and achieve more stable results.

[0070] Specifically, we compare our method with and without the R(2+1)d model that performs mask-based adaptive scale alignment, and with the current state-of-the-art model. Our method performs better in MAE, RMSE, and R... 2 The results achieved were 3.92, 5.16, and 0.822 on the three metrics, respectively, outperforming state-of-the-art work. Compared to the unenhanced results of 4.05, 5.32, and 0.81, our method demonstrates a significant advantage. Figure 6 As shown, Figure 6 (a) shows the training loss decrease curves for different models, while Figure 6 (b) shows the validation loss reduction curves for different models. Green represents the unenhanced R(2+1)d model, blue represents the best-performing EchoCoTr model from previous work, and orange represents the R(2+1)d model using the ASA′ method. The ASA′ method's training loss falls between that of EchoCoTr and the unenhanced R(2+1)d model, while achieving the lowest validation loss. The final experimental results demonstrate that the mask-adaptive scale alignment-based model effectively reduces overfitting and enhances generalization ability.

[0071] In summary, this invention provides a cardiac function assessment method based on mask adaptive scale alignment. This method can reduce segmentation errors between different frames when video annotations are limited, helping to minimize segmentation differences between frames with limited data. Furthermore, it allows for left ventricular segmentation without altering the model architecture, directly using video as input. This not only improves the accuracy of left ventricular segmentation in each frame but also provides better real-time guidance and positioning information for the left ventricle. Additionally, it enhances the model's generalization ability when performing ejection fraction prediction tasks, achieving more accurate predictions and significant performance improvements.

[0072] Although the present invention has been described in detail through the preferred embodiments above, it should be understood that the above description should not be considered as a limitation of the present invention. Various modifications and substitutions to the present invention will be apparent to those skilled in the art after reading the above description. Therefore, the scope of protection of the present invention should be defined by the appended claims.

Claims

1. A cardiac function assessment method based on mask adaptive scale alignment, characterized in that, Includes the following steps: S1. Train a left ventricular segmentation model using the original dataset to locate and segment the left ventricular region to generate alignment factors; S2. Using the alignment factor, construct an interpolated aligned dataset and a variant dataset based on the original dataset; S3. Optimize the left ventricular segmentation model and reconstruct the segmentation results using the interpolated and aligned dataset; S4. Expand the variant dataset to obtain the flipped dataset, and train the ejection fraction prediction model using the variant dataset and the flipped dataset; S5. Analysis of experimental results; In S1, two labeled frames from each original dataset's echocardiogram video and their corresponding labeling information are selected for cross-entropy loss, and this process is repeated several times to obtain an M. best-rough The model outputs a segmentation mask that generates adaptive detection boxes for each frame of image, serving as an alignment factor. In S2, for the left ventricular segmentation task, an adaptive scale alignment mechanism is used to construct an interpolated alignment dataset. By applying adaptive scale alignment and interpolation to the original dataset, the different detection boxes of the left ventricular region in each frame are enlarged to a uniform scale to obtain the interpolated alignment dataset. In the interpolated alignment dataset, the left ventricular regions of different frames are aligned in terms of spatial distribution and size. In S2, for the ejection fraction prediction task, a variant dataset is constructed using a variant of the adaptive scale alignment mechanism. By applying the variant operation of adaptive scale alignment to the original dataset, the left ventricular regions of different videos are spatially aligned without changing the relative size of key regions in different frames of the same video, so as to obtain the variant dataset.

2. The cardiac function assessment method based on mask adaptive scale alignment as described in claim 1, characterized in that, In S3, the interpolated dataset is used to re-align the M. best-rough The model undergoes several rounds of optimization training to obtain the optimized model M. best .

3. The cardiac function assessment method based on mask adaptive scale alignment as described in claim 2, characterized in that, The model M best The segmentation mask obtained from the prediction comes from the interpolation alignment dataset. Therefore, the final prediction result needs to be subjected to an inverse transformation of the adaptive scale alignment mechanism to obtain the accurate segmentation result before the adaptive scale alignment mechanism.

4. The cardiac function assessment method based on mask adaptive scale alignment as described in claim 1, characterized in that, In S4, the variant dataset and the flipped dataset are used as inputs to the model, the R(2+1)d model is used to learn features, and then the ejection fraction is predicted.

Images