Left ventricular ejection fraction prediction device

Abstract

The application discloses a left ventricular ejection fraction prediction device, comprising: a data input unit, an ultrasound image feature representation unit, a contrast learning unit based on multi-section two-dimensional echocardiogram and a left ventricular ejection fraction result prediction unit; the contrast learning unit comprises: an image representation module, a contrast image representation module, a negative sample queue storage module and a feature contrast module; the left ventricular ejection fraction result prediction unit is used for predicting the patient representation h k The combination is carried out and regression fitting is carried out to obtain a prediction result. The left ventricular ejection fraction prediction device can extract left ventricular related features from multiple different sections of two-dimensional echocardiograms of patients, improve ejection fraction prediction and application performance in a clinical environment, reduce the workload of professional data labeling, help better carry out corresponding clinical research, and help patients obtain better diagnosis, treatment and prognosis nursing.

Description

Technical Field

[0001] This invention relates to a device for predicting left ventricular ejection fraction. Background Technology

[0002] Medical image data annotation is highly specialized, making the acquisition of medical image datasets a persistent challenge for the application of supervised deep learning techniques. Two-dimensional echocardiography is widely used in cardiac examinations due to its low cost and high efficiency; the left ventricle, as the heart's primary pumping chamber, has an ejection fraction that is a crucial indicator of cardiac systolic function. Left ventricular ejection fraction measurements are significantly influenced by the measurer, measurement method, and ultrasound imaging modality. Accurate assessment of this indicator is vital for the early detection, prevention, diagnosis, treatment, and prognosis of cardiovascular diseases in patients.

[0003] Clinically, one or two sections of two-dimensional echocardiography are often selected, with the apical four-chamber view being the most important for measuring left ventricular ejection fraction. To improve the accuracy of left ventricular ejection fraction measurement, deep learning technology currently focuses on two main research directions:

[0004] (1) Ventricular segmentation. Ventricular segmentation involves deep learning target detection and segmentation of the left ventricle using a sample of the patient's apical four-chamber view, followed by measurement of the left ventricular end-systolic and end-diastolic volumes using traditional automatic measurement methods, and calculation of the left ventricular ejection fraction.

[0005] (2) Direct prediction. Direct prediction involves directly inputting the patient's apical four-chamber ultrasound image into a three-dimensional convolutional neural network for model training and prediction to obtain the left ventricular ejection fraction value.

[0006] While both ventricular segmentation and direct prediction have their unique advantages, they also share significant limitations: ventricular segmentation accurately pinpoints the location of the segmented left ventricle and offers strong interpretability, but it suffers from substantial cumulative error when used to measure left ventricular ejection fraction; direct prediction is convenient, fast, and highly accurate, but lacks interpretability. A common problem with both is that they typically use only one two-dimensional ultrasound plane, and there is no automated ultrasound plane classification method available clinically. This makes preliminary data preparation time-consuming and laborious, limiting their application performance in clinical practice. Summary of the Invention

[0007] This invention provides a left ventricular ejection fraction prediction device, which adopts the following technical solution:

[0008] A left ventricular ejection fraction prediction device includes: a data input unit, an ultrasound image feature representation unit, a contrastive learning unit based on multi-plane two-dimensional echocardiography, and a left ventricular ejection fraction result prediction unit;

[0009] The data input unit is used to input training data into the contrastive learning unit and the left ventricular ejection fraction prediction unit based on multi-plane two-dimensional echocardiography for training. The training data includes the source dataset. Where N represents the total number of patients in the source dataset D, and each patient has K ultrasound cross-sectional images x = (f 0 ,f 1 ,…,f F-1 ) and the true label y, f, representing the clinical measurement result of left ventricular ejection fraction. i F represents a single frame of an image, and F is a constant representing the total number of frames in the image.

[0010] The contrastive learning unit based on multi-plane two-dimensional echocardiography includes:

[0011] The image representation module is used to extract patient ultrasound cross-sectional images from the source dataset D. Feature representation h k ;

[0012] The contrast image representation module is used to extract feature representations h of ultrasound cross-sectional images used for comparison in a small batch during training. k′ ;

[0013] The negative sample queue storage module is used to store all negative sample feature representations required during a single parameter update of the model. Where m is a constant representing the queue size;

[0014] A feature comparison module based on multi-view two-dimensional echocardiography is used to calculate positive and negative similarities between the feature representations of different ultrasound view images of a patient and the feature representations of negative samples belonging to other patients. And by comparing and optimizing the model parameters w of the image representation module q This updates the model parameters w of the contrast image representation module. k ;

[0015] The left ventricular ejection fraction result prediction unit is used to predict the patient representation output by the image representation module. k By combining and regressing the data, the predicted values ​​of the left ventricular ejection fraction are obtained.

[0016] Furthermore, both the image representation module and the contrast image representation module consist of an ultrasound image feature representation unit connected to a projection layer. The contrast image representation module extracts the corresponding feature representation h from the training data input by the data input unit. k and h k′ and represent the features h k and h k′ The representation used for comparison is obtained through the projection layer, and the calculation formula is as follows:

[0017]

[0018]

[0019] Among them, the fully connected layer fc I A linear mapping of vectors with invariant dimension, fc low The vectors are dimensionality reduced by mapping, and R is a ReLU layer to enhance nonlinear fitting ability. The model structures of the image representation module and the contrast image representation module are exactly the same. The parameters of the former are updated by gradient descent, while the parameters of the latter are updated according to the parameter update results of the former in each training.

[0020] Furthermore, the negative sample queue storage module pushes the representations of the current mini-batch training data from the image representation module into the queue each time the model parameters are updated, and dequeues the same number of the earliest entered representations.

[0021] Furthermore, the feature comparison module based on multi-section two-dimensional echocardiography treats the feature vectors obtained from different ultrasound images of the same patient through the image representation module as positive sample groups, and treats all feature vectors in the negative sample queue as corresponding negative samples. The similarity value between sample features is calculated by vector inner product, and the calculation formula is as follows:

[0022]

[0023]

[0024] Positive sample similarity is calculated using feature vectors obtained from the image representation module and the contrast image representation module, while negative sample similarity is calculated using feature vectors obtained from the image representation module and feature vectors from the negative sample queue storage module. The specific training method is as follows:

[0025] Using InfoNCE loss as the contrastive loss function.

[0026]

[0027] in Let τ be the loss function for the contrastive learning unit based on multi-view two-dimensional echocardiography, and τ be the temperature coefficient, where a smaller value indicates a higher degree of attention to difficult negative samples. It is the formula for permutations;

[0028] Optimize the loss function using the SGDM algorithm Each optimization yields new image representation module model parameters w. q Then, the model parameters w of the contrast image representation module are updated based on the momentum parameter k. k The calculation formula is as follows:

[0029] w k =kw k +(1-k)w q ,

[0030] The parameter update formula applies to every specific parameter of the model.

[0031] Furthermore, the left ventricular ejection fraction prediction unit consists of an ultrasound image feature representation unit and a regression head connected together. The ultrasound image feature representation unit shares parameters with the image representation module of the contrastive learning unit based on multi-section two-dimensional echocardiography. The feature representations of all ultrasound images of a patient are summed and averaged, and then input into the regression head to obtain the predicted left ventricular ejection fraction value. The calculation formulas for the averaging and regression processes are as follows:

[0032]

[0033] Where R is a nonlinear ReLU layer, and fc is a fully connected layer. to1 The specific method for mapping the combined patient-level feature vector to a single value and training it is as follows:

[0034] Use SmoothL1 loss as the regression loss function.

[0035]

[0036] in The loss function for the left ventricular ejection fraction prediction unit is ef. i and y i These are the model's predicted value and the actual value, respectively.

[0037] Furthermore, the ultrasound image feature representation unit includes:

[0038] The spatial feature extraction module is used to extract spatial features from each frame of echocardiogram images and convert them into vector representations through training of a residual neural network;

[0039] The temporal feature extraction module is used to combine the spatial feature vectors extracted from echocardiogram images through spatial feature extraction and then extract temporal features through the training of the Transformer encoder deep network, transforming them into vector representations.

[0040] Furthermore, the spatial feature extraction module consists of a set of residual neural networks with shared parameters, where each image frame f of a patient's ultrasound cross-section image... i Spatial feature vectors are extracted using a residual neural network. And combine them into a spatial feature matrix H s The residual neural network consists of 5 residual blocks and a fully connected layer, and is calculated using the following formula:

[0041]

[0042] Specifically, conv1 uses 64 convolutional kernels of size 7×7 with a stride of 2 and max-conv downsampling of size 3×3 with a stride of 2. 2～5 Each consists of two basic blocks, bb, which are calculated using the following formula:

[0043] conv i =x+R(bb(x))+R(bb(x+R(bb(x)))),

[0044] bb(x)=(n×3×3conv+BN)[(n×3×3conv+BN+R)(x)],

[0045] Where n represents the number of 3×3 convolution kernels used in a certain residual block, which are 64, 128, 256, and 512 in conv2-5 respectively. Standardization refers to the process of standardizing the mean and variance of corresponding positional data in a batch during training. Here, I represents the ReLU activation function, and I represents the input to the residual neural network at a certain position during the forward propagation.

[0046] The spatial feature matrix is ​​obtained by adding the location code p and the regression vector r to each spatial feature vector obtained from an ultrasound cross-sectional image, and then combining them in the following form:

[0047]

[0048]

[0049] Where, p i，j In this context, i represents the matrix row number, j represents the matrix column number, d represents the dimension of the input vector, and H... s Each comma-separated part is a 1×d vector, which is finally combined into a (F+1)×d matrix. The spatial feature matrix serves as the input to the temporal feature extraction module.

[0050] The temporal feature extraction module consists of a series of consecutive Transformer encoder blocks. Each Transformer encoder block includes a multi-head self-attention part (MHA) and a feedforward neural network (FFN). The entire module is constructed using residual connections and is calculated using the following formula:

[0051]

[0052] The LN representation layer normalizes each row of the same spatial feature matrix. It consists of M Transformer encoder blocks connected sequentially. In the multi-head self-attention part, each self-attention head uses scaled dot product attention, calculated as follows:

[0053] Q = W Q ·H s K = W K ·H s V = W V ·H s ,

[0054]

[0055] Each self-attention point has a unique set of (F+1)×(F+1) W. Q W K W V The transformation matrix is ​​used to extract different temporal information during the self-attention calculation process. The attention weights of the weight matrix V are calculated using the softmax function. Make the sum of the weights at each position equal to 1, d k Represents matrix QK T The dimension of the feedforward neural network is calculated using the following formula:

[0056]

[0057] The fully connected layer fc1 reduces the dimensionality of the matrix, fc2 restores the matrix to its original dimension, and R is a ReLU layer that enhances nonlinear fitting capabilities, outputting the matrix... Corresponding spatial characteristic matrix H s The vector h at the position of the regression vector r k As a patient's ultrasound image at a certain cross section The characteristics are represented.

[0058] Furthermore, the specific method for the data input unit to train the contrastive learning unit and the left ventricular ejection fraction prediction unit based on multi-section two-dimensional echocardiography is as follows:

[0059] The source dataset D is divided into training set, validation set and test set according to a certain ratio.

[0060] The training set of the source dataset D is input into the contrastive learning unit and the left ventricular ejection fraction result prediction unit based on multi-section two-dimensional echocardiography, and they are jointly trained.

[0061] Input the validation set of the source dataset D into the left ventricular ejection fraction prediction unit, and compare the output prediction results with the true values ​​to validate them.

[0062] Furthermore, the total loss function based on the joint training of the contrastive learning unit and the left ventricular ejection fraction prediction unit based on multi-section two-dimensional echocardiography is... for:

[0063]

[0064] Where α is a hyperparameter.

[0065] Furthermore, the total loss function is optimized using the SGDM algorithm. In the total loss function Stop training when the temperature stops dropping.

[0066] The advantage of this invention lies in the fact that the provided left ventricular ejection fraction prediction device, combined with clinical practice of echocardiography, can extract left ventricular-related features from multiple different sections of a patient's two-dimensional echocardiogram, thereby improving the performance of ejection fraction prediction and application in a clinical setting, reducing the workload of professional data annotation, helping to better conduct relevant clinical research, and helping patients receive better diagnosis, treatment, and prognosis care. Attached Figure Description

[0067] Figure 1 This is a model diagram of the left ventricular ejection fraction prediction device of the present invention. Detailed Implementation

[0068] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0069] like Figure 1 The image shows a left ventricular ejection fraction prediction device according to this application, including: a data input unit, a contrast learning unit based on multi-plane two-dimensional echocardiography, and a left ventricular ejection fraction result prediction unit.

[0070] The data input unit is used to input training data into the contrast learning unit and the left ventricular ejection fraction result prediction unit based on multi-section two-dimensional echocardiography to train them.

[0071] Training data includes the source dataset Where N represents the total number of patients in the source dataset D, and each patient has K ultrasound cross-sectional images x = (f 0 ,f 1 ,…,f F-1 ) and the true label y, f, representing the clinical measurement result of left ventricular ejection fraction. i F represents one frame of a certain image, and F is a constant representing the total number of frames in the image.

[0072] The contrastive learning unit based on multi-view two-dimensional echocardiography includes: an image representation module, a contrastive image representation module, a negative sample queue storage module, and a feature contrast module based on multi-view two-dimensional echocardiography.

[0073] The image representation module is used to extract patient ultrasound cross-sectional images from the source dataset D. Feature representation h k The contrastive image representation module is used to extract feature representations h of ultrasound cross-sectional images used for contrast in a small batch during training. k′ The negative sample queue storage module is used to store all negative sample feature representations required during a single parameter update of the model. Where m is a constant representing the queue size. The feature comparison module based on multi-view two-dimensional echocardiography is used to calculate positive and negative similarity based on the feature representations of different ultrasound view images of a patient and the feature representations of negative samples belonging to other patients. And by comparing and optimizing the model parameters w of the image representation module q This updates the model parameters w of the contrast image representation module. k .

[0074] The left ventricular ejection fraction result prediction unit is used to predict the patient representation output by the image representation module. k By combining and regressing the data, the predicted values ​​of the left ventricular ejection fraction are obtained.

[0075] In a preferred embodiment, both the image representation module and the contrast image representation module consist of an ultrasound image feature representation unit connected to a projection layer. The contrast image representation module extracts the corresponding feature representation h from the training data input by the data input unit. k and h k′ and represent the features h k and h k′ The representation used for comparison is obtained through the projection layer, and the calculation formula is as follows:

[0076]

[0077]

[0078] Among them, the fully connected layer fc I A linear mapping of vectors with invariant dimension, fc low The vectors are dimensionality reduced by mapping, and R is a ReLU layer to enhance nonlinear fitting ability. The model structures of the image representation module and the contrast image representation module are exactly the same. The parameters of the former are updated by gradient descent, while the parameters of the latter are updated according to the parameter update results of the former in each training.

[0079] In a preferred implementation, the negative sample queue storage module pushes the representations of the current mini-batch training data from the image representation module into the queue each time the model parameters are updated, and dequeues the same number of the earliest entries.

[0080] As a preferred implementation, the feature comparison module based on multi-section two-dimensional echocardiography treats the feature vectors obtained from different ultrasound images of the same patient through the image representation module as positive sample groups, and treats all feature vectors in the negative sample queue as corresponding negative samples. The similarity value between sample features is calculated by vector inner product, and the calculation formula is as follows:

[0081]

[0082]

[0083] Positive sample similarity is calculated using feature vectors obtained from the image representation module and the contrast image representation module, while negative sample similarity is calculated using feature vectors obtained from the image representation module and feature vectors stored in the negative sample queue module. The feature comparison module based on multi-section two-dimensional echocardiography is based on the assumption that the image feature representations of the same patient should be closer, and its goal is to maximize the similarity between positive samples and minimize the similarity between positive and negative samples. The specific method for training it is as follows:

[0084] Using InfoNCE loss as the contrastive loss function.

[0085]

[0086] in Let τ be the loss function for the contrastive learning unit based on multi-view two-dimensional echocardiography, and τ be the temperature coefficient, where a smaller value indicates a higher degree of attention to difficult negative samples. It is the formula for permutations.

[0087] Optimize the loss function using the SGDM algorithm Each optimization yields new image representation module model parameters w. q Then, the model parameters w of the contrast image representation module are updated based on the momentum parameter k. k The calculation formula is as follows:

[0088] w k =kw k +(1-k)w q ,

[0089] The parameter update formula applies to every specific parameter of the model.

[0090] In a preferred embodiment, the left ventricular ejection fraction prediction unit consists of the aforementioned ultrasound image feature representation unit and a regression head. The ultrasound image feature representation unit shares parameters with the image representation module of the contrastive learning unit based on multi-sectional two-dimensional echocardiography. The feature representations of all ultrasound images of a patient are summed and averaged before being input into the regression head to obtain the predicted left ventricular ejection fraction value. The calculation formulas for the averaging and regression processes are as follows:

[0091]

[0092] Where R is a nonlinear ReLU layer, and fc is a fully connected layer. to1 The specific method for mapping the combined patient-level feature vector to a single value and training it is as follows:

[0093] Use SmoothL1 loss as the regression loss function.

[0094]

[0095] in The loss function for the left ventricular ejection fraction prediction unit is ef. i and y i These are the model's predicted value and the actual value, respectively.

[0096] The total loss function based on the joint training of the contrastive learning unit and the left ventricular ejection fraction prediction unit of multi-view two-dimensional echocardiography for:

[0097]

[0098] Where α is a hyperparameter.

[0099] In this application, the total loss function is optimized using the SGDM algorithm. In the total loss function Stop training when the temperature stops dropping.

[0100] Specifically, the aforementioned ultrasound image feature representation unit includes a spatial feature extraction module and a temporal feature extraction module.

[0101] The spatial feature extraction module is used to extract spatial features from each frame of echocardiogram images and convert them into vector representations through training of a residual neural network.

[0102] The temporal feature extraction module is used to combine the spatial feature vectors extracted from echocardiogram images and then extract temporal features through training a Transformer encoder deep network, transforming them into vector representations.

[0103] In a preferred embodiment, the spatial feature extraction module consists of a set of residual neural networks with shared parameters, where each image frame f of a patient's ultrasound cross-section image... i Spatial feature vectors are extracted using a residual neural network. And combine them into a spatial feature matrix H s The residual neural network consists of 5 residual blocks and a fully connected layer, and is calculated using the following formula:

[0104]

[0105] Specifically, conv1 uses 64 convolutional kernels of size 7×7 with a stride of 2 and max-conv downsampling of size 3×3 with a stride of 2. 2～5 Each consists of two basic blocks, bb, which are calculated using the following formula:

[0106] conv i =x+R(bb(x))+R(bb(x+R(bb(x)))),

[0107] bb(x)=(n×3×3conv+BN)[(n×3×3conv+BN+R)(x)],

[0108] Where n represents the number of 3×3 convolution kernels used in a certain residual block, which are 64, 128, 256, and 512 in conv2-5 respectively. Standardization refers to the process of standardizing the mean and variance of corresponding positional data in a batch during training. Here, I represents the ReLU activation function, and I represents the input to the residual neural network at a certain position during the forward propagation.

[0109] The spatial feature matrix is ​​obtained by adding the location code p and the regression vector r to each spatial feature vector obtained from an ultrasound cross-sectional image, and then combining them in the following form:

[0110]

[0111]

[0112] Where, p i,j In this context, i represents the matrix row number, j represents the matrix column number, d represents the dimension of the input vector, and H... s Each comma-separated part is a 1×d vector, which is finally combined into a (F+1)×d matrix. The spatial feature matrix serves as the input to the temporal feature extraction module.

[0113] The temporal feature extraction module consists of a series of consecutive Transformer encoder blocks. Each Transformer encoder block includes a multi-head self-attention part (MHA) and a feedforward neural network (FFN). The entire module is constructed using residual connections and is calculated using the following formula:

[0114]

[0115] The LN representation layer normalizes each row of the same spatial feature matrix. It consists of M Transformer encoder blocks connected sequentially. In the multi-head self-attention part, each self-attention head uses scaled dot product attention, calculated as follows:

[0116] Q = W Q .H s K = W K .H s V = W v H s ,

[0117]

[0118] Each self-attention point has a unique set of (F+1)×(F+1) W. Q W K W V The transformation matrix is ​​used to extract different temporal information during the self-attention calculation process. The attention weights of the weight matrix V are calculated using the softmax function. Make the sum of the weights at each position equal to 1, d k Represents matrix QK T Dividing the dimension of the model by its square root can prevent gradient vanishing during model training. The calculation formula for a feedforward neural network is as follows:

[0119]

[0120] This part performs dimensionality scaling. The fully connected layer fc1 reduces the dimensionality of the matrix, and fc2 restores the matrix to its original dimensionality. R is a ReLU layer to enhance nonlinear fitting capabilities. The nonlinear ReLU layer enhances the feature fitting ability of the temporal feature extraction module. An optional random discard layer is used to mitigate overfitting during model training.

[0121] Output matrix Corresponding spatial characteristic matrix H s The vector h at the position of the regression vector r k As a patient's ultrasound image at a certain cross section The feature representation of h. Similarly, the ultrasound image feature representation unit can obtain h. k′ .

[0122] As a preferred implementation, the specific method for the data input unit to train the contrast learning unit and the left ventricular ejection fraction prediction unit based on multi-section two-dimensional echocardiography is as follows:

[0123] The source dataset D is divided into training set, validation set and test set according to a certain ratio.

[0124] The training set of the source dataset D is input into the contrastive learning unit and the left ventricular ejection fraction result prediction unit based on multi-section two-dimensional echocardiography, and they are jointly trained.

[0125] Input the validation set of the source dataset D into the left ventricular ejection fraction prediction unit, and compare the output prediction results with the true values ​​to validate them.

[0126] When applying this method, first, all echocardiographic images of a specific patient are processed. The spatiotemporal feature representation is obtained from the ultrasound image feature representation units in the input image representation module and the contrast image representation module. and The former inputs the regression head of the left ventricular ejection fraction prediction unit to obtain the predicted left ventricular ejection fraction value and calculates the regression loss.

[0127] Subsequently, the two representations are input into the projection heads of the image representation module and the contrast image representation module, respectively, to obtain the contrast feature representation. and The contrastive feature representations stored in the two representations and the negative sample queue storage module. The input is a feature comparison module based on multi-view 2D echocardiography. Similarity *s* between positive sample groups and between positive and negative samples is calculated to determine the contrast loss. The training objective of the contrast learning unit based on multi-view 2D echocardiography is to make the image feature representations of different views of the same patient as similar as possible in the feature space, thus aiding regression prediction results. The total loss is used to jointly optimize the training model and update the parameters *w* of the image representation module model. q Further momentum updates the parameters w of the image representation module model. k .

[0128] After the model training is completed, the patient samples of the validation dataset are input into the trained left ventricular ejection fraction prediction unit. The model unit can output the left ventricular ejection fraction prediction result by combining the features of different echocardiographic sections of the patient.

[0129] To better compare the superiority of the left ventricular ejection fraction prediction device proposed in this invention, taking the prediction of left ventricular ejection fraction in hospital patients with heart failure as an example, based on the real clinical multi-section two-dimensional echocardiogram dataset ZH2 extracted from a general cardiology ward of a hospital in China, the prediction performance of the model of this invention was compared with that of the existing model based on three-dimensional convolutional neural networks (3D-CNN) R(2+1)D on the dataset.

[0130] Experimental results are expressed using mean absolute error (MAE), root mean square error (RMSE), and R0. 2 As evaluation metrics, lower MAE and RMSE scores indicate more accurate predictions. 2 The larger the value, the better the fit to the overall test data. The specific calculation formula is as follows:

[0131]

[0132]

[0133]

[0134] Where f(x) i ) represents the model's predicted value, y i This is the actual measured value. This is the average of the measured values.

[0135] The final comparison results of the two models on the ZH2 dataset are shown in Table 1. It can be seen that the model of the present invention has a significantly better prediction effect on the ZH2 dataset.

[0136] Table 1. Performance of the model of this invention and existing models on the left ventricular ejection fraction regression task.

[0137]

[0138] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the above embodiments do not limit the present invention in any way, and all technical solutions obtained by equivalent substitution or equivalent transformation fall within the protection scope of the present invention.

Claims

1. A device for predicting left ventricular ejection fraction, characterized in that, include: The system includes a data input unit, a contrastive learning unit based on multi-plane two-dimensional echocardiography, and a left ventricular ejection fraction prediction unit. The data input unit is used to input training data into the contrast learning unit based on multi-section two-dimensional echocardiography and the left ventricular ejection fraction prediction unit for training. The training data includes the source dataset. ,in Represents the source dataset The CCP There are 10 patients, each patient carries 100 patients. Ultrasound cross-sectional image and the true label representing the clinical measurement result of left ventricular ejection fraction , Represents a single frame of a video. It is a fixed value representing the total number of frames in the image; The contrastive learning unit based on multi-plane two-dimensional echocardiography includes: The image representation module is used to extract the source dataset. Ultrasound cross-sectional images of patients Feature representation ; The contrastive image representation module is used to extract feature representations of ultrasound cross-sectional images used for comparison in a small batch during training. ; The negative sample queue storage module is used to store all negative sample feature representations required during a single parameter update of the model. ,in This is a fixed value, representing the queue size; A feature comparison module based on multi-view two-dimensional echocardiography is used to calculate positive and negative similarities between the feature representations of different ultrasound view images of a patient and the feature representations of negative samples belonging to other patients. And by comparing and optimizing the model parameters of the image representation module. This updates the model parameters of the contrast image representation module. ; The left ventricular ejection fraction prediction unit is used to analyze the patient representation output by the image representation module. By combining and regressing the data, the predicted values ​​of the left ventricular ejection fraction are obtained. Both the image representation module and the contrast image representation module consist of an ultrasound image feature representation unit and a projection layer connected together. The contrast image representation module extracts the corresponding feature representations from the training data input by the data input unit. and and represent the features and The representation used for comparison is obtained through the projection layer, and the calculation formula is as follows: ， ， Among them, fully connected layer Perform a linear mapping on a vector that does not change its dimension. Perform dimensionality reduction mapping on the vector. To enhance the nonlinear fitting capability of the ReLU layer, the model structure of the image representation module and the contrast image representation module is exactly the same. The parameters of the former are updated through gradient descent, while the parameters of the latter are updated according to the parameter update results of the former in each training iteration.

2. The left ventricular ejection fraction prediction device according to claim 1, characterized in that, The negative sample queue storage module pushes the representation of the current mini-batch training data by the contrast image representation module into the queue each time the model parameters are updated, and removes the same number of the earliest representations that were entered into the queue.

3. The left ventricular ejection fraction prediction device according to claim 2, characterized in that, The feature comparison module based on multi-section two-dimensional echocardiography treats the feature vectors obtained from different ultrasound images of the same patient through the image representation module as positive sample groups, and treats all feature vectors in the negative sample queue as corresponding negative samples. The similarity value between sample features is calculated by vector inner product, and the calculation formula is as follows: ， ， The positive sample similarity is calculated using the feature vectors obtained from the image representation module and the contrast image representation module, while the negative sample similarity is calculated using the feature vectors obtained from the image representation module and the feature vectors in the negative sample queue storage module. The specific method for training them is as follows: Using InfoNCE loss as the contrastive loss function. ， in Let be the loss function of the contrastive learning unit based on multi-plane two-dimensional echocardiography. This is a temperature coefficient; the smaller the value, the higher the attention given to difficult negative samples. It is the formula for permutations; Optimize the loss function using the SGDM algorithm Each optimization yields new model parameters for the image representation module. Then, based on momentum parameters Update the model parameters of the contrast image representation module. The calculation formula is as follows: ， The parameter update formula applies to every specific parameter of the model.

4. The left ventricular ejection fraction prediction device according to claim 3, characterized in that, The left ventricular ejection fraction prediction unit consists of an ultrasound image feature representation unit and a regression head. The ultrasound image feature representation unit shares parameters with the image representation module of the contrastive learning unit based on multi-section two-dimensional echocardiography. The feature representations of all ultrasound images of a patient are summed and averaged, and then input into the regression head to obtain the predicted left ventricular ejection fraction value. The calculation formulas for the averaging and regression processes are as follows: ， in, It is a nonlinear ReLU layer, a fully connected layer. The specific method for mapping the combined patient-level feature vector to a single value and training it is as follows: Use SmoothL1 loss as the regression loss function. ， in The loss function of the left ventricular ejection fraction prediction unit is... and These are the model's predicted value and the actual value, respectively.

5. The left ventricular ejection fraction prediction device according to claim 4, characterized in that, The ultrasound image feature representation unit includes: The spatial feature extraction module is used to extract spatial features from each frame of echocardiogram images and convert them into vector representations through training of a residual neural network; The temporal feature extraction module is used to combine the spatial feature vectors of the echocardiogram images after spatial feature extraction and then extract temporal features through the training of the Transformer encoder deep network, and convert them into vector representations.

6. The left ventricular ejection fraction prediction device according to claim 5, characterized in that, The spatial feature extraction module consists of a set of residual neural networks with shared parameters, for each image frame of a patient's ultrasound cross-section. Spatial feature vectors are extracted using a residual neural network. And combine them into a spatial feature matrix The residual neural network consists of 5 residual blocks and a fully connected layer, and is calculated using the following formula: ， in, Use 64 sizes A convolutional kernel with a stride of 2 and Maximum convergence downsampling with a step size of 2 Both consist of two basic blocks, namely The composition is calculated using the following formula: ， ， in, Indicates the use of a certain residual block The number of convolution kernels, in The numbers are 64, 128, 256, and 512 respectively. Standardization refers to the process of standardizing the mean and variance of corresponding positional data in a batch during training. It is the ReLU activation function. This represents the input at a certain position during the forward propagation of the residual neural network. Add a position code to each spatial feature vector obtained from an ultrasound cross-section image. and regression vector The resulting spatial feature matrix is ​​then combined in the following manner: ， ， in, middle Indicates the row number of the matrix. Indicates the column number of the matrix. This represents the dimension of the input vector. Each comma-separated part is 1 The vectors are ultimately combined to form The spatial feature matrix is ​​used as the input to the temporal feature extraction module. The temporal feature extraction module consists of a series of consecutive Transformer encoder blocks, each of which includes a multi-head self-attention part. and a feedforward neural network The entire structure is built using residual connections, and the calculation is performed using the following formula: ， in, Representation layer standardization involves standardizing each row of the feature matrix in the same space. The Transformer encoder blocks are sequentially connected. In the multi-head self-attention part, each self-attention head uses scaled dot product attention, calculated as follows: ， ， Each self-focused head has a unique set of... of The transformation matrix is ​​used to extract different temporal information during the self-attention calculation process. Calculate the weight matrix Attention weights, through function Make the sum of the weights at each position equal to 1. Representation matrix The dimension of the feedforward neural network is calculated using the following formula: ， Fully connected layer Dimensionality reduction of the matrix To restore the matrix to its original dimension, To enhance the nonlinear fitting capability of the ReLU layer, the output matrix is... Corresponding spatial feature matrix Middle regression vector position vector As a patient's ultrasound image at a certain cross section The characteristics are represented.

7. The left ventricular ejection fraction prediction device according to claim 6, characterized in that, The specific method for the data input unit to input training data into the contrast learning unit based on multi-section two-dimensional echocardiography and the left ventricular ejection fraction result prediction unit for training is as follows: Source dataset The dataset is divided into training, validation, and test sets according to a certain ratio. Source dataset The training set is input into the contrast learning unit based on multi-section two-dimensional echocardiography and the left ventricular ejection fraction result prediction unit, and they are jointly trained. Source dataset The validation set is input into the left ventricular ejection fraction prediction unit, and the output prediction result is compared with the true value to verify it.

8. The left ventricular ejection fraction prediction device according to claim 7, characterized in that, The total loss function of the comparison learning unit based on multi-section two-dimensional echocardiography and the left ventricular ejection fraction prediction unit jointly trained for: ， in This is a hyperparameter.

9. The left ventricular ejection fraction prediction device according to claim 8, characterized in that, Optimize the total loss function using the SGDM algorithm. In the total loss function Stop training when the temperature stops dropping.

Images