A Deep Learning-Based Method for Detecting Flow Fields in the Left Atrium and Left Atrial Appendage

Abstract

This invention discloses a deep learning-based method for detecting the flow field within the left atrium and left atrial appendage. A deep learning model is constructed, and a point cloud dataset is input into the model for training. After training, the point cloud data to be tested is processed to obtain the flow field detection results. Specifically, the deep learning model converts the wall point cloud and the internal point cloud into two-dimensional tensors, which are then input into the wall point cloud channel and the internal point cloud channel respectively to obtain global and local geometric information. These are then concatenated into a single tensor and input into the convergence channel for further processing to obtain the flow field results. This invention, through computer data processing, enables non-invasive and non-surgical detection, and can obtain hemodynamic parameters in real time. It has the advantages of low computational cost, high speed, and high accuracy, while ensuring high precision and predictive accuracy.

Description

Technical Field

[0001] This invention relates to a method for obtaining flow field data in the field of biomedical engineering, and more particularly to a method for obtaining hemodynamic parameters of the atria and atrial appendages based on deep learning. Background Technology

[0002] Currently, the gold standard for assessing the risk of left atrial appendage thrombosis is based on transesophageal echocardiography (TEE). However, this method is invasive, difficult and costly for patients, and can only measure local flow data in a plane or from a given perspective, thus failing to provide a complete assessment of the complex three-dimensional flow field within the left atrial appendage. Later, 4D Flow MRI began to be used to acquire flow field data of the left atrium and left atrial appendage; however, its direct measurements are insufficient in both temporal and spatial resolution, leading to the loss of flow field details. Furthermore, computational fluid dynamics (CFD) is a more general and accurate method for acquiring flow field details. Current research directions applying CFD to left atrial thrombosis include: the influence of different atrial appendage geometries on the left atrial flow field, the influence of different thrombosis treatment regimens on the left atrial flow field, and the relationship between left atrial hemodynamic parameters and thrombosis formation. While traditional CFD methods can provide high-resolution 3D hemodynamic parameters within the left atrium based on medical image reconstruction, they require significant memory resources, time-consuming computations, and tedious preprocessing, such as mesh generation and boundary condition setting, hindering their clinical application. Therefore, obtaining hemodynamic parameters, including velocity and pressure, in an efficient and non-invasive manner is crucial for rapidly assessing the thrombotic risk in patients with atrial fibrillation.

[0003] Machine learning is now widely used in hemodynamics. Different CFD numerical simulation steps have corresponding machine learning techniques, including reconstructing 3D models of cardiovascular structures from medical images, automatic mesh generation and mesh quality assessment, and numerical simulation calculations. However, current research mostly focuses on two-dimensional flow fields or 3D flows with ideal geometry, with insufficient research on complex geometries. To improve the accuracy of predictions in non-Euclidean domains, various geometric deep learning (GDL) models have been proposed, such as PointNet and PointNet++, which can directly process point cloud data using spatial transformation neural networks (T-net) and symmetric operations. However, research on applying machine learning to obtain hemodynamic parameters of the left atrium and left atrial appendage is still relatively limited. Summary of the Invention

[0004] To address the problems existing in the background technology, the present invention provides a deep learning-based method for detecting the flow field in the left atrium and left atrial appendage. The method can calculate the instantaneous three-dimensional velocity field and pressure field of the left atrium and left atrial appendage of a patient with atrial fibrillation using a geometric model of the two organs, thereby providing a more comprehensive analysis of the flow field in the left atrium and left atrial appendage and assisting in assessing the risk of thrombosis in patients with atrial fibrillation.

[0005] The technical solution adopted in this invention is:

[0006] 1) Construct a deep learning model;

[0007] 2) Input the point cloud dataset into the deep learning model for training;

[0008] 3) Use the trained deep learning model to perform detection and prediction processing on the point cloud data to be tested to obtain the flow field detection results.

[0009] The flow field mentioned includes a pressure field or a velocity field.

[0010] The point cloud dataset is divided into wall point clouds and interior point clouds. The point cloud data to be tested also includes wall point clouds and interior point clouds.

[0011] Wall point cloud refers to the lattice set of points on the outermost surface of a geometric model of the left atrium;

[0012] Internal point cloud refers to the set of points inside a geometric model of the left atrium, excluding the surface.

[0013] like Figure 1 As shown, the deep learning model specifically includes two parallel channels and one convergence channel. The two parallel channels are the wall point cloud channel and the internal point cloud channel, respectively. The input wall point cloud and internal point cloud are both converted into two-dimensional tensors. The two-dimensional tensors of the wall point cloud and internal point cloud are input into the wall point cloud channel and the internal point cloud channel, respectively, to obtain global geometric information (corresponding to the wall point cloud channel) and local geometric information (corresponding to the internal point cloud channel) containing a left atrium model. The global geometric information (corresponding to the wall point cloud channel) and the local geometric information are concatenated along the dimension of the number of points to form a tensor, which is then input into the convergence channel for processing to obtain the flow field result.

[0014] In the deep learning model described above, the global geometric information output from the wall point cloud channel and the local geometric information output from the internal point cloud channel are concatenated along the dimension of the number of points to form a tensor.

[0015] The wall point cloud channel includes a spatial transformation layer, three convolutional layers, two edge convolutional layers, and a max pooling layer. The wall point cloud first undergoes spatial transformation through the spatial transformation layer to scale and rotate into a standard point cloud. Then, it is input into two consecutive sets of convolutional modules to extract local features of the wall point cloud. Each set of convolutional modules consists of an edge convolutional layer and a convolutional layer connected in sequence. Finally, the local features of the wall point cloud are processed by a convolutional layer and a max pooling layer in sequence and output to obtain global geometric information.

[0016] The internal point cloud channel includes a spatial transformation layer, three convolutional layers, and two edge convolutional layers. The internal point cloud first undergoes spatial transformation through the spatial transformation layer to scale and rotate into a standard point cloud. Then, it is input into two consecutive sets of convolutional modules to extract local features of the internal point cloud. Each set of convolutional modules consists of an edge convolutional layer and a convolutional layer connected in sequence. Finally, the local features of the internal point cloud are processed by a convolutional layer and output to obtain local geometric information.

[0017] The two sets of convolutional modules in the wall point cloud channel and the internal point cloud channel constitute a topology structure in which edge convolutional layers and convolutional layers appear alternately.

[0018] The edge convolutional layers in the wall point cloud channel and the internal point cloud channel specifically refer to convolution operations based on the adjacency relationship of the point cloud.

[0019] The beneficial effects of this invention are:

[0020] The method of this invention is non-invasive and has low requirements for clinical diagnosis. It only requires patients to provide common non-invasive data (such as cardiovascular CTA medical images) to calculate hemodynamic parameter solutions with certain specificity.

[0021] The method of this invention has the advantages of low computational cost, high speed, and high accuracy, and can obtain hemodynamic parameters in real time while ensuring accuracy;

[0022] Compared to other classic deep learning models, the method of this invention has higher prediction accuracy;

[0023] Compared to other studies on thrombosis risk assessment, the method of this invention can obtain more comprehensive three-dimensional flow field information, rather than just a single indicator of the wall surface. This provides more objective and convincing flow field data for assessing thrombosis risk. Attached Figure Description

[0024] Figure 1 This is a diagram of the neural network structure based on deep learning in this invention;

[0025] Figure 2 This is a flowchart of the process of the method of the present invention. Detailed Implementation

[0026] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0027] like Figure 2 As shown, the specific implementation and process of the present invention are as follows:

[0028] 1. Dataset Setup

[0029] In practice, the patient data used came from Beijing Fuwai Hospital. All data received ethical approval from the institutional review committee, and informed consent was obtained from all participants.

[0030] The patient was an atrial fibrillation patient. The enhanced CT medical images of the atrial fibrillation patient were used to obtain unsteady velocity and pressure fields through model reconstruction, mesh generation, and CFD simulation calculation.

[0031] Subsequently, the CFD results are converted into point cloud data: specifically, the CFD mesh nodes are extracted into point clouds, and the velocity and pressure values ​​corresponding to each mesh node are used as labels for each point in the point cloud.

[0032] Enhanced CT images of atrial fibrillation patients are used to reconstruct a three-dimensional model of the left atrium. Next, this model is divided into discrete small units, i.e., mesh generation. In this step, the outermost surface mesh of the three-dimensional left atrial model is set as a wall mesh, with negligible wall thickness, and is considered a rigid solid domain; all internal space meshes, except for the walls, are set as internal meshes, and are considered fluid domains. Correspondingly, after CFD calculations, all nodes on the wall mesh are converted into wall point clouds, and all internal meshes are converted into internal point clouds.

[0033] To enhance sample diversity, the point cloud data of each atrial fibrillation patient's real case was randomly sampled four times. The final dataset was constructed by normalizing the point cloud coordinates and pressure and velocity labels.

[0034] 2. Model Construction

[0035] The structure of the model is as follows Figure 1 As shown, there are two parallel channels, one above the other, which are divided into a wall point cloud channel and an internal point cloud channel, for inputting wall point clouds and internal point clouds respectively.

[0036] The wall point cloud and the interior point cloud are each represented by a two-dimensional tensor. The size of the first dimension of the two-dimensional tensor is equal to the number of points in the wall point cloud or the interior point cloud, and the size of the second dimension is 3, representing the xyz coordinates.

[0037] After the two two-dimensional tensors are input into the two parallel channels above and below the model, each tensor first passes through a spatial transformation layer. The spatial transformation layer performs spatial transformation on the two-dimensional tensors, scaling and rotating the wall point cloud and the interior point cloud into standard point clouds, respectively, reducing the impact of spatial diversity of the point cloud.

[0038] Subsequently, the tensors from the two parallel channels are fed into two sets of convolutional modules that alternate between edge convolutional layers and convolutional layers. Edge convolutional layers are convolutional operations based on the adjacency relationships of the point cloud, used to extract local features from the point cloud data; convolutional layers are used to extract and transform features for each point in the point cloud. The alternating use of multiple sets of edge convolutional layers and convolutional layers allows for the progressive extraction of local features.

[0039] Then, the tensors output by the two sets of convolutional modules in the two parallel channels each enter a new convolutional layer, increasing their feature count to match their point count, thus fully extracting the geometric features of the point cloud and obtaining new tensors. At this point, the new tensors obtained from the two parallel channels respectively contain global geometric information (wall point cloud channel) and local geometric information (internal point cloud channel) of a left atrial model.

[0040] The tensor containing global geometric information from the wall point cloud channels is processed through a max-pooling layer and then concatenated with the tensor containing local geometric information to achieve the aggregation and summarization of tensor features. Max pooling can reduce data size, improve the robustness of extracted features, and also satisfy the rotation and translation invariance and global awareness of the point cloud. After the above max-pooling operation, the tensor of the wall point cloud channels is deformed to have the same shape as the tensor of the internal point cloud channels. Then, the final tensors output by the two are concatenated along the dimension of the number of points to form a merged tensor.

[0041] The merged tensor is then passed through two fully connected layers to obtain the output tensor. The output tensor is also two-dimensional. The first dimension is the number of points in the internal point cloud, and the second dimension is 3, representing the three components of the velocity field (vector, 3 corresponds to the three components u, v, w) or the pressure field p (scalar, 3 represents repetition three times).

[0042] The frequency of occurrence, sequential arrangement, dual-channel structure of the entire model, training settings, and optimal parameters of the above three components (spatial transformation layer, edge convolutional layer + convolutional layer, and convolutional layer) are innovations of this invention.

[0043] 3. Training the model

[0044] All convolutional layers use two-dimensional convolutional layers with a kernel size of (1,1); all MLPs use one-dimensional convolutional layers with a kernel size of 1. The loss function is the mean absolute error (MAE), ensuring that the predicted values ​​approach the true values, and its expression is as follows:

[0045]

[0046] in, This refers to the predicted value for each point in the point cloud, which is the pressure or speed predicted by the deep learning model. N1 represents the true value of each point in the point cloud, which is the pressure or velocity in the CFD simulation; N1 is the number of points in the internal point cloud of a left atrial point cloud model.

[0047] Except for the output layer, the activation function is tanh, and the learning rate is lr = 0.00001. All testing and training processes were performed on the TensorFlow platform (tensorflow-2.12.0 and python-3.8.0 on NVIDIA GeForceRTX 4090). Each training iteration lasted at least 1000 epochs to ensure that the loss function had fully converged.

[0048] 4. Test Model

[0049] Before randomly sampling real data, it is divided into training, validation, and test sets. The entire model training process uses the training set, ensuring that the trained model has never seen data from the test set. The test set is then input into the trained model to obtain the test results. NMAE and MRE are chosen as two metrics to evaluate model performance, defined as follows:

[0050]

[0051]

[0052] The meaning of each letter is the same as in the MAE expression above.

[0053] To better evaluate the model's performance, the two metrics mentioned above are compared with two classic models, as detailed in Examples 1 and 2.

[0054] Example 1: Prediction of pressure field in left atrium and left atrial appendage based on deep learning

[0055] Using the aforementioned dataset creation method and a deep learning-based neural network model, the pressure field within the left atrium and left atrial appendage was predicted. Enhanced CT images of atrial fibrillation patients were reconstructed, meshed, and subjected to CFD simulation to obtain an unsteady pressure field. Subsequently, the CFD results were converted into point cloud data: CFD mesh nodes were extracted as point clouds, and the pressure value corresponding to each mesh node was used as the label for each point in the point cloud. To enhance sample diversity, each set of real cases underwent four rounds of random sampling. The final dataset was obtained by normalizing the point cloud coordinates and pressure labels.

[0056] The training and validation sets were used to train the model, and the test set was then input into the trained model for testing. All test results were based on the loss function having fully converged. The pressure distribution maps of the deep learning predictions and CFD calculations were compared, including anterior and posterior views of the left atrium and pressure distributions in three cross-sections, revealing relatively consistent pressure distributions. The model demonstrates good accuracy and stability in predicting the pressure field for left atrium and left atrial appendage shapes.

[0057] To quantitatively and accurately evaluate the model's performance, NMAE and MRE were selected as evaluation indicators. The performance of the model was compared with that of two classic models, as shown in Table 1.

[0058] Table 1. Comparison of pressure field prediction results between this model and two classical models.

[0059] MRE 2.3 5.5 8.5 NMAE 4.9 9.7 15.2

[0060] The first comparison is with PointNet, a classic model for point cloud data; the second comparison is with Dual Channel Network, a model for predicting coronary artery flow fields.

[0061] As can be seen from the table results, the model of this invention has higher accuracy in predicting pressure fields than the other two types of models, achieving a significant and outstanding effect.

[0062] Example 2: Velocity field prediction of the left atrium and left atrial appendage based on deep learning

[0063] Using the aforementioned dataset creation method and a deep learning-based neural network model, the velocity field within the left atrium and left atrial appendage was predicted. Enhanced CT images of atrial fibrillation patients were reconstructed, meshed, and subjected to CFD simulation to obtain an unsteady velocity field. Subsequently, the CFD results were converted into point cloud data: CFD mesh nodes were extracted as point clouds, and the velocity value corresponding to each mesh node served as the label for each point in the point cloud. To enhance sample diversity, each set of real cases underwent four rounds of random sampling. The final dataset was obtained by normalizing the point cloud coordinates and velocity labels. Unlike pressure field prediction, velocity is a vector; therefore, the three features of the output layer of the velocity field prediction model correspond to the three directional components of the velocity. For the pressure field prediction model, these three features are identical.

[0064] The training and validation sets were used to train the model, and the test set was then input into the trained model for testing. All velocity field results were based on the loss function having fully converged. The results from deep learning and CFD calculations were also compared: the velocity magnitude distributions in the three cross-sections of the left atrium were relatively consistent. To compare the prediction performance of the three velocity vector directions, streamline diagrams obtained by the two methods were plotted, and they were found to be relatively consistent.

[0065] To quantitatively evaluate the model's prediction performance on the velocity field, NMAE and MRE were also selected as evaluation metrics. The model of this invention was compared with two classic models, Pointnet and Dual Channel Network, as shown in Table 2.

[0066] Table 2. Comparison of velocity field prediction results between this model and two classical models.

[0067] MRE 11.1 28.5 27.2 NMAE 10.9 25.3 21.7

[0068] As shown in the table above, the model of this invention outperforms both classical models in velocity field prediction, achieving excellent velocity field prediction performance.

Claims

1. A method for detecting the flow field in the left atrium and left atrial appendage based on deep learning, characterized in that: 1) Construct a deep learning model; 2) Input the point cloud dataset into the deep learning model for training; 3) Use the trained deep learning model to perform detection and prediction processing on the point cloud data to be tested to obtain the flow field detection results; The flow field includes a pressure field or a velocity field; The point cloud dataset is divided into wall point clouds and interior point clouds; Input an enhanced CT medical image, and after model reconstruction, obtain a three-dimensional model of the left atrium; divide the three-dimensional model of the left atrium into discrete small units, that is, mesh generation; set the outermost surface mesh of the three-dimensional model of the left atrium as a wall mesh; except for the walls, all internal space meshes are set as internal meshes; all nodes on the wall mesh are converted into wall point clouds, and all internal meshes are converted into internal point clouds. The deep learning model specifically includes two parallel channels and one convergence channel. The two parallel channels are the wall point cloud channel and the internal point cloud channel, respectively. The input wall point cloud and internal point cloud are both converted into two-dimensional tensors. The two-dimensional tensors of the wall point cloud and internal point cloud are input into the wall point cloud channel and the internal point cloud channel, respectively, to obtain global geometric information and local geometric information. The global geometric information and local geometric information are concatenated into a tensor and then input into the convergence channel for processing to obtain the flow field results. The wall point cloud channel includes a spatial transformation layer, three convolutional layers, two edge convolutional layers, and a max pooling layer. The wall point cloud first undergoes spatial transformation through the spatial transformation layer, and then is input into two consecutive sets of convolutional modules to extract local features. Each set of convolutional modules consists of an edge convolutional layer and a convolutional layer connected in sequence. Finally, the local features are processed by a convolutional layer and a max pooling layer in sequence and then output to obtain global geometric information. The internal point cloud channel includes a spatial transformation layer, three convolutional layers, and two edge convolutional layers. The internal point cloud first undergoes spatial transformation through the spatial transformation layer, and then is input into two consecutive sets of convolutional modules to extract local features. Each set of convolutional modules consists of an edge convolutional layer and a convolutional layer connected in sequence. Finally, the local features are processed by a convolutional layer and output to obtain local geometric information.

2. The method for detecting the flow field in the left atrium and left atrial appendage based on deep learning according to claim 1, characterized in that: In the deep learning model, the global geometric information output from the wall point cloud channel and the local geometric information output from the internal point cloud channel are concatenated along the dimension of the number of points to form a tensor.

3. The method for detecting the flow field in the left atrium and left atrial appendage based on deep learning according to claim 1, characterized in that: The edge convolutional layers in the wall point cloud channel and the internal point cloud channel specifically refer to convolutional operations based on the adjacency relationship of the point cloud.

Images