Method for training an artificial deep neural network for estimating hemodynamic parameters, method for estimating hemodynamic parameters, computer program product and computer system

Abstract

A method for training an artificial deep neural network (ADNN) for estimating hemodynamic parameters from vascular tree geometry, comprising the step of acquiring a set of vascular tree geometries, is implemented using an ADNN having an architecture adapted for point cloud processing with distance-based point grouping. Distance is defined as geodesic distance along the vascular tree. The method for estimating hemodynamic parameters from vascular tree geometry using an ADNN according to the present invention involves the step of using an ADNN adapted for point cloud processing with geodesic distance-based point grouping. The present invention also relates to a computer program product comprising a set of instructions for implementing the method according to the present invention when executed on a computing system. The present invention also relates to a computer system adapted for implementing the method according to the present invention.

Description

[Technical Field]

[0001] The present invention relates to a method for training an artificial deep neural network to estimate hemodynamic parameters, a method for estimating hemodynamic parameters using an artificial deep neural network, a computer program product that implements the method according to the present invention, and a computer system adapted to implement the method according to the present invention. [Background technology]

[0002] Cardiovascular disease is one of the leading causes of death worldwide. The development of computed tomography (CT) has enabled a non-invasive approach to accurate diagnosis in patients suspected of having ischemic heart disease.

[0003] CT scans are generally represented as 3D volumetric images. Such 3D volumetric images represent physical quantities as functions of three spatial coordinates. In digital volumetric images, each sample (voxel) represents this quantity measured at a specific location. The image consists of a series of 2D spatial slices containing the object. Typically, slices are represented as an image matrix of pixels (X and Y coordinates). The slice number indicates the Z coordinate.

[0004] Interpreting CT images and diagnosing cardiovascular diseases are not easy tasks and require skilled physicians. A common problem is that interpreting the visible projections in CT images requires highly skilled radiologists, while diagnosing cardiovascular diseases requires skilled cardiologists. There is a shortage of personnel proficient in both fields. Therefore, computer-aided technology is needed.

[0005] One aspect of computer-assisted diagnosis of cardiac lesions is the generation of diagnostically aiding 3D models of blood vessels. In particular, detailed 3D models of the coronary arteries are required in the field of cardiac lesions. Detailed models enable precise estimation of patient-specific anatomical and functional characteristics, such as hemodynamic parameters. Current technology includes numerous techniques that utilize computational fluid dynamics (CFD) for blood flow simulation. The geometry of the vascular tree, represented in 3D volumetric images, can be expressed as a subset of voxels in the 3D image, a surface mesh, a volumetric mesh, or a point cloud. An example of a CFD technique used to calculate the fractional flow reserve (FFR) from a 3D model of the coronary arteries obtained by computed tomography angiography (CCTA) is disclosed in Patent Document 1. Generally, CFD simulations are configured to return the FFR or other hemodynamic parameters or a combination of parameters for a patient under study. Other examples are shown in Non-Patent Document 1.

[0006] CFD simulation is a superior alternative to manual diagnostic methods performed by physicians analyzing CT images, or even to invasive measurements. CFD simulation is performed using geometric models of blood vessels or vascular trees. Geometric models require extraction from patient volumetric images, typically DICOM images (Digital Imaging and Communications in Medicine is a standard for the communication and management of medical imaging information and related data). Well-known methods exist for the automatic or semi-automatic extraction of geometric models from patient 3D volumetric images, as well as methods for augmenting them.

[0007] Patent Document 2 discloses a super-resolution reconstruction method for time-varying flow fields for hemodynamic simulation. This method comprises the following steps: (1) Dataset generation: Performing vascular simulation using SimVascular software to construct a time-varying flow field dataset through the steps of image acquisition, geometric modeling, grid generation, and simulation; (2) Velocity field feature extraction: Extracting input data features of the dataset using PointNet to generate a 1024-dimensional feature vector fv; (3) Time and resistance feature extraction: Extracting time and resistance feature vectors of the input data using a resistance-time encoder to generate a 1024-dimensional feature vector frt; (4) Performing feature decoding to reconstruct a high time-resolution velocity field; and (5) Evaluating and analyzing the reconstruction results, training the network using the amplitude and direction loss functions of the velocity field, and evaluating and analyzing the reconstruction results using the average modulus length error and relative error. The method described in Patent Document 2 is demonstrated in relation to modeling the geometry of arterial segments, which are relatively simple segments of a vascular tree suitable for point cloud modeling. It is a very commonly recognized fact that processing point clouds becomes difficult when they represent complex, elongated structures with multiple branches. This is particularly problematic when at least two distinct branches are spatially close to each other. This problem is discussed in Non-Patent Documents 2 and 3.

[0008] A drawback of CFD simulations is that they require considerable time and computational power to perform. Therefore, there is a need for alternatives. A promising alternative is the computer analysis of vascular tree geometry using artificial intelligence trained on real-world data, artificial data, or hybrids thereof. Real-world data consists of the actual vascular tree geometry and measured hemodynamic parameters obtained from patients. Artificial data includes a computer-generated model of the vascular tree geometry and simulated hemodynamic parameters. Hybrid data may consist of the actual vascular tree geometry and simulated hemodynamic parameters. Other types of hybrid data include real-world and artificial examples.

[0009] Systems and methods for identifying individual-specific blood flow characteristics, i.e., hemodynamic parameters, with the assistance of artificial intelligence are disclosed in Patent Documents 3, 4, and 5. The method includes, first, acquiring an individual-specific, actual or artificial geometric model and blood flow characteristics of at least a portion of the individual's vascular system; creating feature vectors corresponding to the geometric model in a predefined form; and training an artificial deep neural network using the geometric model, the feature vectors, and the blood flow characteristics. After training is complete, the hemodynamic parameters become estimable by extracting feature vectors from the geometric model and obtaining estimates of the hemodynamic parameters using the trained artificial deep neural network given the feature vectors. Generally, artificial intelligence is used to map specific predefined features extracted from the geometric model to hemodynamic parameters. Features include, but are not limited to, the dimensions of blood vessels, changes in diameter, lumen size, and length of stenosis, with respect to geometry. These features are design choices for the method, and their specific definitions are rarely publicly shared. [Prior art documents] [Patent Documents]

[0010] [Patent Document 1] European Patent No. 3820357 Specification [Patent Document 2] Chinese Patent Application Publication No. 114399608 Specification [Patent Document 3] European Patent No. 3218872 Specification [Patent Document 4] U.S. Patent Application Publication No. 2014 / 073976 Specification [Patent Document 5] U.S. Patent Application Publication No. 2016 / 166209 Specification [Non-Patent Document]

[0011] [Non-Patent Document 1] Paper by Gaoyang Li et al., "Prediction of 3D Cardiovascular hemodynamics before and after coronary artery bypass surgery via deep learning", doi.org / 10.1038 / 542003-020-01638-1 [Non-Patent Document 2] Patryk Rygiel, Maciej Zieba, Tomasz Konopczynski, "Eigenvector Grouping for Point Cloud Vessel Labeling", Proceedings of the First International Workshop on Geometric Deep Learning in Medical Image Analysis, PMLR194:72-84, 2022 [Non-Patent Document 3] He, J. et al. (2020) Learning hybrid representations for automatic 3D vessel centerline extraction, arXiv.org. Available at: https: / / arxiv.org / abs / 2012.07262 (Accessed on February 15, 2023)

Summary of the Invention

Problems to be Solved by the Invention

[0012] In addition, it is difficult and rather ineffective to train an artificial deep neural network using only the overall geometry of the vascular tree in its global context. On the other hand, using only local context is also ineffective. Further, since the above vectors cannot be defined in a universal manner suitable for all shapes in all patients, it is difficult to use a predefined vector of physical features to be extracted from the geometry of the vascular tree. Moreover, the process of feature extraction is prone to numerical errors and the features need to be predefined. Furthermore, a set of features known in the art is usually assigned to specific candidate points along the vascular tree, and the process of selecting candidate points is also prone to errors.

Means for Solving the Problems

[0013] A computer-implemented method according to the present invention for training an artificial deep neural network for estimating hemodynamic parameters from vascular tree geometry in order to diagnose a patient without invasive measurement includes the steps of: obtaining a set of vascular tree geometries; obtaining hemodynamic parameters corresponding to the geometries in the set; preparing training data for the artificial deep neural network; and training the artificial deep neural network. The artificial deep neural network according to the present invention is an artificial deep neural network (ADNN) having an architecture adapted for point cloud processing using distance-based point grouping. Distance is defined as geodesic distance along the vascular tree. The step of preparing training data for the artificial deep neural network comprises representing the geometries as a point cloud and a centerline graph. One or more parameters used in the training set are further used by the artificial deep neural network for estimating these hemodynamic parameters from other geometries. The use of geodesic distance for selecting and grouping neighbors within a point cloud representation allows for an elastic construction that adapts itself to the actual dynamics of fluid in the vascular tree without the need for a predefined set of features. This improves accuracy and eliminates errors related to feature selection. Geodesic distance grouping enables analysis of the vascular tree in both global and local contexts, and as a result, vectors of specific features can be implicitly constructed by the artificial deep neural network itself. Therefore, simulations become more reliable and accurate, and less dependent on the typicality of the vascular tree. However, point grouping using Euclidean distance does not work, while geodesic distance requires more computational power. The use of a centerline graph speeds up the calculation of geodesic distance and reduces its dependence on patient-specific deviations in vascular geometry. Geodesic distance can be calculated along a centerline, along the blood mainstream calculated using the centerline, or along a curve in the vicinity of the centerline. The use of a centerline reduces errors and the risk of forming "shortcuts" through the vessel walls.

[0014] Advantageously, the computer-implemented method comprises the step of obtaining a vascular tree centerline graph, and the geodesic distance along the vascular tree is measured along the centerline graph. The centerline graph reflects the topology of the vascular tree and is very suitable for calculating geodesic distances because it enables more accurate creation of blood flow simulations. Explicitly measuring the geodesic distance along the centerline is the simplest way to enhance geodesic distance calculations using the centerline.

[0015] Advantageously, the step of obtaining values of hemodynamic parameters comprises the step of using a computational fluid dynamics simulation. This approach enables the generation of large learning datasets without complex measurements. This enables more efficient learning. When artificial geometries are used, learning can be further accelerated. A hybrid approach using both real and artificial models can mitigate this. Similarly, CFD simulations can be validated by the addition of actual measurement data.

[0016] The present invention provides a computer-assisted method for estimating hemodynamic parameters from the geometry of a vascular tree for diagnosing a patient without invasive measurement, comprising the steps of: obtaining the geometry of a vascular tree; and applying the artificial deep neural network for estimating hemodynamic parameters, with the step of using an artificial deep neural network. The artificial deep neural network according to the present invention is an artificial deep neural network (ADNN) having an architecture adapted for point cloud processing using distance-based point grouping. Distance is defined as geodesic distance along the vascular tree, and geometry is represented as a point cloud and a centerline graph. The use of a point cloud allows for a more elastic modeling of the geometry within the structure of the artificial deep neural network. The use of geodesic distance for selecting and grouping neighborhoods within a point cloud representation with a centerline graph allows for an elastic configuration that adapts itself to the actual dynamics of the fluid in the vascular tree without the need for a predefined set of features. This improves accuracy and eliminates errors in feature selection. Geodesic distance grouping enables the analysis of vascular trees in both global and local contexts, and as a result, vectors of specific features can be implicitly constructed by the artificial deep neural network itself.

[0017] Conveniently, geodesic distances along the vascular tree are measured along the centerline graph.

[0018] Advantageously, the step of obtaining geometry includes the steps of loading mesh geometry, converting geometry into a point cloud, and obtaining centerlines.

[0019] Advantageously, the steps of processing with an artificial deep neural network include encoding using at least one centerline set-abstraction block, decoding using at least one decoder block, processing using a shared multilayer perceptron block, and post-processing using a one-dimensional convolutional layer.

[0020] The computer program product according to the present invention, when executed on a computing system, comprises a set of instructions that cause the computing system to implement the method for estimating hemodynamic parameters according to the present invention.

[0021] The computer program product, when executed on a computing system, comprises a set of instructions that cause the computing system to implement a method for training an artificial deep neural network according to the present invention.

[0022] The computing system is for extracting an estimate of at least one hemodynamic parameter from the geometry of a vascular tree, adapted to realize a method for estimating hemodynamic parameters according to the present invention.

[0023] Advantageously, the system can be further adapted to implement the learning method according to the present invention.

[0024] The present invention will be described in detail below with reference to the following drawings. [Brief explanation of the drawing]

[0025] [Figure 1] A flowchart of an embodiment of a method for calculating hemodynamic parameters to which the present invention can be applied is shown. [Figure 2] This paper presents an architecture for an artificial deep neural network model used to estimate hemodynamic parameters. [Figure 3] The diagram shows a centerline set-abstract (CSA) block used in an ADNN model applied to an embodiment of the method according to the present invention. [Figure 4] A diagram of a decoder block used in an ADNN model applied to an embodiment of the method according to the present invention is shown. [Figure 5] The diagram shows a shared multilayer perceptron (MLP) block used in an ADNN model applied to an embodiment of the method according to the present invention. [Figure 6] A diagram of the PointNet block 305 used according to the present invention is shown. [Figure 7] This document shows a specific architecture used for experiments according to the present invention. [Figure 8] A flowchart of an embodiment of the learning method according to the present invention is shown. [Figure 9] An example of synthesized input data is shown. [Modes for carrying out the invention]

[0026] An embodiment of the method for estimating hemodynamic parameters from the geometry of a vascular tree using an artificial deep neural network according to the present invention will be described below with reference to Figure 1, which shows a schematic flowchart.

[0027] First, in step 101, patient-specific metadata and patient-specific volume image data are received. Patient-specific metadata is optional but can improve accuracy. Patient-specific metadata may include, but is not limited to, characteristics such as age, sex, non-invasive blood pressure monitoring, and medical history. Patient-specific volume image data is not optional and may include, but is not limited to, computed tomography (CT) scans, computed tomography angiography (CTA) scans, coronary computed tomography angiography (CCTA) scans, magnetic resonance imaging (MRI) scans, etc. Step 101 of data retrieval may involve actual measurements or simply downloading data from an external source. It may be retrieved from, for example, an external drive, a picture archiving and communication system (PACS), or any other means.

[0028] The recovered data is then loaded into a computing system that performs the method (102). This system may be a general-purpose computer, a dedicated digital processing machine, a distributed architecture, a virtual machine, or a cloud resource. It must be selected to complete the work within a desired time. In this embodiment, an Azure cloud environment is used.

[0029] Within the computing system, arterial anatomical geometry is acquired in step 103. Step 103, which acquires the geometry, simply involves reading a surface mesh. However, step 103 may also involve geometry format conversion, or even acquisition by volume image segmentation.

[0030] Step 103, which involves obtaining the geometry, can be performed manually by a human expert, automatically by a computer algorithm, or semi-automatically by a computer algorithm and a human expert. The input arterial anatomical geometry can be described as a surface mesh, a point cloud surface, or any other relevant data structure. If the data structure differs from that of a point cloud surface, the data structure of the arterial anatomical geometry should be converted to a point cloud surface. Points in the point cloud have at least three parameters and an additional number of features that constitute their coordinates in three-dimensional space. Exemplary features include, but are not limited to, the distance to the centerline or the geodesic distance from the vessel inlet (starting point). The surface mesh representation facilitates the determination of the centerline graph.

[0031] Then, in the next step 104, the centerline graph is extracted from the arterial anatomical geometry. The centerline graph can be extracted manually by a human expert, automatically by a computer algorithm, or semi-automatically by a computer algorithm and a human expert. The centerline graph is described as a connected polygonal chain of 3D points in space. Because the centerline graph is widely used for the diagnosis, visualization, and analysis of vascular models, it is often available for other applications, i.e., it needs to be extracted in any case. Thus, computational power is saved and reused. By using the centerline, the time-consuming calculation of geodesic distances is eliminated.

[0032] The centerline graph and arterial anatomical geometry are used for the artificial deep learning model described in the present invention with reference to Figures 2-7, and in step 105, selected hemodynamic parameters, particularly FFR and blood pressure reduction, are estimated. Specifically, step 105 is implemented in steps 204, 205, 206, and 207.

[0033] Hemodynamic parameters may be calculated for each point of the arterial anatomical geometry in the point cloud and cast to a centerline graph, or estimated for each point on the centerline. Hemodynamic parameters may be, but are not limited to, hypotension, fractional flow reserve (FFR), or any other relevant hemodynamic parameters. FFR is defined as the ratio of blood pressure at the arterial origin to blood pressure at the measurement point.

[0034] Then, in output step 106, the results are visualized, a report is generated, and it is returned to or presented to the user.

[0035] Figure 2 shows the architecture of a schematic artificial deep neural network (ADNN) model used in this embodiment of the present invention. The ADNN model shown in Figure 2 is an example of an ADNN model having an architecture adapted for point cloud processing. The ADNN model comprises a CSA block 204, a decoder block 205, and a shared MLP block 206. The CSA block is given the centerline graph 202 and the point cloud 203 and the geometry of the vascular tree represented as a centerline graph, which were acquired in step 104. In this embodiment, both the centerline graph 202 and the surface point cloud 203 are acquired from the geometry initially represented as a surface mesh 201.

[0036] The CSA block 204 forms an encoder module. The inputs to the CSA block are the centerline graph 202 and the surface point cloud 203. The CSA block returns a vector representing the entire surface point cloud 203.

[0037] The decoder block 205 returns the decoded feature point cloud, given the vector returned by the CSA block 204 and the surface point cloud 203.

[0038] The decoded feature point cloud is fed into a shared MLP block 206 for processing into selected hemodynamic parameters, for example, using a 1D convolutional layer. Finally, in block 207, the results are post-processed and output.

[0039] Figures 3-6 show the components of the schematic ADNN model architecture used in this embodiment of the present invention, as shown in Figure 2. The corresponding centerline set-abstract (CSA) block is shown in Figure 3, its corresponding decoder block in Figure 4, and its corresponding shared multilayer perceptron (MLP) block in Figure 5. The PointNet block 305 shown in Figure 3 will be described in detail with reference to Figure 6. Figure 7 shows a specific configuration of another specific embodiment and an architecture used for experimentation.

[0040] A specific embodiment is described below. In this embodiment, the ADNN model is a centerline graph 202 having V vertices and E edges, and N in It has a number of points, and for each point F in A surface point cloud 203 with a number of input features is taken. In one embodiment, the centerline graph 701 has an arbitrary number V vertices and a corresponding number E edges. The number of points in the surface point cloud 702 is N. in This is also optional and depends on the required resolution. The surface point cloud 702 has 5 F values ​​set. in It has the following characteristics: the first three represent the 3D spatial position, the fourth is the distance to the centerline, and the fifth is the geodesic distance from the point to the entrance, where the entrance is the starting point of the centerline. Both the centerline graph and the surface point cloud are extracted from the input vessel geometry, which is represented as surface mesh 201.

[0041] A CSA block is shown in Figure 3. The inputs to the CSA block are the input surface point cloud 301 and the centerline graph 303. The CSA block has a sampled number of representatives N. out Grouping scales D1, D2, ..., D n The number of points grouped for each scale is K1, K2, ..., K n , and are parameterized using the PointNet305 configuration. The number n can be chosen empirically.

[0042] The input surface point cloud 301 is first processed using the Farthest Point Sampling (FPS) 302 algorithm, and then N outIt is downsampled to a number of points. The FPS algorithm starts with a representative set R consisting of an arbitrarily selected single point P_0 from the point cloud. The point farthest from P_0 in terms of Euclidean distance is extracted and added to the set R. The next point is selected as the one farthest from all points in R in terms of Euclidean distance. This procedure is repeatedly iterated until the set R reaches the required size. The resulting point cloud is called the representative point cloud. In the centerline grouping 304, multi-scale grouping is performed independently for each point from the representative point cloud. Multi-scale grouping is defined as performing a plurality of independent groupings with different parameterizations. One grouping is a procedure for finding K i neighborhoods of points in the surface point cloud according to the specified strategy. The strategy is a process that determines an order in some aspect of the points in the surface point cloud according to the representative point for which the grouping is performed.

[0043] One possible grouping strategy is the centerline grouping described with reference to the present embodiment. On the other hand, those skilled in the art can propose alternative mechanisms for local grouping. An advantageous effect of the centerline-based grouping is that the centerline is also very useful in the generation of diagnostic images and is therefore often already available in the patient-specific geometry. Another advantageous effect of the centerline is that it reflects information about the topology of the blood vessels. In this strategy, the vertex of the centerline closest in terms of Euclidean distance is assigned to all points in the surface point cloud and points in the representative point cloud. For each point in the representative point cloud, the geodesic distance between the vertex of the centerline assigned to the representative point and all vertices of the centerline graph is calculated.

[0044] The scale parameter D i is such that the geodesic distance is D iThis method is used to extract only the vertices of centerlines smaller than a certain value. The centerline vertices extracted in this embodiment are used to query the points of all surface point clouds to which these vertices are assigned. The points of the surface point clouds extracted in this embodiment are considered a set of neighborhoods of a representative point. The set of neighborhoods is given by a specified number of neighborhoods K. i Further downsampling or upsampling is performed. In this embodiment, the grouping procedure is performed for each scale specified in the CSA parameterization.

[0045] The extracted multiscale set of neighbors is processed using PointNet305 to obtain a global neighbor feature vector. For each scale, F i An independent PointNet block is used to extract feature vectors of size N. The PointNet configuration is passed as parameterization to the CSA block. The output of the PointNet for each scale is concatenated point by point. The output surface point cloud 306 is N out ×(F1+F2+…F n ) are the sizes. F1, F2, ..., F n is the size of the feature vector for each grouping scale. Applicable PointNet processing methods are known from the publication PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space by Charles R.Qi, Li Yi, Hao Su, and Leonidas J.Guibas (https: / / doi.org / 10.48550 / arXiv.1706.02413), and are also available in software architecture (https: / / github.com / charlesq34 / pointnet2). However, this requires a modification of the definition of distance for geodesic distance for use in embodiments of the present invention. In particular, many other known PointNet-based architectures are not suitable for realizing point grouping by distance and are therefore not suitable for use in the present invention and for realizing point grouping by geodesic distance based on other lines reflecting the topology of a centerline or vascular tree.

[0046] The final CSA block groups all residual points and generates a single global embedding vector representing the entire surface point cloud 203. This global embedding vector, along with the embedding vectors from the corresponding CSA blocks and the surface point cloud 203, is used as input to the decoder block 205.

[0047] Figure 6 shows the details of the PointNet block 305. The input to the PointNet block is an input surface point cloud 601 with a spatial size of N × F. The input surface point cloud is processed using multiple shared MLP blocks 602 with specified parameterization. The number of shared MLP blocks is a design choice. The last shared MLP block is F out N×F processed using global max pooling 603, which generates a single feature vector 604 representing the surface point cloud of size 604. out Outputs a surface point cloud of the specified spatial size.

[0048] The number of CSA blocks used, their scales, corresponding distance parameters, and output spatial dimensions are design choices. Stacking more blocks provides greater generalization ability at the cost of more weights and the risk of overfitting the overall model. Stacking blocks allows for the extraction of local features, capturing fine geometry from a small neighborhood in the first block. Such local features are further grouped into larger units in the next block and processed to generate higher-order features. It is common to gradually expand the number of output feature channels from the previous block, usually using the next power of 2, or by raising the number of channels to 2. The number of blocks, along with the corresponding scales and distance parameters, should be chosen so that the model considers the entire point cloud in the last CSA block. In at least some vascular trees, a single block may not be sufficient to collect higher-order features, and too many blocks can lead to overfitting. In the embodiments described below, these settings were determined empirically, as shown in Figure 7. In the specific embodiment described below, this number is equal to 2 for all CSA blocks except the final CSA block: n=2. Thus, the blocks are parameterized using D1, D2, K1, and K2.

[0049] The first CSA block 703 takes the centerline graph 701 and the surface point cloud 702 as input. The parameters D1 and D2 of block 703 are set to 0.001m and 0.002m, respectively. The numbers K1 and K2 are set to K1=128 and K2=256. The spatial dimension of the output point cloud for block 703 is set to 2048 × (16 + 16).

[0050] The next CSA block 704 takes the output of the previous block 703 and the centerline graph 701 as input. The parameters D1 and D2 of block 704 are set to 0.002m and 0.004m, respectively. The numbers K1 and K2 are set to K1=16 and K2=32. The spatial dimension of the output point cloud for block 704 is set to 1024 × (32 + 32).

[0051] The next CSA block 705 takes the output of the previous block 704 and the centerline graph 701 as input. The parameters D1 and D2 of block 705 are set to 0.004m and 0.008m, respectively. The numbers K1 and K2 are set to K1=16 and K2=32. The spatial dimension of the output point cloud for block 705 is set to 512 × (64 + 64).

[0052] The next CSA block 706 takes the output of the previous block 705 and the centerline graph 701 as input. The parameters D1 and D2 of block 706 are set to 0.008m and 0.012m, respectively. The numbers K1 and K2 are set to K1=16 and K2=32. The spatial dimension of the output point cloud for block 706 is set to 256 × (128 + 128).

[0053] The next CSA block 707 takes the output of the previous block 706 and the centerline graph 701 as input. The parameters D1 and D2 of block 707 are set to 0.012m and 0.016m, respectively. The numbers K1 and K2 are set to K1=16 and K2=32. The spatial dimension of the output point cloud for block 707 is set to 128 × (128 + 128).

[0054] The next CSA block 708 takes the output of the previous block 707 and the centerline graph 701 as input. The parameters D1 and D2 of block 708 are set to 0.016m and 0.032m, respectively. The numbers K1 and K2 are set to K1=16 and K2=32. The spatial dimension of the output point cloud for block 708 is set to 64 × (128 + 128).

[0055] The final CSA block 709 takes the output of the previous block 708 and the centerline graph 701 as input. The parameter D1(708) is set to NONE, which is defined as grouping all residual points. For the final CSA block, n=1. The final CSA block aggregates all residual points, and the number K1 is set to K1=64. The spatial dimension of the output point group for block 709 is set to 256.

[0056] The decoder module consists of decoder blocks 205. The input to the decoder blocks is the output of the CSA block 204, and similarly for the last decoder block, the surface point cloud 203 is also input.

[0057] The decoder block is shown in Figure 4. The inputs to the decoder block are the input surface point cloud from the previous decoder block 401 and the surface point clouds from each CSA block 402. The decoder block is parameterized using a shared MLP configuration.

[0058] The input surface point clouds from the previous decoder block 401 and the surface point clouds from each CSA block 402 become the input to the feature interpolation block 403. In the feature interpolation block 403, the point-by-point features from the input surface point clouds from the previous decoder block are interpolated onto the surface point clouds from each CSA block 402 based on three nearest neighbors in terms of Euclidean distance. The surface point clouds from each CSA block with the interpolated features are then processed further.

[0059] The interpolated surface point cloud is processed using a shared MLP 404 to obtain a decoded feature point cloud 405. MLP blocks are a common mechanism used to extract features from a larger set of features, particularly for data that is not linearly separable. The shared MLP configuration is passed to the decoder block as a parameter. The output surface point cloud, i.e., the decoded feature point cloud 405, is N out ×F out It will be this size.

[0060] The number of decoder blocks is a design choice and depends on the number of CSA blocks. The spatial dimension of the output is also a design choice. Each decoder block requires input from the corresponding CSA block. Therefore, in this embodiment, the number of decoder blocks is 7.

[0061] It is common practice to gradually reduce the number of output feature channels from the previous block, usually by using powers of 2 or by dividing the number of channels by 2. In this embodiment, seven decoder blocks are used, as shown in Figure 7.

[0062] The first decoder block 716 receives input from the previous decoder block 715 and from the surface point cloud 702. Its spatial output dimension is set to 64 × 256.

[0063] Decoder block 715 receives input from the previous decoder block 714 and from CSA block 703. Its spatial output dimension is set to 128 × 128.

[0064] Decoder block 714 receives input from the previous decoder block 713 and from CSA block 704. Its spatial output dimension is set to 256 × 128.

[0065] Decoder block 713 receives input from the previous decoder block 712 and from CSA block 705. Its spatial output dimension is set to 512 × 128.

[0066] Decoder block 712 receives input from the previous decoder block 711 and from CSA block 706. Its spatial output dimension is set to 1024 × 64.

[0067] Decoder block 711 receives input from the previous decoder block 710 and from CSA block 707. Its spatial output dimension is set to 2048 × 32.

[0068] The final decoder block 715 receives input from CSA blocks 708 and 709. Its spatial output dimension is N. in It will be set to ×128.

[0069] If the surface is fully decoded by the last decoder block, the point features are passed through the shared MLP block 206.

[0070] A shared MLP block is shown in Figure 5. The input to the shared MLP block is an input surface point cloud 501 with a spatial size of N × F. The input surface point cloud is processed using a 1D convolution 502 with a specified number of output channels C. The output of the 1D convolution is then processed according to a 1D batch normalization layer 503, an activation function 504, and a dropout layer 505. The output surface point cloud 506 has a spatial size of N × C. The estimated hemodynamic parameters are determined by the parameters used in the training data set for training the network. If the ADNN trains using FFR data, as in this embodiment, it estimates the FFR data. On the other hand, the ADNN according to the embodiment of the present invention can similarly train using other hemodynamic parameters, including hypotension and wall shear stress.

[0071] The number of shared MLP blocks is a design choice.

[0072] In one embodiment, as shown in Figure 7, only one shared MLP block 717 is used. In one embodiment, the number of convolutional channels is set to 128.

[0073] The final step is to obtain the output 207 of the estimated point-by-point hemodynamic features. This involves applying a series of post-processing steps. In one embodiment, a single 1D convolutional layer 718 is used to obtain the required number N in The output is used to cast to a point, which is then output in step 719. Post-processing may include, as in one embodiment, the steps of casting the output estimated hemodynamic parameters from points on the point cloud to points on the centerline graph, or using these estimated hemodynamic parameters for further calculations in a CFD algorithm.

[0074] An embodiment of the method for training an ADNN according to the present invention will be described below with reference to Figure 8. This method comprises the step of acquiring a set of training geometries in the form of surface mesh geometry or surface point cloud geometry.

[0075] The hemodynamic parameter used in this embodiment is hypotension or FFR. The hypotension or FFR corresponding to each individual point in each training geometry is determined in the CFD simulation to form a set of training ground truth values.

[0076] The training data is formed by representing the training geometry as a surface point cloud with a centerline graph and assigning blood pressure reduction or FFR values ​​to specific points in the training geometry, and is used to train the ADNN model. The training dataset may consist of real data, artificial data, hybrid data, or a combination thereof, and may include patient-specific images, arterial geometry, and all relevant metadata.

[0077] The learning procedure is adapted to the estimation method. Generally, it comprises the step of obtaining a set of vascular tree geometries corresponding to either an actual patient or an artificial model, or both. Furthermore, it includes the step of obtaining the values ​​of parameters to be estimated by the trained network. These values ​​may be obtained in actual measurements, using CFD simulations, or both. Subsequently, the training data for the ADNN is prepared using the geometries and corresponding parameters, and includes representing the geometries as a point cloud with a centerline graph. This is important to the present invention because point cloud representation by geodesic distance grouping avoids the need to set arbitrarily selected parameters. Then the ADNN learns. The ADNN is an artificial deep neural network having an architecture adapted for point cloud processing by grouping points based on distance. Distance is defined as the geodesic distance along the vascular tree. The geodesic distance is preferably the distance along the centerline, and therefore the step of obtaining the centerline graph of the vascular tree is required. A specific example of the learning procedure is described in detail below with reference to Figure 8. However, a number of alternative specific learning modes are available to those skilled in the art and are applicable insofar as they satisfy the requirements set forth in claim 1.

[0078] The learning procedure is preceded by step 804, which involves model initialization 801 and preparing the input data. Model initialization 801 includes the step of initializing the weights of the ADNN model, which are sampled from a selected distribution.

[0079] In one embodiment, step 804, which involves loading the dataset, loads the sample-input surface mesh or point cloud and the hemodynamic features to be regressed into memory. The samples are then preprocessed (805), and if necessary, the mesh is decomposed into a centerline graph and a surface point cloud. Additional features are then calculated as needed and incorporated into each point on the point cloud.

[0080] Once the data is preprocessed, data loaders are created for both the training set 807, where training takes place, and the validation set 815, where the model is evaluated during the training process (806).

[0081] Model training involves a set of actions that are repeated iteratively, and one such set is called an epoch. Each epoch 802 begins with a training procedure 803 that utilizes a training data loader 807. The data loader generates batches of samples that are loaded into memory (808) and processed by a forward network procedure (809).

[0082] If the network output is obtained, a loss function between the desired result and the generated result is calculated (810). In one embodiment, the loss function is the mean squared error (MSE), and the loss is calculated for each point in the input surface point cloud, with the results averaged. Once the loss is calculated, its gradient is used to perform a backpropagation procedure that estimates how much the network weights need to be fine-tuned to obtain less loss in the current epoch (811). Model parameters are updated according to the calculated loss gradient (812).

[0083] A set of instructions applied to a data batch is called a training step. The training step is repeated until there are no remaining data batches left in the training data loader (813).

[0084] Once the final training batch is processed, the validation procedure begins (814). During the validation procedure, the model weights remain unchanged. The validation process is performed to evaluate and monitor the model's performance on data not included in the training set.

[0085] The validation step is similar to the training step, comprising instructions for loading the next data batch (816), executing a forward procedure (817), and calculating the loss function (818). The validation step differs from the training step in that it does not involve a backpropagation procedure or model parameter updates. The validation step is performed iteratively until all validation batches have been generated (819).

[0086] Once the validation procedure is complete, the requirements for the learning process are checked (820). The requirements may include checking whether the validation loss has decreased compared to the previous minimum. If it has decreased, the ADNN model is saved (821). Next, the stopping criterion 822 is checked. In one embodiment, the stopping step is defined as the number of epochs to be reached. If the stopping criterion is not met, the next epoch begins (802); otherwise, learning ends (823).

[0087] The process of evaluating the present invention used a dataset of 1700 synthetically generated vascular geometries in the form of a surface mesh. The training set, validation set, and test set consisted of 1500 samples, 100 samples, and 100 samples, respectively. Figure 9 shows an example of a synthetic vascular. The sample comprises a centerline graph 901 (node ​​density shown in 902) and a mesh 903 representing the vascular geometry.

[0088] Attempts to use distance-based grouping with Euclidean distance instead of geodesic distance were unsuccessful, resulting in low correlation between CFD results and actual data.

[0089] In the context of the above description, a computer is understood as a computer, microcontroller, signal processing, programmable gate array, hardware computer, hardware device including graphics card, application-specific integrated circuit or other digital processing unit used for image processing, and distributed solution including cloud computing environments.

[0090] Those skilled in the art, considering the teachings of the above description, can routinely propose multiple hardware and software solutions for computer-implemented execution of the apparatus and methods according to the present invention, as well as methods for acquiring learned information. In particular, learning and estimation may be performed on the same computer system or on completely separate computer systems. The use of a trained system may include further learning of it.

[0091] Note that FFR and blood pressure reduction are given merely as examples, and the described invention is applicable to the estimation of various hemodynamic parameters. The parameters to be estimated depend on the parameters provided in the training dataset.

[0092] It should be emphasized that the above description is merely illustrative of the present invention, and that those skilled in the art can propose numerous alternative embodiments encompassed by the scope of protection as defined in the appended claims.

[0093] In the claims, symbols placed in parentheses should not be construed as limiting the scope of the claims. The use of the verb "comprise" does not imply that there are no elements or steps other than those described in the claims. The article "a" or "an" preceding an element does not exclude the existence of multiple such elements.

Claims

1. A computer-based method for training an artificial deep neural network to estimate hemodynamic parameters from the geometry of a patient's vascular tree, Steps to obtain a set of vascular tree geometries, The steps include obtaining the values ​​of hemodynamic parameters corresponding to the geometry within the set, The steps include preparing training data for the artificial deep neural network, The steps include training the aforementioned artificial deep neural network, Equipped with, The aforementioned artificial deep neural network is an artificial deep neural network having an architecture adapted for point cloud processing using distance-based point grouping, The aforementioned distance is defined as the geodesic distance along the vascular tree, which is represented as a point cloud and a centerline graph. A computer-implemented method comprising the step of preparing the training data for the artificial deep neural network, the step of representing the geometry as a point cloud and a centerline graph.

2. The computer-aided method according to claim 1, comprising the step of obtaining a vascular tree centerline graph, wherein the geodesic distance along the vascular tree is measured along the centerline graph.

3. The computer-implemented method according to claim 1 or 2, wherein the step of obtaining values ​​of hemodynamic parameters comprises the step of using computational fluid dynamics simulation in the geometry within the set.

4. A computer-aided method for estimating hemodynamic parameters from the geometry of a vascular tree for patient diagnosis using an artificial deep neural network, comprising the steps of: obtaining the geometry of a vascular tree represented as a point cloud (103); and applying an artificial deep neural network for estimating hemodynamic parameters (105). The geometry is represented as a point cloud having a centerline graph, The aforementioned artificial deep neural network is an artificial deep neural network having an architecture adapted for point cloud processing using distance-based point grouping, The aforementioned distance is defined as the geodesic distance along the vascular tree, which is represented as a point cloud and a centerline graph. A computer-based method.

5. The computer-aided method according to claim 4, wherein the geodesic distance along the vascular tree is measured along the centerline graph.

6. The computer-aided method according to claim 5, wherein the step of obtaining the geometry comprises the steps of reading mesh geometry, converting the geometry into a point cloud, and obtaining a center line (104).

7. The computer-aided method according to claim 6, wherein the step of using the artificial deep neural network comprises the steps of: encoding using at least one centerline set-abstraction block (204); decoding using at least one decoder block (205); processing using a shared multilayer perceptron block (206); and post-processing using a one-dimensional convolutional layer (718).

8. A computer program product comprising, when executed on a computing system, a set of instructions for causing the computing system to implement the computer implementation method described in any one of claims 1 to 3.

9. A computer program product comprising, when executed on a computing system, a set of instructions for causing the computing system to implement the computer implementation method described in any one of claims 4 to 7.

10. A computing system for extracting an estimate of at least one hemodynamic parameter from the geometry of a vascular tree, adapted to implement the computer-assisted method specified in any one of claims 4 to 7.

11. A computing system according to claim 10, further adapted to implement a computer-implemented method for learning as defined in claim 1, 2, or 3.

Images