Cardiac structure remodeling neural network training, cardiac structure remodeling method and product

Abstract

The application discloses a kind of heart structure reconstruction neural network training, heart structure reconstruction method and product, it is related to artificial intelligence and three-dimensional reconstruction technical field, method includes: obtaining heart structure three-dimensional template and the three-dimensional point cloud data of heart structure;Heart structure three-dimensional template is registered with three-dimensional point cloud data, and registration data is obtained;Then registration data is input, target heart structure three-dimensional model is output, trains heart structure reconstruction neural network, and obtains heart structure reconstruction neural model, to obtain target heart structure three-dimensional model using deep learning technology after model is trained, solve the problem that existing KBR method needs multiple rounds of iteration in each modeling, and time-consuming is longer, and the application uses neural network to obtain target heart structure three-dimensional model directly, compared with the method based on implicit function in prior art, no extraction process is needed.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence and 3D reconstruction technology, and in particular to a template-based neural network training method for cardiac structure reconstruction, and a cardiac structure reconstruction product. Background Technology

[0002] Echocardiography is one of the most commonly used imaging techniques in cardiac examinations. Obtaining a three-dimensional model of the heart structure from echocardiography helps doctors more accurately quantify and analyze a patient's cardiac function, such as measuring ventricular volume and ejection fraction. However, ultrasound images are often noisy and have sparse sampling, making it difficult to directly obtain a three-dimensional model of the heart structure from echocardiography.

[0003] Marking the locations of anatomical structures on echocardiograms and reconstructing them from point clouds in a 3D coordinate system is an effective method. Currently, the main method for cardiac structure reconstruction from echocardiograms is Knowledge-based Reconstruction (KBR), which uses a piece-wise smooth subdivision surface (PSSS) algorithm to fit a template mesh to a 3D point cloud. However, this method requires labeling the features of each edge of the template mesh, which is not only demanding in terms of specialized knowledge but also time-consuming and labor-intensive. Furthermore, the KBR algorithm requires multiple rounds of iterative optimization in each reconstruction process, resulting in a long recovery time.

[0004] With the development of deep learning technology, techniques such as neural networks have been applied to surface reconstruction based on point clouds. Existing methods utilize implicit function fitting combined with mesh (3D mesh) extraction to reconstruct the surface of point clouds. These methods first use neural networks to fit the implicit functions of the object in 3D space (e.g., sighed / unsined distance function, occupancy function), and then use algorithms such as marching cubes to extract the mesh based on the implicit functions. However, these methods cannot directly obtain the 3D model of the target object. Summary of the Invention

[0005] The purpose of this application is to provide a neural network training method and product for cardiac structure reconstruction, which can directly obtain a three-dimensional model of the target cardiac structure without requiring multiple rounds of iterative optimization in each reconstruction process.

[0006] To achieve the above objectives, this application provides the following solution:

[0007] In a first aspect, this application provides a method for training a neural network for cardiac structure reconstruction, the method comprising:

[0008] Acquire a 3D template of the heart structure and 3D point cloud data of the heart structure; the 3D template of the heart structure is a triangular mesh structure.

[0009] Spatial registration is performed between the three-dimensional template of the heart structure and the three-dimensional point cloud data to obtain registration data;

[0010] Using the registration data as input and the target three-dimensional model of the heart structure as output, a neural network for heart structure reconstruction is trained to obtain a neural model for heart structure reconstruction. The neural network for heart structure reconstruction includes a point cloud encoder, a triangular mesh structure encoder, a feature fusion module, and a triangular mesh deformation module.

[0011] Optionally, the point cloud encoder is used to extract features from the 3D point cloud data in the input registration data to obtain point cloud features;

[0012] The triangular mesh structure encoder is used to extract features from the three-dimensional template of the heart structure in the input registration data to obtain template features;

[0013] The feature fusion module is used to fuse the point cloud features into the template features to obtain fused features;

[0014] The triangular mesh deformation module is used for:

[0015] Predict the offset of each vertex in the three-dimensional template of the heart structure based on the fusion features;

[0016] The vertices of the three-dimensional template of the heart structure are moved according to the offset to obtain the three-dimensional model of the target heart structure.

[0017] Optionally, the point cloud encoder includes a multilayer perceptron with channel dimensions;

[0018] The triangular mesh structure encoder includes a graph convolutional network;

[0019] The triangular mesh deformation module includes a graph convolutional network.

[0020] Optionally, the loss function of the cardiac structure reconstruction neural model includes: mean squared error loss, point-to-triangular mesh distance error in the point cloud, and volume error.

[0021] Optionally, acquiring the three-dimensional template of the heart structure and the three-dimensional point cloud data of the heart structure specifically includes:

[0022] A three-dimensional template of the heart structure is obtained; the three-dimensional template of the heart structure is a triangular mesh structure; the triangular mesh structure includes: a vertex matrix, an edge matrix, and a face matrix;

[0023] Acquire echocardiographic data and corresponding three-dimensional spatial information;

[0024] The echocardiogram data is converted into three-dimensional point cloud data based on the three-dimensional spatial information.

[0025] Optionally, after acquiring the three-dimensional template of the heart structure and the three-dimensional point cloud data of the heart structure, the method further includes:

[0026] The three-dimensional template of the heart structure and the three-dimensional point cloud data are normalized, and the translation distance and scaling ratio are recorded.

[0027] In a second aspect, this application provides a method for reconstructing a cardiac structure, the method comprising:

[0028] Acquire three-dimensional point cloud data of the heart structure to be reconstructed;

[0029] The three-dimensional point cloud data of the heart structure to be reconstructed is input into the neural model for heart structure reconstruction to obtain the heart structure reconstruction model; the neural model for heart structure reconstruction is trained using any of the above-described neural network training methods for heart structure reconstruction.

[0030] In a third aspect, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the cardiac structure reconstruction neural network training method or cardiac structure reconstruction method described in any of the preceding claims.

[0031] In a fourth aspect, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the cardiac structure reconstruction neural network training method or cardiac structure reconstruction method described in any of the preceding claims.

[0032] In a fifth aspect, this application provides a computer program product, including a computer program that, when executed by a processor, implements the cardiac structure reconstruction neural network training method or cardiac structure reconstruction method described in any of the preceding claims.

[0033] According to the specific embodiments provided in this application, the following technical effects are disclosed:

[0034] This application provides a method and product for training a neural network for cardiac structure reconstruction, as well as a method for cardiac structure reconstruction. First, a 3D template of the cardiac structure and 3D point cloud data of the cardiac structure are acquired. Then, the 3D template and the 3D point cloud data are spatially registered to obtain registration data. Next, using the registration data as input and the target 3D model of the cardiac structure as output, a neural network for cardiac structure reconstruction is trained to obtain a neural model for cardiac structure reconstruction. By using deep learning technology, after the model is trained, only one inference iteration is needed to obtain the target 3D model of the cardiac structure each time. This solves the problem that existing KBR methods require multiple iterations and are time-consuming in each modeling process. Furthermore, existing technologies based on implicit functions do not directly obtain the 3D model but require an extraction process, which increases errors. This application, however, uses a neural network to directly obtain the target 3D model of the cardiac structure without the need for an extraction process. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0036] Figure 1 This is an application environment diagram of a neural network training method for cardiac structure reconstruction according to an embodiment of this application;

[0037] Figure 2 A flowchart illustrating a method for training a neural network for reconstructing cardiac structures, provided in an embodiment of this application;

[0038] Figure 3 This is a schematic diagram of a cardiac structure reconstruction method provided in an embodiment of this application;

[0039] Figure 4 This is a schematic diagram of a constructed neural network model provided in one embodiment of this application;

[0040] Figure 5 A schematic diagram illustrating the final result of a cardiac structure reconstruction method provided in an embodiment of this application;

[0041] Figure 6 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0042] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0043] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0044] The neural network training method for cardiac structure reconstruction provided in this application can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be set up independently, integrated into server 104, or placed in the cloud or on another server. Terminal 102 can send a 3D template of the heart structure to be processed and 3D point cloud data of the heart structure to server 104. After receiving the 3D template and point cloud data, server 104 spatially registers the 3D template with the point cloud data to obtain registration data. Using the registration data as input and the target 3D heart structure model as output, a heart structure reconstruction neural network is trained to obtain a heart structure reconstruction neural model. The heart structure reconstruction neural network includes a point cloud encoder, a triangular mesh structure encoder, a feature fusion module, and a triangular mesh deformation module. Server 104 can feed back the obtained heart structure reconstruction neural model to terminal 102. In addition, in some embodiments, the neural network training method for cardiac structure reconstruction can also be implemented by the server 104 or the terminal 102 alone. For example, the terminal 102 can directly process the three-dimensional template of the cardiac structure and the three-dimensional point cloud data of the cardiac structure to be processed, or the server 104 can obtain the three-dimensional template of the cardiac structure and the three-dimensional point cloud data of the cardiac structure to be processed from the data storage system and process them.

[0045] Furthermore, the cardiac structure reconstruction method provided in this application embodiment can also be applied to, for example... Figure 1In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be set up independently, integrated into server 104, or placed in the cloud or on another server. Terminal 102 can send the three-dimensional point cloud data of the heart structure to be reconstructed to server 104. After receiving the three-dimensional point cloud data, server 104 inputs it into the neural model for heart structure reconstruction to obtain the heart structure reconstruction model. Server 104 can then feed back the obtained heart structure reconstruction model to terminal 102. Furthermore, in some embodiments, the heart structure reconstruction method can also be implemented independently by server 104 or terminal 102. For example, terminal 102 can directly process the three-dimensional point cloud data of the heart structure to be reconstructed, or server 104 can obtain the three-dimensional point cloud data of the heart structure to be reconstructed from the data storage system and process it.

[0046] The terminal 102 can be, but is not limited to, various desktop computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, and smart in-vehicle devices. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted devices. The server 104 can be implemented using a standalone server or a server cluster composed of multiple servers, or it can be a cloud server.

[0047] In one exemplary embodiment, such as Figure 2 As shown, a method for training a neural network for cardiac structure reconstruction is provided. This method is executed by a computer device, specifically by a terminal or server alone, or by both a terminal and a server. In this embodiment, the method is applied to... Figure 1 Taking server 104 as an example, the explanation includes the following steps S1 to S3. Wherein:

[0048] S1. Obtain a three-dimensional template of the heart structure and three-dimensional point cloud data of the heart structure; the three-dimensional template of the heart structure is a triangular mesh structure.

[0049] 3D point cloud data acquisition: First, echocardiographic data is acquired using ultrasound equipment and sensors, and its 3D spatial information is recorded. Then, the coordinates of key anatomical structures are marked on the 2D echocardiogram. Finally, the 2D coordinates are transformed to 3D using the recorded 3D spatial information to obtain 3D point cloud data.

[0050] Specifically, the acquired 3D point cloud data consists of two parts: point cloud coordinates and point cloud anatomical structure categories. Suppose there are np points in the point cloud, then the point cloud coordinates are represented by a (np,3) matrix (i.e., np xyz coordinates); the anatomical structures are represented by a (np,c) matrix, which is the one-hot encoding of the point categories in the point cloud (c is the number of anatomical structure categories).

[0051] Obtain a 3D template of the heart structure, which is represented by a mesh (triangular mesh structure) consisting of vertices (V), edges (E), and faces (F). Here, V is a (nv,3) matrix representing the 3D xyz coordinates of nv points, E is a (ne,2) matrix representing the indices of the two endpoints of ne edges in V, and F is a (nf,3) matrix representing the indices of the three endpoints of nf triangles in V.

[0052] V matrix: Vertex, with an shape of nv rows and 3 columns (nv: numbervertices). Represents the 3D coordinates of nv points. Each row contains the x, y, and z coordinates of a single point.

[0053] The E matrix represents edges, with a shape of ne rows and 2 columns (ne: number of edges). It represents ne edges, where a line connecting two vertices constitutes an edge. Each row contains two values: the indices of the two vertices corresponding to that edge. The index indicates the row number of a vertex in the V matrix.

[0054] Matrix F: Face, a matrix with shape nf rows and 3 columns (nf: number of faces). It represents nf faces. A face is a triangle formed by connecting three vertices. The three values ​​in each row are the indices of the three vertices of the triangle in matrix V.

[0055] In this process, the coordinates of the vertices of the aforementioned 3D point cloud data and the corresponding 3D template of the heart structure are normalized, and the translation distance and scaling ratio are recorded.

[0056] The 3D template of the heart structure was constructed by averaging the values ​​of all triangular mesh structures in the training dataset. Additionally, the category of each vertex in the triangular mesh structure was labeled (there are c possible categories), and each vertex may belong to multiple categories.

[0057] S2. Spatial registration is performed between the three-dimensional template of the heart structure and the three-dimensional point cloud data to obtain registration data.

[0058] This step is the initial spatial registration, the purpose of which is to match the 3D template of the heart structure with the labeled 3D point cloud data in spatial location. Specifically, the ICP (Iterative ClosestPoint) algorithm is used to minimize the distance between vertices and point clouds of the same category of heart structure 3D templates by rotating, translating, and scaling the templates.

[0059] S3. Using the registration data as input and the target three-dimensional model of the heart structure as output, train the heart structure reconstruction neural network to obtain the heart structure reconstruction neural model; the heart structure reconstruction neural network includes a point cloud encoder, a triangular mesh structure encoder, a feature fusion module, and a triangular mesh deformation module.

[0060] The point cloud encoder is used to extract features from the input registration data of 3D point cloud data to obtain point cloud features; the triangular mesh structure encoder is used to extract features from the input registration data of the 3D template of the heart structure to obtain template features; the feature fusion module is used to fuse the point cloud features into the template features to obtain fused features; the triangular mesh deformation module is used to: predict the offset of each vertex in the 3D template of the heart structure based on the fused features; and move each vertex of the 3D template of the heart structure according to the offset to obtain the target 3D model of the heart structure. Specifically, it includes the following steps:

[0061] The point cloud encoder consists of a channel-dimensional multilayer perceptron (MLP). The input (np, 3+c) of this module is composed of point cloud coordinates (np, 3) and class-specific one-hot codes (np, c), where np is the number of points in the point cloud, 3 represents the three-dimensional coordinates of the points, and c is the number of class numbers of the points. The output is the point cloud feature (np, d), where d is the feature dimension.

[0062] The triangular mesh structure encoder consists of a graph convolutional network (GCN). The input is the points (V:nm,3) and edges (E:ne,2) of the triangular mesh structure, and the output is the features (nm,d) of each point of the triangular mesh structure.

[0063] The feature fusion module fuses point cloud features into a template feature in the form of a triangular mesh. Specifically, it first aggregates the point cloud features of (np,d) along the point dimension to obtain the global point cloud feature (d,). Then, it copies the global point cloud feature nm times (nm,d) and splices them onto the template feature in the form of a triangular mesh to obtain the fused feature, which is the fused triangular mesh vertex feature (nm,2*d).

[0064] The triangular mesh deformation module is also a GCN. It takes the fused features as input, predicts the offset Δ(nm,3) of each vertex of the three-dimensional template of the heart structure, and then moves the vertex position of the three-dimensional template of the heart structure according to Δ to obtain the predicted triangular mesh vertex coordinates, thereby obtaining the three-dimensional model of the target heart structure.

[0065] The specific method for predicting vertex coordinates based on vertex position is as follows:

[0066] The process of predicting vertex positions is equivalent to moving each vertex from its initial position (x, y, z) to its target position (x', y', z'). During this movement, the vertex moves by distances of x'-x, y'-y, and z'-z along the x-axis, y-axis, and z-axis, respectively. Let Δx = x'-x, Δy = y'-y, and Δz = z'-z. Specifically, the prediction process is as follows: the input nm vertices represent nm 3D coordinates (x, y, z); the model predicts the offsets Δx, Δy, and Δz for each vertex along the x-axis, y-axis, and z-axis. Finally, the vertex is moved to (x+Δx, y+Δy, z+Δz).

[0067] To improve training performance, a suitable loss function needs to be designed. In this embodiment, the loss function consists of three parts: Mean Square Error (MSE), Point-to-Mesh Distance (P2MD) error, and Volume Error (VE). MSE is the mean square error between the predicted triangular mesh vertex coordinates and the actual vertex coordinates; P2MD is the mean distance from each point in the point cloud to the target heart structure 3D model; and VE is the ratio of the root mean square error between the target heart structure 3D model and the actual volume to the target heart structure 3D model volume. The final loss function is a weighted sum of the three loss functions:

[0068]

[0069] Where, n v v is the number of vertices in the triangular mesh. i and These are the actual and predicted values ​​of the vertex coordinates of the triangular mesh, respectively.

[0070]

[0071] Where, n p p represents the number of points in the point cloud. i For points in a point cloud, For the predicted triangular mesh, For point p i Distance to the predicted triangular mesh.

[0072]

[0073] Where V is the actual volume. To predict the volume of the triangular mesh.

[0074] After the model is trained, the trained neural model for reconstructing the heart structure is used to predict the 3D point cloud data. Based on the translation and scaling information recorded during the normalization process, the predicted results are restored to the true size and pose.

[0075] In summary, this embodiment helps doctors accurately and intuitively analyze a patient's cardiac function using echocardiography. The method takes annotated 3D point clouds of anatomical structures as input and utilizes a neural network to fit a template model to a target model, achieving 3D reconstruction of the heart structure on echocardiography. Specifically, this embodiment first collects 3D point cloud data and designs an initial template as input to the algorithm. Then, a point cloud reconstruction network based on the initial template is built and trained to extract features from the template model and the 3D point cloud data, ultimately outputting the target model. Finally, a series of post-processing operations are performed on the model prediction results to obtain a heart structure model with true size and pose.

[0076] In one exemplary embodiment, a method for reconstructing a cardiac structure is provided, the method comprising:

[0077] A1 acquires the three-dimensional point cloud data of the heart structure to be reconstructed;

[0078] A2 inputs the three-dimensional point cloud data of the heart structure to be reconstructed into the heart structure reconstruction neural model to obtain the heart structure reconstruction model; the heart structure reconstruction neural model is trained using the heart structure reconstruction neural network training method described in the above embodiment.

[0079] In one exemplary embodiment, a method for reconstructing cardiac structures is provided; see [link to relevant documentation]. Figure 3 It includes the following steps:

[0080] Step 1: Obtain 3D point cloud data of the heart structure and a target 3D model. Specifically:

[0081] 1.1 The process of acquiring 3D point cloud data is as follows: First, echocardiograms are acquired using ultrasound equipment, with 7-15 views from different angles acquired for each case. The 3D pose information of the ultrasound probe is also obtained using sensors. Then, the coordinates of anatomical structures (np,2) are marked on the 2D echocardiogram, specifically including: tricuspid annulus, right ventricular septum, right ventricular endocardium, base, apex, pulmonary valve annulus, and right ventricular septum edge (7 types). Finally, the 2D coordinates are transformed to 3D space using the recorded 3D pose information, and the coordinates of different views of the same heart at the same stage are merged to obtain a 3D point cloud (np,3) and its category code (np,7). In this embodiment, multiple views are generally not acquired simultaneously; each view is acquired separately. The merging of different views here refers to merging the coordinates of video frames representing the same stage (the moment the heart contracts to its minimum / diffuse to its maximum) in the cardiac cycle.

[0082] 1.2 The target 3D model is represented by a triangle mesh, which is represented by nodes (V:(np,3)), edges (E:(ne,2)) and faces (F:(nf,3)).

[0083] To facilitate subsequent training, all point cloud and mesh coordinates were normalized to [-1,1] through translation and scaling. It is worth noting that the normalized range here and the coordinate range of the template mesh below do not necessarily have to be [-1,1], they can also be [0,1] or other values, as long as different point clouds are normalized to the same range and the template mesh also conforms to this range.

[0084] Step 2: Construct a template mesh. The construction process involves averaging all target meshes in the training dataset. The coordinates of the template mesh are also within the range of [-1, 1].

[0085] Step 3: Register the template mesh with the 3D point cloud data. Specifically, the ICP algorithm is used to iteratively optimize and obtain the transformation matrix of the template mesh, which minimizes the distance between template meshes of the same category and the 3D point cloud data.

[0086] Step 4, build as follows Figure 4 The neural network model shown includes a point cloud encoder, a template mesh encoder, a feature fusion module, and a mesh deformation module. The point cloud encoder and template mesh encoder extract features from the input point cloud and template mesh, respectively. The feature fusion module fuses these features. The deformation module predicts the offsets of each node in the template mesh based on the fused features. Finally, the mesh nodes are moved according to these offsets to obtain the deformed target mesh. The specific structures of each module are as follows:

[0087] 4.1 The point cloud encoder is an MLP used to extract 3D point cloud features. As an example, this invention uses a three-channel MLP with ReLU activation layers and hidden layers of 32, 64, and 128 dimensions. Other MLPs or point cloud representation networks (such as point-wise convolution, PointNet) can also be used.

[0088] 4.2 The mesh encoder uses a GCN. As an example, the GCN of this invention includes six graph convolutional blocks, each containing two graph convolutional operations, and residual connections are used within the block. The activation function is ReLU. Other graph convolutional networks are also applicable.

[0089] 4.3 The feature fusion module is used to fuse point cloud features into Mesh nodes. Specifically, the point cloud features are first globally aggregated along the point dimension. As an example, this invention uses global max pooling to obtain the global point cloud feature: (d,). Then, this feature is copied nv times (nv, d) and concatenated to the Mesh feature respectively to obtain the fused mesh feature (nv, 2*d).

[0090] Alternatively, feature fusion based on an attention mechanism can be used. Specifically, point cloud features are used as the key, and mesh features as the query. Attention is calculated; in this case, soft attention is used (other methods such as multi-head attention, transformers, etc. can also be used). Then, the weighted sum of point cloud features corresponding to each node of the mesh is obtained based on the attention and concatenated to the mesh features. The attention calculation process is as follows:

[0091]

[0092] Where Q and K are the mesh features and point cloud features, respectively, and d k is the dimension of the feature vector.

[0093] 4.4 The mesh deformation module uses GCN. As an example, this embodiment uses the same structure as in step 4.2 and sets the output dimension to 3 (representing the xyz coordinates of the offset Δ).

[0094] Step 5: Train the neural network described above. The specific configuration is as follows: use Adamw as the optimizer, cosine annealing as the learning rate scheduler, with an initial learning rate of 0.001 and a batch size of 32. The loss function weights are 1*MSE + 1*P2MD + 0.5*VE.

[0095] Step 6: Using the model trained above, perform predictions and post-processing on the test set. Based on the recorded translation and scaling information, reconstruct the predicted mesh to its true size and pose. The final result is as follows: Figure 5 As shown.

[0096] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 6 As shown, the computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores 3D templates of the heart structure and 3D point cloud data of the heart structure, or 3D point cloud data of the heart structure to be reconstructed. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a neural network training method for heart structure reconstruction or a heart structure reconstruction method.

[0097] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0098] In one exemplary embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0099] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0100] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0101] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0102] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0103] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0104] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0105] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for training a neural network for reconstructing cardiac structure, characterized in that, The method for training the neural network for cardiac structure reconstruction includes: Acquire a 3D template of the heart structure and 3D point cloud data of the heart structure; the 3D template of the heart structure is a triangular mesh structure. Spatial registration is performed between the three-dimensional template of the heart structure and the three-dimensional point cloud data to obtain registration data; Using the registration data as input and the target three-dimensional model of the heart structure as output, a neural network for heart structure reconstruction is trained to obtain a neural model for heart structure reconstruction. The neural network for heart structure reconstruction includes a point cloud encoder, a triangular mesh structure encoder, a feature fusion module, and a triangular mesh deformation module. The point cloud encoder is used to extract features from the input registration data of the three-dimensional point cloud data to obtain point cloud features. The triangular mesh structure encoder is used to extract features from the three-dimensional template of the heart structure in the input registration data to obtain template features; The feature fusion module is used to fuse the point cloud features into the template features to obtain fused features; The triangular mesh deformation module is used for: Predict the offset of each vertex in the three-dimensional template of the heart structure based on the fusion features; The vertices of the three-dimensional template of the heart structure are moved according to the offset to obtain the three-dimensional model of the target heart structure.

2. The method for training a neural network for cardiac structure reconstruction according to claim 1, characterized in that, The point cloud encoder includes a multilayer perceptron in the channel dimension; The triangular mesh structure encoder includes a graph convolutional network; The triangular mesh deformation module includes a graph convolutional network.

3. The method for training a neural network for cardiac structure reconstruction according to claim 1, characterized in that, The loss function of the neural model for reconstructing the cardiac structure includes: mean squared error loss, distance error from points to triangular meshes in the point cloud, and volume error.

4. The method for training a neural network for cardiac structure reconstruction according to claim 1, characterized in that, The acquisition of the three-dimensional template of the heart structure and the three-dimensional point cloud data of the heart structure specifically includes: A three-dimensional template of the heart structure is obtained; the three-dimensional template of the heart structure is a triangular mesh structure; the triangular mesh structure includes: a vertex matrix, an edge matrix, and a face matrix; Acquire echocardiographic data and corresponding three-dimensional spatial information; The echocardiogram data is converted into three-dimensional point cloud data based on the three-dimensional spatial information.

5. The method for training a neural network for cardiac structure reconstruction according to claim 1, characterized in that, After acquiring the 3D template of the heart structure and the 3D point cloud data of the heart structure, the process further includes: The three-dimensional template of the heart structure and the three-dimensional point cloud data are normalized, and the translation distance and scaling ratio are recorded.

6. A method for reconstructing cardiac structure, characterized in that, The cardiac structure reconstruction method includes: Acquire three-dimensional point cloud data of the heart structure to be reconstructed; The three-dimensional point cloud data of the heart structure to be reconstructed is input into the neural model for heart structure reconstruction to obtain the heart structure reconstruction model; the neural model for heart structure reconstruction is trained using the neural network training method for heart structure reconstruction as described in any one of claims 1-5.

7. A computer device, comprising: The memory and processor contain a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the cardiac structure reconstruction neural network training method according to any one of claims 1-5 or the cardiac structure reconstruction method according to claim 6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the neural network training method for cardiac structure reconstruction as described in any one of claims 1-5 or the cardiac structure reconstruction method as described in claim 6.

9. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the neural network training method for cardiac structure reconstruction as described in any one of claims 1-5 or the cardiac structure reconstruction method as described in claim 6.

Images