Three-dimensional heart reconstruction method, device and equipment based on multi-section two-dimensional ultrasound and medium
Through the neural network model based on shape prior training and alternate iterative optimization technology, the problem of difficult fusion of multi-section two-dimensional ultrasound images is solved, and high-precision three-dimensional cardiac reconstruction is achieved to adapt to the anatomical characteristics of different individuals.
Patent Information
- Application Number
- CN202510515802.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-23
- Publication Date
- 2025-07-25
AI Technical Summary
The existing multi-section two-dimensional ultrasound images are difficult to accurately fusion, resulting in insufficient accuracy and applicability of three-dimensional heart reconstruction.
A neural network model based on shape prior training is adopted. By obtaining three-dimensional spatial coordinates mapped to the three-dimensional heart shape representation model, combined with alternating iterative optimization, the ultrasonic slice position is determined and a triangular grid is constructed to generate the target three-dimensional heart model.
It improves the accuracy and applicability of three-dimensional reconstruction of the heart, and can reconstruct the heart structure more accurately and adapt to the anatomical characteristics of different individuals.
Smart Images
Figure CN120374866A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the technical fields of artificial intelligence and medical image analysis, and particularly to a three-dimensional cardiac reconstruction method, device, equipment, and medium based on multi-plane two-dimensional ultrasound. Background Art
[0002] Cardiovascular diseases are one of the main causes of death globally, and their early diagnosis and accurate assessment are crucial for clinical intervention. Echocardiography has become an important means for clinical assessment of cardiac structure and function due to its non-invasiveness, real-time nature, and economy. However, traditional ultrasound imaging methods mainly rely on two-dimensional (2D) slices to obtain local information of the heart, and the accurate reconstruction of three-dimensional (3D) cardiac morphology still faces many challenges.
[0003] Currently, although three-dimensional ultrasound can provide stereoscopic structural information, its spatial and temporal resolutions are low, and it is highly dependent on manual delineation, which limits its clinical application. Therefore, researching how to perform three-dimensional cardiac reconstruction using conventional multi-plane two-dimensional ultrasound images has become an important research direction in the field of cardiovascular image analysis. Some existing computer-aided methods attempt to perform three-dimensional reconstruction using ultrasound images, but most rely on regular geometric models or statistical shape models and cannot fully utilize the detailed structural information in the echocardiogram segmentation results. In addition, the imaging differences and relative pose uncertainties between different ultrasound planes make it difficult for existing methods to effectively integrate multi-plane information, thus affecting the reconstruction accuracy and clinical applicability. Summary of the Invention
[0004] The purpose of the embodiments of this application is to propose a three-dimensional cardiac reconstruction method, device, equipment, and medium based on multi-plane two-dimensional ultrasound to solve the problem of difficult accurate fusion of multi-plane ultrasound images and improve the accuracy and applicability of cardiac three-dimensional reconstruction.
[0005] To solve the above technical problems, the embodiments of this application provide a three-dimensional cardiac reconstruction method based on multi-plane two-dimensional ultrasound, including:
[0006] Obtain the three-dimensional spatial coordinates to be modeled, and map the three-dimensional spatial coordinates to be modeled into a pre-constructed three-dimensional cardiac shape representation model to calculate the occupancy field value, where the three-dimensional cardiac shape representation model is a neural network model trained based on shape priors, and the occupancy field value is used to represent cardiac morphology information;
[0007] Extract shape images at multiple positions from the cardiac mesh of the shape prior as ultrasound segmentation masks, calculate the occupancy field value image based on the pre-constructed section coordinates, and compare the occupancy field value image with the ultrasound segmentation masks to determine the ultrasound slice positions;
[0008] The ultrasonic slice position and the cardiac morphology information are alternately optimized by an alternating iterative optimization method to obtain updated model parameters and section position parameters;
[0009] Based on the updated model parameters and section position parameters, a triangular mesh is constructed to generate a target three-dimensional cardiac model.
[0010] To solve the above technical problems, an embodiment of the present application provides a three-dimensional cardiac reconstruction device based on multi-section two-dimensional ultrasound, including:
[0011] A three-dimensional cardiac shape representation module, configured to obtain three-dimensional spatial coordinates to be modeled, map the three-dimensional spatial coordinates to be modeled into a pre-constructed three-dimensional cardiac shape representation model to calculate an occupancy field value, where the three-dimensional cardiac shape representation model is a neural network model trained based on a shape prior, and the occupancy field value is used to represent cardiac morphology information;
[0012] An ultrasonic section position estimation module, configured to extract shape images at multiple positions from the cardiac mesh of the shape prior as ultrasonic segmentation masks, calculate an occupancy field value image based on pre-constructed section coordinates, and compare the occupancy field value image with the ultrasonic segmentation masks to determine the ultrasonic slice position;
[0013] An iterative optimization module, configured to alternately optimize the ultrasonic slice position and the cardiac morphology information by an alternating iterative optimization method to obtain updated model parameters and section position parameters;
[0014] A three-dimensional cardiac model generation module, configured to construct a triangular mesh based on the updated model parameters and section position parameters to generate a target three-dimensional cardiac model.
[0015] To solve the above technical problems, a technical solution adopted by the present invention is: to provide a computer device, including one or more processors; a memory for storing one or more programs, so that one or more processors implement the three-dimensional cardiac reconstruction method based on multi-section two-dimensional ultrasound described in any one of the above.
[0016] To solve the above technical problems, a technical solution adopted by the present invention is: a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the three-dimensional cardiac reconstruction method based on multi-section two-dimensional ultrasound described in any one of the above is implemented.
[0017] An embodiment of the present invention provides a three-dimensional cardiac reconstruction method, device, equipment and medium based on multi-plane two-dimensional ultrasound. The method includes: obtaining three-dimensional space coordinates to be modeled, mapping the three-dimensional space coordinates to be modeled into a pre-constructed three-dimensional cardiac shape representation model to calculate occupancy field values, where the three-dimensional cardiac shape representation model is a neural network model trained based on shape priors, and the occupancy field values are used to represent cardiac morphological information; extracting shape images at multiple positions from the cardiac mesh of the shape prior as ultrasound segmentation masks, calculating an occupancy field value image based on pre-constructed section coordinates, and comparing the occupancy field value image with the ultrasound segmentation masks to determine the ultrasound slice positions; alternately optimizing the ultrasound slice positions and the cardiac morphological information in an alternating iterative optimization manner to obtain updated model parameters and section position parameters; constructing a triangular mesh based on the updated model parameters and section position parameters to generate a target three-dimensional cardiac model. The embodiment of the present invention generates cardiac morphological information based on shape priors, determines the ultrasound slice positions, and alternately optimizes the ultrasound slice positions and the cardiac morphological information in an alternating iterative optimization manner. After the optimization is completed, a target three-dimensional cardiac model is constructed, which solves the problem that it is difficult to accurately fuse multi-plane ultrasound images and is beneficial to improving the accuracy and applicability of cardiac three-dimensional reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] In order to more clearly illustrate the solutions in the present application, the following will briefly introduce the drawings required for the description of the embodiments of the present application. Obviously, the drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.
[0019] Figure 1 is a flowchart of an implementation of the three-dimensional cardiac reconstruction method based on multi-plane two-dimensional ultrasound provided by the embodiment of the present application;
[0020] Figure 2 is a flowchart of an implementation of the first sub-process in the three-dimensional cardiac reconstruction method based on multi-plane two-dimensional ultrasound provided by the embodiment of the present application;
[0021] Figure 3 is a schematic diagram of the network structure of the three-dimensional cardiac representation provided by the embodiment of the present application;
[0022] Figure 4 is a flowchart of an implementation of the second sub-process in the three-dimensional cardiac reconstruction method based on multi-plane two-dimensional ultrasound provided by the embodiment of the present application;
[0023] Figure 5 is a flowchart of an implementation of the third sub-process in the three-dimensional cardiac reconstruction method based on multi-plane two-dimensional ultrasound provided by the embodiment of the present application;
[0024] Figure 6 It is the implementation flowchart of the fourth sub - process in the three - dimensional cardiac reconstruction method based on multi - sectional two - dimensional ultrasound provided by the embodiments of the present application;
[0025] Figure 7 It is a schematic diagram of the three - dimensional point set generation process provided by the embodiments of the present application;
[0026] Figure 8 It is the implementation flowchart of the fourth sub - process in the three - dimensional cardiac reconstruction method based on multi - sectional two - dimensional ultrasound provided by the embodiments of the present application;
[0027] Figure 9 It is a schematic diagram of the three - dimensional cardiac reconstruction results of different sections provided by the embodiments of the present application;
[0028] Figure 10 It is a schematic diagram of the three - dimensional cardiac reconstruction device based on multi - sectional two - dimensional ultrasound provided by the embodiments of the present application;
[0029] Figure 11 It is a schematic diagram of the computer device provided by the embodiments of the present application. Detailed implementation manners
[0030] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the technical field to which this application belongs; the terms used in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application; the terms "including" and "having" and any variations thereof in the specification and claims of this application and the above - mentioned drawings are intended to cover non - exclusive inclusion. The terms "first", "second", etc. in the specification and claims of this application or the above - mentioned drawings are used to distinguish different objects and not to describe a specific order.
[0031] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in connection with the embodiments can be included in at least one embodiment of this application. The phrase appears in various places in the specification and does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will explicitly and implicitly understand that the embodiments described herein can be combined with other embodiments.
[0032] In order to enable those skilled in the technical field to better understand the solution of this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.
[0033] The present invention will be described in detail below with reference to the accompanying drawings and embodiments.
[0034] It should be noted that the three-dimensional heart reconstruction method based on multi-plane two-dimensional ultrasound provided by the embodiments of the present application is generally executed by a server. Correspondingly, the three-dimensional heart reconstruction device based on multi-plane two-dimensional ultrasound is generally configured in the server.
[0035] Please refer to Figure 1 , Figure 1 which shows a specific implementation of the three-dimensional heart reconstruction method based on multi-plane two-dimensional ultrasound.
[0036] It should be noted that if there are substantially the same results, the method of the present invention is not limited to Figure 1 the process sequence shown, and the method includes the following steps:
[0037] S1: Obtain the three-dimensional space coordinates to be modeled, and map the three-dimensional space coordinates to be modeled into a pre-constructed three-dimensional heart shape representation model to calculate the occupancy field value. Among them, the three-dimensional heart shape representation model is a neural network model trained based on shape prior, and the occupancy field value is used to represent heart morphology information.
[0038] Specifically, obtain the three-dimensional space coordinates to be modeled, where the three-dimensional space coordinates to be modeled are grid sampling points or random sampling points. Map the three-dimensional space coordinates to be modeled into a pre-constructed three-dimensional heart shape representation model to calculate the occupancy field value. Among them, the three-dimensional heart shape representation model is a neural network model trained based on shape prior, and the occupancy field value is used to represent heart morphology information. In the embodiments of the present application, by calculating the occupancy field value (Occupancy Field), a continuous and differentiable implicit heart representation is constructed.
[0039] Please refer to Figure 2 and Figure 3 , Figure 2 which shows a specific implementation of step S1, Figure 3 which is a schematic diagram of the network structure of the three-dimensional heart representation provided by the embodiments of the present application, and is described in detail as follows:
[0040] S11: Obtain the three-dimensional space coordinates to be modeled, and project the three-dimensional space coordinates to be modeled into a high-dimensional feature space by using the Fourier feature mapping method to generate high-dimensional features.
[0041] S12: Input the high-dimensional features into a multi-layer perceptron in the three-dimensional heart shape representation model for modeling to generate the occupancy field value. Among them, the multi-layer perceptron is three independent multi-layer perceptrons, which are respectively used to represent the left ventricular endocardium, the right ventricular endocardium, and the left and right ventricular epicardiums.
[0042] Specifically, in the three-dimensional heart shape representation model, three independent multi-layer perceptrons (MLPs) are designed to represent the left ventricular endocardium, the right ventricular endocardium, and the epicardium of the left and right ventricles respectively, so as to refine the geometric features of different cardiac tissues and improve the accuracy of morphological modeling. In the embodiment of the present application, in order to enhance the model's ability to express complex shapes, the Fourier Feature Mapping method is used to project the three-dimensional space coordinates to be modeled into a high-dimensional feature space to generate high-dimensional features. This embodiment enhances the model's learning ability for local and global details by introducing rich spatial frequency information. Then, the high-dimensional features are input into the multi-layer perceptron in the three-dimensional heart shape representation model for modeling to generate occupancy field values, realizing continuous and differentiable three-dimensional morphological modeling.
[0043] In a specific embodiment, a 7-layer multi-layer perceptron is used, with each layer containing 64 nodes, and 10 iterations of optimization are performed during the calculation process to improve the stability and accuracy of morphological modeling. In the output occupancy field values, if the coordinate point is inside the heart, the occupancy field value is set to 0, indicating belonging to cardiac tissue; if it is outside the heart, it is assigned 1, indicating that the point does not belong to the cardiac structure; if it is on the heart surface, it is assigned 0.5, which is used to describe the transition region of the morphological boundary. This implicit representation method ensures the continuity of the three-dimensional heart shape and provides a basis for subsequent optimization.
[0044] Please refer to Figure 4 , Figure 4 which shows a specific implementation before step S1, described in detail as follows:
[0045] S1A: Use the cardiac mesh as the shape prior, and train a neural network model based on the shape prior using a binary classification loss function and a smooth regularization loss function to generate a three-dimensional cardiac structure model.
[0046] Specifically, the extracted cardiac mesh is used as the shape prior to train the neural network model to more accurately describe the cardiac anatomical features of different individuals, providing a morphological standard for ultrasound section positioning and three-dimensional cardiac reconstruction based on multi-section two-dimensional ultrasound. During the model training process, binary classification loss and smooth regularization loss are used for constraint. Specifically, the binary classification loss function is used to optimize the occupancy field values, enabling the neural network model to accurately predict the belonging of points, while the smooth regularization loss is used to control the model gradient, making the generated three-dimensional shape smooth and conform to the morphological constraints of the ultrasound image segmentation mask. The binary classification loss function (1) and the smooth regularization loss function (2) are respectively:
[0047]
[0048] where x iare the point coordinates in the input three-dimensional space, c i is the binary classification label of the point (taking values of +1 or -1, indicating whether the point belongs to the target area), the occupancy field value output by the neural network, Φ θ (x i ) represents the predicted value of the model for the attribution of point x i belonging to, is the binary classification loss function, is the smooth regularization loss function, λ is the regularization coefficient, controlling the weight of the smooth loss in the total loss, is the absolute value of the gradient of the occupancy field Φ at point x (used to reflect the degree of change of the field), α is the gradient threshold.
[0049] Among them, the three-dimensional heart structure model includes a three-dimensional heart shape representation model, an ultrasonic section position estimation model, and an iterative optimization model. The three-dimensional heart shape representation model is used to output an occupancy scene to construct a continuous and differentiable implicit heart representation. The ultrasonic section position estimation model is used to determine the position of the ultrasonic section in the three-dimensional space and generate a plane point set to achieve accurate positioning of the ultrasonic image in the three-dimensional space. The iterative optimization model is used to alternately optimize the position and three-dimensional shape of the ultrasonic section, so that the reconstructed model gradually approaches the target heart shape, thereby enhancing the adaptability and generalization ability of the model to different individuals.
[0050] S2: Extract shape images at multiple positions from the heart mesh of the shape prior as ultrasonic segmentation masks, calculate the occupancy field value image based on the pre-constructed section coordinates, and compare the occupancy field value image with the ultrasonic segmentation masks to determine the ultrasonic section position.
[0051] In the embodiment of the present application, in order to construct the correspondence between the ultrasonic image and the three-dimensional heart shape, the ultrasonic section position estimation model is used to determine the position of the ultrasonic section in the three-dimensional space based on the heart mesh and the section coordinates, and generate a plane point set.
[0052] Please refer to Figure 5 , Figure 5 shows a specific implementation manner of step S2, which is described in detail as follows:
[0053] S21: Extract shape images at multiple positions from the heart mesh of the shape prior as the ultrasonic segmentation masks, where the shape images include two-chamber view, three-chamber view, four-chamber view, short-axis view of the heart base, short-axis view of the papillary muscle, and short-axis view of the mitral valve.
[0054] S22: Input the pre-constructed section coordinates into the three-dimensional heart shape representation model to calculate the occupancy field value image corresponding to the section coordinates.
[0055] S23: Compare the occupancy field value image with the ultrasound segmentation mask to determine the ultrasound slice position.
[0056] Specifically, shape images at the positions of the two-chamber view (2CH), three-chamber view (3CH), four-chamber view (4CH), short-axis view at the base of the heart (PSAX), short-axis view at the papillary muscle (PMAX), and short-axis view of the mitral valve (MVAX) are extracted from the cardiac mesh, and the shape images are used as the ultrasound segmentation mask. Then, the section coordinates are input into the three-dimensional cardiac shape representation model to calculate the corresponding occupancy field value image, which is compared with the ultrasound segmentation mask, and the model parameters are continuously optimized to determine the exact position of the ultrasound slice. Among them, the position of the ultrasound image in the three-dimensional space is controlled by the parameters inside the model, including the above six independent section position parameters (respectively controlling the spatial distribution of the six ultrasound slices), and a global scaling factor (used to adjust the overall size of all slices). Through this method, it is ensured that the segmentation mask of the ultrasound image can accurately match the three-dimensional cardiac model, improving the accuracy and stability of morphological reconstruction.
[0057] Among them, the section coordinates refer to the geometric definition parameters of the ultrasound section in the three-dimensional space, specifically including: the center point coordinate P0: the position of the ultrasound section in the three-dimensional space (such as the center point of the four-chamber view of the heart). The normal vector n: defines the spatial orientation of the ultrasound section (such as the direction perpendicular to the section). The basis vectors t1, t2: orthogonal direction vectors for constructing a local coordinate system within the section plane, used to expand the two-dimensional plane. The scaling factor s: controls the size of the area covered by the section (such as adapting to different sizes of the ventricles or atria). The offset parameters o1, o2: adjust the offset of the section within the local coordinate system to adapt to anatomical position changes. These parameters jointly define the position, orientation, and scale of the ultrasound section in the three-dimensional space.
[0058] Please refer to Figure 6 and Figure 7 , Figure 6 which shows a specific implementation manner of step S22, Figure 7 is a schematic diagram of the three-dimensional point set generation process provided by the embodiment of the present application, which is described in detail as follows:
[0059] S221: Calculate two orthogonal basis vectors according to two preset normal vectors.
[0060] S222: Uniformly sample points within a preset two-dimensional plane region to generate a grid point set.
[0061] S223: Adjust the plane point set in the grid point set using the scaling factor to obtain an adjusted plane point set.
[0062] S224: Map the adjusted planar point set to three-dimensional space based on the central point coordinates and the two base vectors to generate the target occupancy field value for each point, and generate the occupancy field value image for each cross-section based on the target occupancy field value.
[0063] As Figure 7 shown, first, two orthogonal base vectors t1 and t2 are calculated according to the given normal vector, thereby establishing a local coordinate system on the plane. These two base vectors t1 and t2 are respectively in the perpendicular direction of the normal vector, providing a reference direction for the subsequent generation of points. Points are uniformly sampled in the two-dimensional plane region of [-1, 1] × [-1, 1] to form a regular grid point set. This sampled point set is used to define the relative coordinates on the plane, and the value range of each point is between [-1, 1] to ensure that the entire plane region is fully covered. Then, the sampled planar point set is adjusted using a scaling factor to adapt to planar regions of different scales. The role of scaling is to control the distribution range of the planar points. For example, by reducing the value range of the point set, it can be adapted to different-sized cardiac anatomical regions. Finally, based on the central point coordinates P0 and the two base vectors t1 and t2, the points on the two-dimensional plane are mapped to three-dimensional space to generate the target occupancy field value for each point, and the occupancy field value image for each cross-section is generated based on the target occupancy field value. Among them, the cross-section coordinates are the respective planar coordinates in the adjusted planar point set.
[0064] Specifically, for any point (x, y) in the two-dimensional plane, its corresponding three-dimensional coordinates are calculated by the following formula:
[0065] P = P0 + s·(x·t1 + y·t2) + o1·t1 + o2·t2.
[0066] In the embodiment of the present application, the regular grid points originally on the two-dimensional plane are successfully mapped to three-dimensional space, thereby constructing a three-dimensional planar point set that conforms to the given normal vector constraint, providing a basis for subsequent calculations, morphological analysis, and cardiac reconstruction.
[0067] S3: Alternately optimize the ultrasound slice position and the cardiac morphological information in an alternately iterative optimization manner to obtain updated model parameters and cross-section position parameters.
[0068] Specifically, an iterative optimization model is used to alternately optimize the ultrasound slice position and the cardiac morphological information, enabling the reconstruction model to gradually approach the target cardiac morphology, thereby enhancing the adaptability and generalization ability of the model to different individuals. Among them, the cardiac morphological information refers to the three-dimensional shape of the heart.
[0069] Specifically, an iterative optimization mechanism that alternates between ultrasonic section position estimation and cardiac morphology information optimization is adopted to dynamically adjust the position of the ultrasonic section during the training process and optimize the morphological modeling of the implicit neural network, so that the occupancy field value image and the segmentation mask at the corresponding position of the ultrasonic section are fitted to ensure that the reconstructed model approximates the real cardiac anatomical structure.
[0070] In a specific embodiment, step S3 includes: using the ultrasonic section position as the spatial input of the three-dimensional cardiac shape representation model to optimize the cardiac morphology information, and reversely adjusting the ultrasonic section position based on the optimization result of the cardiac morphology information to alternately optimize the ultrasonic section position and the cardiac morphology information, so as to obtain the updated model parameters and section position parameters.
[0071] Specifically, accurate spatial input is provided for the three-dimensional cardiac shape representation model through ultrasonic section position estimation. Subsequently, based on the result of morphological optimization, the position of the ultrasonic section is reversely adjusted to form a closed-loop optimization mechanism, gradually reducing the morphological deviation. As the optimization progresses, the model can adaptively adjust the position and shape of the ultrasonic image to more accurately fit the real cardiac anatomical morphology and improve the adaptability and generalization ability for different individuals. During the continuous optimization and update process, the model parameters and section position parameters of the iterative optimization model are continuously optimized, thereby obtaining the updated model parameters and section position parameters.
[0072] S4: Construct a triangular mesh based on the updated model parameters and section position parameters to generate a target three-dimensional cardiac model.
[0073] Please refer to Figure 8 and Figure 9 , Figure 8 which shows a specific implementation manner of step S4, Figure 9 is a schematic diagram of the three-dimensional cardiac reconstruction results of different sections provided by the embodiment of the present application, which is described in detail as follows:
[0074] S41: Use the Marching Cubes algorithm to extract the isosurface of the implicit surface based on the updated model parameters and section position parameters.
[0075] S42: Construct the triangular mesh based on the isosurface of the implicit surface and generate the target three-dimensional cardiac model based on the triangular mesh.
[0076] Specifically, after the above-mentioned alternating iterative optimization, the Marching Cubes algorithm is used to extract the isosurface of the implicit surface based on the updated model parameters and section position parameters. A triangular mesh is constructed based on the isosurface of the implicit surface, and a target three-dimensional cardiac model is generated based on the triangular mesh.
[0077] AsFigure 9 As shown Figure 9 The reconstruction results of the left ventricular endocardium, right ventricular endocardium, and left and right ventricular epicardiums are shown. The color of the points represents the shortest distance to the gold standard. The bluer the color, the smaller the deviation and the more accurate the reconstruction result, while the redder the color indicates a larger deviation. In the embodiments of the present application, even when relying on only a small number of sections, a relatively accurate three-dimensional heart structure can still be generated. And as the number of sections increases, the reconstruction accuracy is further improved, and finally the heart anatomical structure can be restored more completely. To quantitatively evaluate the accuracy of the three-dimensional reconstruction, the embodiments of the present application use 3D IoU (Intersection over Union), Average Shortest Distance (ASD), and Volume Error for evaluation. 3D IoU mainly measures the regional overlap between the predicted heart structure and the gold standard. The Average Shortest Distance measures the average distance of the closest point pairs between the predicted mesh and the gold standard mesh, and the Volume Error is used to evaluate the volume deviation of the predicted heart structure.
[0078] There are a total of four combinations of sections, namely the two-chamber view and four-chamber view (2CH&4CH); the two-chamber view, three-chamber view, and four-chamber view (3AP); the four-chamber view, short-axis view at the base of the heart, short-axis view of the papillary muscle, and short-axis view of the mitral valve (4CH&3SAX); and all six sections (ALL6); and when reconstructing using the six-section combination of the two-chamber view (2CH), three-chamber view (3CH), four-chamber view (4CH), short-axis view at the base of the heart (PSAX), short-axis view of the papillary muscle (PMAX), and short-axis view of the mitral valve (MVAX), the 3D IoU of the left ventricle reaches 89.48%, and the 3D IoU of the right ventricle reaches 84.11%. At the same time, the volume error of the left ventricle is only 2.45 ml, and the volume error of the right ventricle is 6.43 ml, indicating that the embodiments of the present application can effectively improve the reconstruction accuracy of the three-dimensional heart structure. The specific reconstruction results are shown in Tables 1 and 2 below:
[0079]
[0080]
[0081] Table 1
[0082]
[0083] Table 2
[0084] In the embodiments of the present application, the three-dimensional space coordinates to be modeled are obtained, and the three-dimensional space coordinates to be modeled are mapped to a pre-constructed three-dimensional heart shape representation model to calculate the occupancy field value. Among them, the three-dimensional heart shape representation model is a neural network model trained based on shape priors, and the occupancy field value is used to represent heart morphological information; shape images at multiple positions are extracted from the heart mesh of the shape prior as ultrasound segmentation masks, the occupancy field value image is calculated based on the pre-constructed section coordinates, and the occupancy field value image is compared with the ultrasound segmentation mask to determine the ultrasound slice position; the ultrasound slice position and the heart morphological information are alternately optimized in an alternating iterative optimization manner to obtain updated model parameters and section position parameters; a triangular mesh is constructed based on the updated model parameters and section position parameters to generate a target three-dimensional heart model. The embodiments of the present invention generate heart morphological information based on shape priors, determine the ultrasound slice position, and alternately optimize the ultrasound slice position and the heart morphological information in an alternating iterative optimization manner. After the optimization is completed, a target three-dimensional heart model is constructed, which solves the problem that multi-section ultrasound images are difficult to accurately fuse, and is beneficial to improving the accuracy and applicability of three-dimensional heart reconstruction.
[0085] The embodiments of the present application use an implicit neural network to represent the three-dimensional structure of the heart, combine an ultrasound section position estimation module to determine the spatial position of the ultrasound plane, and gradually approximate the true three-dimensional heart morphology based on an alternating iterative mechanism of position estimation and three-dimensional shape optimization. Finally, a three-dimensional grid is extracted from the implicit neural network through the Marching Cubes algorithm to achieve the reconstruction of the complete heart structure. The embodiments of the present application not only effectively solve the problem of spatial registration of three-dimensional ultrasound sections, but also provide an efficient and accurate solution for three-dimensional heart reconstruction based on sparse ultrasound sections. The embodiments of the present application are aimed at the three-dimensional reconstruction task of sparse ultrasound segmentation masks, realizing the accurate registration of ultrasound sections and the high-precision three-dimensional reconstruction of the heart structure, being able to effectively calculate the three-dimensional morphology and volume information of the heart, while ensuring a small error. This method provides important technical support for clinical heart evaluation and surgical planning through the spatial positioning and three-dimensional structure reconstruction of ultrasound images.
[0086] Please refer to Figure 10 , as an implementation of the method shown above Figure 1 , the present application provides an embodiment of a three-dimensional heart reconstruction device based on multi-section two-dimensional ultrasound. This device embodiment corresponds to the method embodiment shown in Figure 1 , and this device can be specifically applied to various electronic devices.
[0087] As Figure 10As shown in the figure, the three-dimensional cardiac reconstruction device based on multi-plane two-dimensional ultrasound in this embodiment includes: a three-dimensional cardiac shape representation module 51, an ultrasound section position estimation module 52, an iterative optimization module 53, and a three-dimensional cardiac model generation module 54, where:
[0088] The three-dimensional cardiac shape representation module 51 is configured to obtain the three-dimensional spatial coordinates to be modeled, map the three-dimensional spatial coordinates to be modeled into a pre-constructed three-dimensional cardiac shape representation model to calculate the occupancy field value, where the three-dimensional cardiac shape representation model is a neural network model trained based on shape priors, and the occupancy field value is used to represent cardiac morphology information;
[0089] The ultrasound section position estimation module 52 is configured to extract shape images at multiple positions from the cardiac mesh of the shape prior as ultrasound segmentation masks, calculate the occupancy field value image based on the pre-constructed section coordinates, and compare the occupancy field value image with the ultrasound segmentation masks to determine the ultrasound section position;
[0090] The iterative optimization module 53 is configured to alternately optimize the ultrasound section position and the cardiac morphology information in an alternating iterative optimization manner to obtain updated model parameters and section position parameters;
[0091] The three-dimensional cardiac model generation module 54 is configured to construct a triangular mesh based on the updated model parameters and section position parameters to generate a target three-dimensional cardiac model.
[0092] Furthermore, the three-dimensional cardiac shape representation module 51 includes:
[0093] The high-dimensional feature generation unit is configured to obtain the three-dimensional spatial coordinates to be modeled, and project the three-dimensional spatial coordinates to be modeled into a high-dimensional feature space by using Fourier feature mapping to generate high-dimensional features;
[0094] The occupancy field value calculation unit is configured to input the high-dimensional features into a multi-layer perceptron in the three-dimensional cardiac shape representation model for modeling to generate the occupancy field value, where the multi-layer perceptron is three independent multi-layer perceptrons, which are respectively used to represent the left ventricular endocardium, the right ventricular endocardium, and the left and right ventricular epicardiums.
[0095] Furthermore, before the three-dimensional cardiac shape representation module 51, there is also:
[0096] The model training unit is configured to use the cardiac mesh as the shape prior, and train a neural network model based on the shape prior by using a binary classification loss function and a smoothing regularization loss function to generate a three-dimensional cardiac structure model, where the three-dimensional cardiac structure model includes a three-dimensional cardiac shape representation model, an ultrasound section position estimation model, and an iterative optimization model.
[0097] Furthermore, the ultrasound section position estimation module 52 includes:
[0098] A mask generation unit, configured to extract shape images at multiple positions from the cardiac mesh of the shape prior as the ultrasound segmentation mask, where the shape images include two-chamber view, three-chamber view, four-chamber view, short-axis view at the base of the heart, short-axis view of the papillary muscles, and short-axis view of the mitral valve;
[0099] A coordinate input unit, configured to input the pre-constructed section coordinates into the three-dimensional cardiac shape representation model to calculate the occupancy field value image corresponding to the section coordinates;
[0100] A comparison unit, configured to compare the occupancy field value image with the ultrasound segmentation mask to determine the ultrasound slice position.
[0101] Furthermore, the coordinate input unit includes:
[0102] A base vector calculation unit, configured to calculate two orthogonal base vectors according to two preset normal vectors;
[0103] A grid point set generation unit, configured to uniformly sample points within a preset two-dimensional plane region to generate a grid point set;
[0104] A point set adjustment unit, configured to adjust the plane point set in the grid point set using a scaling factor to obtain an adjusted plane point set;
[0105] A mapping unit, configured to map the adjusted plane point set to three-dimensional space based on the center point coordinates and the two base vectors to generate the target occupancy field value for each point, and generate the occupancy field value image for each section based on the target occupancy field value.
[0106] Furthermore, the iterative optimization module 53 includes:
[0107] An alternating optimization unit, configured to use the ultrasound slice position as the spatial input of the three-dimensional cardiac shape representation model to optimize the cardiac morphology information, and reversely adjust the ultrasound slice position based on the optimization result of the cardiac morphology information to alternately optimize the ultrasound slice position and the cardiac morphology information, so as to obtain the updated model parameters and section position parameters.
[0108] Furthermore, the three-dimensional cardiac model generation module 54 includes:
[0109] An isosurface extraction unit, configured to extract the isosurface of the implicit surface based on the updated model parameters and section position parameters using the Marching Cubes algorithm;
[0110] A triangular mesh construction unit is used to construct the triangular mesh based on the isosurface of the implicit surface and generate the target three-dimensional heart model based on the triangular mesh.
[0111] To solve the above technical problems, an embodiment of the present application further provides a computer device. For details, please refer to Figure 11 , Figure 11 This is the basic structural block diagram of the computer device in this embodiment.
[0112] The computer device 6 includes a memory 61, a processor 62, and a network interface 63 that are communicatively connected to each other through a system bus. It should be noted that Figure 11 only the computer device 6 with three components, namely the memory 61, the processor 62, and the network interface 63, is shown in
[0113] However, it should be understood that it is not required to implement all the shown components, and more or fewer components can be alternatively implemented. Among them, those skilled in the art of the present technology can understand that a computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.
[0114] The memory 61 includes at least one type of readable storage medium, which includes flash memory, hard disk, multimedia card, card-type memory (such as SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disc, etc. In some embodiments, the memory 61 may be an internal storage unit of the computer device 6, such as the hard disk or memory of the computer device 6. In other embodiments, the memory 61 may also be an external storage device of the computer device 6, such as a plug-in hard disk, SmartMedia Card (SMC), Secure Digital (SD) card, Flash Card, etc. equipped on the computer device 6. Of course, the memory 61 may also include both the internal storage unit and the external storage device of the computer device 6. In this embodiment, the memory 61 is generally used to store the operating system and various application software installed on the computer device 6, such as the program code of the three-dimensional heart reconstruction method based on multi-plane two-dimensional ultrasound. In addition, the memory 61 may also be used to temporarily store various data that have been output or will be output.
[0115] In some embodiments, the processor 62 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. The processor 62 is generally used to control the overall operation of the computer device 6. In this embodiment, the processor 62 is used to run the program code stored in the memory 61 or process data, such as running the program code of the above three-dimensional heart reconstruction method based on multi-plane two-dimensional ultrasound to implement various embodiments of the three-dimensional heart reconstruction method based on multi-plane two-dimensional ultrasound.
[0116] The network interface 63 may include a wireless network interface or a wired network interface, and the network interface 63 is generally used to establish a communication connection between the computer device 6 and other electronic devices.
[0117] This application also provides another implementation manner, that is, to provide a computer-readable storage medium storing a computer program, and the computer program can be executed by at least one processor so that at least one processor executes the steps of a three-dimensional heart reconstruction method based on multi-plane two-dimensional ultrasound as described above.
[0118] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-described embodiment methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions for causing a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods of the various embodiments of the present application.
[0119] Obviously, the above-described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The accompanying drawings show the preferred embodiments of the present application, but do not limit the scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosed content of the present application more thorough and comprehensive. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or perform equivalent replacements on some of the technical features. Any equivalent structure directly or indirectly using the content of the specification and drawings of the present application in other related technical fields shall be within the scope of protection of the present application by the same token.
Claims
1. A three-dimensional cardiac reconstruction method based on multi-plane two-dimensional ultrasound, characterized in that, Including: Obtain the three-dimensional space coordinates to be modeled, and map the three-dimensional space coordinates to be modeled into a pre-constructed three-dimensional heart shape representation model to calculate the occupancy field value, where the three-dimensional heart shape representation model is a neural network model trained based on shape priors, and the occupancy field value is used to represent heart morphology information; Extract shape images at multiple positions from the heart mesh of the shape prior as ultrasound segmentation masks, calculate the occupancy field value image based on the pre-constructed section coordinates, and compare the occupancy field value image with the ultrasound segmentation mask to determine the ultrasound slice position; Adopt an alternating iterative optimization method to alternately optimize the ultrasound slice position and the heart morphology information to obtain updated model parameters and section position parameters; Based on the updated model parameters and section position parameters, construct a triangular mesh to generate a target three-dimensional heart model.
2. The three-dimensional cardiac reconstruction method based on multi-plane two-dimensional ultrasound according to claim 1, characterized in that, The obtaining the three-dimensional space coordinates to be modeled and mapping the three-dimensional space coordinates to be modeled into a three-dimensional heart shape representation model to calculate the occupancy field value includes: Obtain the three-dimensional space coordinates to be modeled, and project the three-dimensional space coordinates to be modeled into a high-dimensional feature space by using Fourier feature mapping to generate high-dimensional features; Input the high-dimensional features into a multi-layer perceptron in the three-dimensional heart shape representation model for modeling to generate the occupancy field value, where the multi-layer perceptron is three independent multi-layer perceptrons, which are respectively used to represent the left ventricular endocardium, the right ventricular endocardium, and the left and right ventricular epicardiums.
3. The three-dimensional cardiac reconstruction method based on multi-plane two-dimensional ultrasound according to claim 1, characterized in that Before obtaining the three-dimensional space coordinates to be modeled and mapping the three-dimensional space coordinates to be modeled into a three-dimensional heart shape representation model to calculate the occupancy field value, the method further includes: Use the heart mesh as the shape prior, and train a neural network model based on the shape prior by using a binary classification loss function and a smoothing regularization loss function to generate a three-dimensional heart structure model, where the three-dimensional heart structure model includes a three-dimensional heart shape representation model, an ultrasound section position estimation model, and an iterative optimization model.
4. The three-dimensional cardiac reconstruction method based on multi-plane two-dimensional ultrasound according to claim 1, characterized in that The extracting shape images at multiple positions from the heart mesh of the shape prior as ultrasound segmentation masks, calculating the occupancy field value image based on the pre-constructed section coordinates, and comparing the occupancy field value image with the ultrasound segmentation mask to determine the ultrasound slice position includes: Extract shape images at multiple positions from the heart mesh of the shape prior as the ultrasound segmentation masks, where the shape images include two-chamber view, three-chamber view, four-chamber view, short-axis view of the cardiac base, short-axis view of the papillary muscle, and short-axis view of the mitral valve; Input the pre-constructed section coordinates into the three-dimensional heart shape representation model to calculate the occupancy field value image corresponding to the section coordinates; Compare the occupancy field value image with the ultrasound segmentation mask to determine the ultrasound slice position.
5. The three-dimensional cardiac reconstruction method based on multi-plane two-dimensional ultrasound according to claim 4, wherein The inputting the pre-constructed section coordinates into the three-dimensional heart shape representation model to calculate the occupancy field value image corresponding to the section coordinates includes: Calculate two orthogonal basis vectors according to two preset normal vectors; Uniformly sample points within a preset two-dimensional planar region to generate a grid point set; Use a scaling factor to adjust the planar point set in the grid point set to obtain an adjusted planar point set; Map the adjusted planar point set to three-dimensional space based on the center point coordinates and two of the basis vectors to generate the target occupancy field value for each point, and generate the occupancy field value image for each cross-section based on the target occupancy field value.
6. The three-dimensional cardiac reconstruction method based on multi-plane two-dimensional ultrasound according to any one of claims 1 to 5, characterized in that The ultrasound slice position and the cardiac morphology information are alternately optimized by using an alternating iterative optimization method to obtain updated model parameters and cross-section position parameters, including: Use the ultrasound slice position as the spatial input of the three-dimensional cardiac shape representation model to optimize the cardiac morphology information, and reversely adjust the ultrasound slice position based on the optimization result of the cardiac morphology information to alternately optimize the ultrasound slice position and the cardiac morphology information to obtain the updated model parameters and cross-section position parameters.
7. The three-dimensional cardiac reconstruction method based on multi-plane two-dimensional ultrasound according to any one of claims 1 to 5, characterized in that Based on the updated model parameters and cross-section position parameters, perform triangular mesh construction to generate a target three-dimensional cardiac model, including: Use the Marching Cubes algorithm to extract the isosurface of the implicit surface based on the updated model parameters and cross-section position parameters; Construct the triangular mesh based on the isosurface of the implicit surface and generate the target three-dimensional cardiac model based on the triangular mesh.
8. A three-dimensional cardiac reconstruction device based on multi-plane two-dimensional ultrasound, characterized in that, Including: A three-dimensional cardiac shape representation module for obtaining the three-dimensional spatial coordinates to be modeled, mapping the three-dimensional spatial coordinates to be modeled into a pre-constructed three-dimensional cardiac shape representation model to calculate the occupancy field value, where the three-dimensional cardiac shape representation model is a neural network model trained based on shape priors, and the occupancy field value is used to represent cardiac morphology information; An ultrasound cross-section position estimation module for extracting shape images at multiple positions from the cardiac grid of the shape prior as ultrasound segmentation masks, calculating the occupancy field value image based on the pre-constructed cross-section coordinates, and comparing the occupancy field value image with the ultrasound segmentation masks to determine the ultrasound slice position; An iterative optimization module for alternately optimizing the ultrasound slice position and the cardiac morphology information by using an alternating iterative optimization method to obtain updated model parameters and cross-section position parameters; A three-dimensional cardiac model generation module for performing triangular mesh construction based on the updated model parameters and cross-section position parameters to generate a target three-dimensional cardiac model.
9. A computer device, characterized in that, Including a memory and a processor, where a computer program is stored in the memory, and when the processor executes the computer program, it implements the three-dimensional cardiac reconstruction method based on multi-section two-dimensional ultrasound according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, and when the computer program is executed by the processor, it implements the three-dimensional cardiac reconstruction method based on multi-section two-dimensional ultrasound according to any one of claims 1 to 7.